learning - Academic Conferences Limited
learning - Academic Conferences Limited
learning - Academic Conferences Limited
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Proceedings<br />
of the<br />
10 th European Conference<br />
on e-Learning<br />
Brighton Business School<br />
University of Brighton<br />
UK<br />
10-11 November 2011<br />
Volume One<br />
Edited by<br />
Sue Greener and Asher Rospigliosi<br />
University of Brighton<br />
UK
Copyright The Authors, 2011. All Rights Reserved.<br />
No reproduction, copy or transmission may be made without written permission from the individual authors.<br />
Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper<br />
abstracts were read and selected by the conference panel for submission as possible papers for the<br />
conference.<br />
Many thanks to the reviewers who helped ensure the quality of the full papers.<br />
These Conference Proceeding have been submitted to the Thomson ISI for indexing.<br />
Further copies of this book can be purchased from<br />
http://www.academic-bookshop.com<br />
ISBN: 978-1-908272-22-5 Book<br />
Published by <strong>Academic</strong> Publishing <strong>Limited</strong><br />
Reading<br />
UK<br />
44-118-972-4148<br />
www.academic-publishing.org
Contents<br />
Paper Title Author(s) Page<br />
No.<br />
Preface xii<br />
Biographies of Conference Chairs, Programme<br />
Chair, Keynote Speaker and Mini-track Chairs<br />
Biographies of Contributing authors xv<br />
Revisiting the Personal Transferable Skills<br />
Debate - an eLearning Pedagogical Perspective<br />
Survey of Teachers’ use of Computer/Internet in<br />
Secondary Schools in South West Nigeria<br />
Issues and Challenges in Implementing<br />
eLearning Projects in Higher Education: The<br />
Case of Jordan<br />
The use of Open Educational Resources in Intra-<br />
Organisational eLearning and Continuing<br />
Education<br />
Constructing a Survey Instrument for Assessing<br />
Characteristics of Effective Online Teachers<br />
i<br />
Samuel Adu Gyamfi, Lene Tolstrup<br />
Sorenson and Thomas Ryberg<br />
Babatunde Alabi Alege and Stephen<br />
Olufemi Afolabi<br />
Hussein Al-Yaseen, Saheer Al-Jaghoub<br />
and Nidal Al-Salhi<br />
Antonios Andreatos<br />
xiii<br />
Jonathan Barkand 34<br />
When Agents Make Suggestions About Readings Orlando Belo 41<br />
Some Reflections on the Evaluation of Virtual<br />
Learning Environments<br />
Designing A New Curriculum: Finding The Right<br />
Blend<br />
Critical Success Factors for the Adoption of<br />
eLearning in the Kingdom of Saudi Arabia<br />
Educational Institutions<br />
Challenges in Developing e-Submission Policy<br />
and Practice<br />
Enhancement of e-Testing Possibilities With the<br />
Elements of Interactivity Reflecting the Students’<br />
Attitude to Electronic Testing<br />
e-Assessment Using Digital Pens – a Pilot Study<br />
to Improve Feedback and Assessment Processes<br />
Digital Educational Resources Repositories in<br />
Lower and Middle Education in Portugal: Quality<br />
Criteria in the International Context<br />
Nabil Ben Abdallah and Françoise Poyet 48<br />
Andrea Benn 56<br />
Latefa Bin Fryan and Lampros Stergioulas 63<br />
Alice Bird 73<br />
Martin Cápay, Martin Magdin and<br />
Miroslava Mesárošová<br />
Tim Cappelli 91<br />
Cornélia Castro, Sérgio André Ferreira and<br />
António Andrade<br />
eLearning: Roles in Distance Tertiary Education Ivana Cechova, Dana Zerzanova and Jana<br />
Berankova<br />
Independent Learning in Need or in Crisis?<br />
Independent Learning Under the new Four-Year<br />
Undergraduate Curriculum in Hong Kong<br />
The Development and Application of a web<br />
Based Metacognitive Mapping Tool<br />
An Exploratory Comparative Study of Distance-<br />
Learning Programmes<br />
The Optimal Teaching Style Based on Variability<br />
of Study Materials<br />
Yin Ha Vivian Chan, Delian Dawn Gaskell,<br />
Mei Ah Tan and Lip Yan Felix Chao<br />
1<br />
8<br />
16<br />
23<br />
82<br />
100<br />
109<br />
117<br />
Serdar Çiftci and Mehmet Akif Ocak 124<br />
Marija Cubric, Karen Clark and Mariana<br />
Lilley<br />
Blanka Czeczotková, Kateřina<br />
Kostolányová and Jana Šarmanová<br />
134<br />
145
Paper Title Author(s) Page<br />
No.<br />
Changing <strong>Academic</strong>s, Changing Curriculum: How<br />
Technology Enhanced Curriculum Design can<br />
Deliver Strategic Change<br />
Web Conferencing for us, by us and About us –<br />
the Leeds Met Elluminate User Group<br />
Tools for Evaluating Students’ Work in an<br />
Interactive (Open) Virtual Space: Case Study of<br />
an eLearning Course in an International Network<br />
of Universities<br />
Putting Things in Context - Designing Social<br />
Media for Education<br />
ii<br />
Christine Davies 152<br />
Mark de Groot, Gill Harrison and Rob Shaw 156<br />
Jana Dlouhá, Martin Zahradník, Jiří Dlouhý<br />
and Andrew Barton<br />
Jon Dron, Terry Anderson and George<br />
Siemens<br />
Experimental Assessment of Virtual Students Michaela Drozdová, Ondřej Takács and<br />
Jana Šarmanová<br />
Priming for Modules: A Case Study Evaluation of<br />
‘Pre-Workshop’ Online Resources for an<br />
Executive MBA Course<br />
Computer-Mediated Reading and its Impact on<br />
Learners’ Reading Comprehension Skills<br />
Do you see What I see? - Understanding the<br />
Challenges of Colour-Blindness in Online<br />
Learning<br />
Researching in the Open: How a Networked<br />
Learning Instance can Challenge Ethical<br />
Decision-Making<br />
Making Constraints and Decisions Explicit to<br />
Support Project-Based Collaborative Learning<br />
A Strategy for the Inductive Generation of<br />
Learning Objects in Low-Tech Contexts<br />
Cognitive Communication 2.0 in the Classroom –<br />
Resonance of an Experience in Higher Education<br />
To What Extent Does a Digital Audio Feedback<br />
Strategy Support Large Cohorts?<br />
Messages of Support: Using Mobile Technologies<br />
to Support the Transition of Students on<br />
Articulation Routes From Higher National Level to<br />
Degree<br />
Blended Learning at the Alpen-Adria-Universität<br />
Klagenfurt<br />
Evaluating the use of Social Networking Sites as<br />
a Tool for Knowledge Sharing for Developing<br />
Higher Education in Developing Countries: An<br />
Exploratory Study of Egypt and Iraq<br />
The Relationship Between Mindful Learning<br />
Processes and Course Outcomes in Web-Based<br />
Learning<br />
Researching and Sharing – Business School<br />
Students Creating a Wiki Glossary<br />
Glenn Duckworth<br />
Francisco Perlas Dumanig, Maya Khemlani<br />
David and Rodney Jubilado<br />
Colin Egan, Amanda Jefferies, Edmund<br />
Dipple and David Smith<br />
166<br />
177<br />
186<br />
195<br />
203<br />
210<br />
Antonella Esposito 218<br />
Gert Faustmann 225<br />
Ana Mª Fernández-Pampillón, Elena<br />
Domínguez, José Mª Lahoz, Dolores<br />
Romero, Isabel de Armas, Susana Palmaz<br />
and Jorge Arús<br />
Sérgio André Ferreira, Cornélia Castro and<br />
António Andrade<br />
235<br />
246<br />
Rachel Fitzgerald 256<br />
Julia Fotheringham and Emily Alder<br />
266<br />
Gabriele Frankl and Sofie Bitter 274<br />
Elaine Garcia, Ibrahim Elbeltagi, Sawasn<br />
Al-Husseini and Ahmed Abdelkader<br />
284<br />
Danny Glick and Roni Aviram 295<br />
Andrea Gorra and Ollie Jones 303
Paper Title Author(s) Page<br />
No.<br />
A Qualitative Evaluation of <strong>Academic</strong> Staff’s<br />
Perceptions of Second Life as a Teaching Tool<br />
Introducing and Using Electronic Voting Systems<br />
in a Large Scale Project With Undergraduate<br />
Students: Reflecting on the Challenges and<br />
Successes<br />
A Methodology for Incorporating Usability and<br />
Accessibility Evaluations in Higher Education<br />
The Virtual Learning Environment - Directions for<br />
Development in Secondary Education<br />
Mutlimodal Teaching Through ICT Education: An<br />
e-Twinning Program as a Case Study of<br />
Intercultural Exchange<br />
Effectiveness and Learners’ Evaluation of<br />
Combining Audio and Written Online Formative<br />
Feedback for Language Learning<br />
iii<br />
Rose Heaney and Megan Anne Arroll 311<br />
Amanda Jefferies 319<br />
Anne Jelfs and Chetz Colwell 326<br />
John Jessel 332<br />
Paraskevi Kanari and Georgios Potamias<br />
340<br />
Rosario Kane-Iturrioz 345<br />
Model of eLearning Project Evaluation Jana Kapounova, Jana Sarmanova and<br />
Marketa Dvorackova<br />
Bridging the Gap – From Teacher to eTeacher Andrea Kelz 363<br />
Open Courses: The Next big Thing in eLearning? Kaido Kikkas, Mart Laanpere and Hans<br />
Põldoja<br />
Using a Social Networking Environment to<br />
Facilitate Transition Into Higher Education<br />
Evaluation of the Quality of Learning Scenarios<br />
and Their Suitability to Particular Learners’<br />
Profiles<br />
Models of eLearning: The Development of a<br />
Learner-Directed Adaptive eLearning System<br />
Can eLearning Enhance Practice-Based Design<br />
Courses?<br />
Sophisticated Usability Evaluation of Digital<br />
Libraries<br />
Social Networks, eLearning and Internet Safety:<br />
Analysing the Stories of Students<br />
Learning Management Versus Classroom<br />
Management in Technology-Supported Blended<br />
Learning<br />
How to Represent a Frog That can be Dissected<br />
in a Virtual World<br />
Learning by Wandering: Towards a Framework<br />
for Transformative eLearning<br />
Online Student Engagement: Unfulfilled Promises<br />
or Promises Unfulfilled?<br />
355<br />
370<br />
John Knight and Rebecca Rochon 377<br />
Eugenijus Kurilovas, Inga Zilinskiene and<br />
Natalija Ignatova<br />
380<br />
Stella Lee, Trevor Barker, and Vive Kumar 390<br />
Jake Leith, Joanna Zara and Malcolm<br />
McInnes<br />
399<br />
Stephanie Linek and 408<br />
Birgy Lorenz, Kaido Kikkas and Mart<br />
Laanpere<br />
415<br />
Arno Louw 423<br />
Robert Lucas 434<br />
Marie Martin and Michaela Noakes 442<br />
Linda Martin, Gary Spolander, Imran Ali<br />
and Beulah Maas<br />
Personalized e-Feedback and ICT Maria-Jesus Martinez-Argüelles , Josep-<br />
Maria Batalla-Busquets, Patricia Noguera-<br />
Guerra and Ernest Pons-Fanals<br />
449<br />
456
Paper Title Author(s) Page<br />
No.<br />
Evaluation of Multimedia Tools and e-Feedback<br />
in Virtual Learning Environments<br />
iv<br />
Maria-Jesús Martínez-Argüelles, Marc<br />
Badia-Miro, Carolina Hintzmann and Dolors<br />
Plana-Erta<br />
Cyberbullying: A Workplace Virus David Mathew 473<br />
Learning in Smart Environments – From Here to<br />
There<br />
Using Courseware for More Than Courses: You<br />
May Already Hold the Lease on a Versatile Virtual<br />
Meeting Space<br />
An Analysis of Collaborative Learning as a<br />
Prevalent Instructional Strategy of South Africa<br />
Government eLearning Practices<br />
Ideas for Using Critical Incidents in Oral<br />
Debriefing From a Business Strategy Simulation<br />
Game<br />
Volume Two<br />
eNOSHA and Moodle – the Integration of two<br />
eLearning Systems<br />
CASE Learning to Structure and Analyze a Legal<br />
Decision<br />
A Framework for Decision Support for Learning<br />
Management Systems<br />
Learning for Life - Building Blocks to Holistic<br />
Education<br />
Student's Characteristics for Note Taking Activity<br />
in a Fully Online Course<br />
Freeing Education Within and Beyond <strong>Academic</strong><br />
Development<br />
Why Recording Lectures Requires a new<br />
Approach<br />
465<br />
Peter Mikulecky 479<br />
Karen Hughes Miller and Linda Leake 485<br />
Peter Mkhize, Magda Huisman and Sam<br />
Lubbe<br />
492<br />
Jonathan Moizer and Jonathan Lean 502<br />
Peter Mozelius, Isuru Balasooriya and<br />
Enosha Hettiarachchi<br />
509<br />
Antoinette Muntjewerff 517<br />
Phelim Murnion and Markus Helfert 526<br />
Shekhar Murthy and Devi Murthy 535<br />
Minoru Nakayama, Kouichi Mutsuura and<br />
Hiroh Yamamoto<br />
550<br />
Chrissi Nerantzi 558<br />
Paul Newbury, Phil Watten, Patrick Holroyd<br />
and Clare Hardman<br />
eSubmission – UK Policies, Practice and Support Barbara Newland, Lindsay Martin and Andy<br />
Ramsden<br />
Harnessing the Internet for Authentic Learning:<br />
Towards a new Higher Education Paradigm for<br />
the 21st Century<br />
Motivational Predictors of <strong>Academic</strong>s’ Electronic:<br />
Publishing in Nigerian Colleges of Education<br />
Psycho-Social Predictors of Students With<br />
Disabilities’ eLearning: Usage at the Federal<br />
College of Education (Special), Nigeria<br />
An Integrated Environment for Providing Learning<br />
Style Information in a Unified Manner<br />
Using Lifeworld-led Multimedia to Enhance<br />
Learning<br />
567<br />
578<br />
Abel Nyamapfene 586<br />
Maruff Akinwale Oladejo and Adelua<br />
Olajide Olawole<br />
Adelua Olajide Olawole and Maruff<br />
Akinwale Oladejo<br />
Fatemeh Orooji, Fattaneh Taghiyareh and<br />
Zahra Rahimi<br />
Andy Pulman, Kathleen Galvin, Maggie<br />
Hutchings, Les Todres, Anne Quinney,<br />
Caroline Ellis-Hill and Peter Atkins<br />
593<br />
601<br />
609<br />
620
The Project Mobile Game Based Learning Thomas Putz 628<br />
Using the Common Cartridge Profile to Enhance<br />
Learning Content Interoperability<br />
The Design and Development of an eLearning<br />
System Based on Social Networking<br />
Kansei Design Model for eLearning: A<br />
Preliminary Finding<br />
Changing Teacher Beliefs Through ICT:<br />
Comparing a Blended and Online Teacher<br />
Training Program<br />
Moodle and Affective Computing: Knowing who´s<br />
on the Other Side<br />
Using Google Applications to Facilitate an<br />
Effective Students’ Collaboration in the Teaching<br />
of Informatics to Students of Secondary<br />
Education<br />
Training Methods and Tools: Could eLearning be<br />
a Viable Solution to Solve SMEs Training<br />
Problems?<br />
Using Blended Learning to Develop Critical<br />
Reading Skills<br />
A Mobile aid Tool for Crafting Active Learning<br />
Experiences<br />
King-Sized eLearning - how Effective can an<br />
Online Approach be for Large Module Groups?<br />
v<br />
Ricardo Queirós and José Paulo Leal<br />
637<br />
Andrik Rampun and Trevor Barker 646<br />
Fauziah Redzuan, Anitawati Mohd<br />
Lokman, Zulaiha Ali Othman and Salha<br />
Abdullah<br />
Bart Rienties, Simon Lygo-Baker, Natasa<br />
Brouwer and Danielle Townsend<br />
Manuel Rodrigues, Florentino Fdez-<br />
Riverola and Paulo Novais<br />
658<br />
670<br />
678<br />
Eleni Rossiou and Erasmia Papadopoulou 686<br />
Andrée Roy 697<br />
Zuzana Šaffková 705<br />
Ahmed Salem 716<br />
Marie Sams, Mary Crossan and Kate<br />
Mottram<br />
Designing Effective Online Group Discussions Rowena Santiago, Amy Leh, and Minoru<br />
Nakayama<br />
The Game and the Alternating Roles of<br />
Learner/Teacher as Facilitators of the Learning<br />
Process in Organizations<br />
Implementing and Evaluating Problem-Based<br />
Virtual Learning Scenarios<br />
The Evolution of eLearning Platform TESYS User<br />
Preferences During the Training Processes<br />
Teachers’ Skills set for Personal Learning<br />
Environments<br />
Bridging the Feedback Divide Utilising Inclusive<br />
Technologies<br />
Post-<strong>Academic</strong> Masters Course in Management<br />
of Transfusion Medicine: Why the Difference in<br />
Access to the eLearning Between Countries?<br />
Engagement With Students in ‘Middle Ground’: A<br />
Flexible Learning Environment Allowing<br />
Simultaneous Access to Social Networking Sites<br />
and Formal <strong>Academic</strong> Space<br />
The Learning Management System as a Social<br />
Mediator: A Story With a Happy Ending<br />
Can the Medium Extend the Message? Using<br />
Technology to Support and Enhance Feedback<br />
Practices<br />
724<br />
731<br />
Vitor Santos and Luis Amaral 739<br />
Maggi Savin-Baden, Cathy Tombs and<br />
Katherine Wimpenny<br />
Adriana Schiopoiu Burlea, Amelia Badica<br />
and Carmen Radu<br />
Zaffar Ahmed Shaikh and Shakeel Ahmed<br />
Khoja<br />
746<br />
754<br />
762<br />
Angela Shapiro and Aidan Johnston 770<br />
Cees Th. Smit SibingaI 776<br />
Anne Smith and Sonya Campbell 780<br />
Dina Soeiro, António Dias de Figueiredo<br />
and Joaquim Armando Gomes Ferreira<br />
788<br />
Mekala Soosay 794
Implementation and Analysis of an Online,<br />
Student Centred Learning Environment to<br />
Support Personalised Study<br />
The Danger of the Downward Spiral: Teachers<br />
and Digital Literacy<br />
PeerWise - The Marmite of Veterinary Student<br />
Learning<br />
iSELF: An Internet-Tool for Self-Evaluation and<br />
Learner Feedback<br />
Using a Learning Management System for<br />
Executing Role Play Simulations<br />
The Effects of Self-Directed Learning Readiness<br />
on Learning Motivation in Web 2.0 Environments<br />
Usage Cases: A Useful way to Improve<br />
Effectiveness of eLearning web Based Platforms<br />
The Virtual Path to <strong>Academic</strong> Transition: Enabling<br />
International Students to Begin Their Transition to<br />
University Study Before They Arrive<br />
Identifying and Locating Frames of Reference to<br />
Inform the Design of Virtual Worlds in Higher<br />
Education<br />
Reaction Lecture: Text Messaging to Increase<br />
Student Engagement in Large-Scale Lectures<br />
A Holistic Approach to Instructional Design for<br />
Blended Learning Environments<br />
vi<br />
Iain Stewart, William McKee and Kevin<br />
Porteous<br />
802<br />
Caroline Stockman and Fred Truyen 811<br />
Amanda Sykes, Paul Denny and Lesley<br />
Nicolson<br />
820<br />
Nicolet Theunissen and Hester Stubbé 831<br />
Tone Vold 841<br />
Chien-hwa Wang and Cheng-ping Chen 846<br />
Cristina Wanzeller and Orlando Belo 854<br />
Julie Watson 862<br />
Katherine Wimpenny, Maggi Savin-Baden,<br />
Matt Mawer, Nicole Steils and Gemma<br />
Tombs<br />
Koos Winnips, Joost Heutink, and Hans<br />
Beldhuis<br />
870<br />
878<br />
Li Zhong Zhang 886<br />
PhD Papers 895<br />
Evaluating the Impact of an Arabic Version of an<br />
Adaptive Learning System Based on the Felder-<br />
Silverman’s Learning Style Instrument<br />
Negotiating Doctoral Practices and <strong>Academic</strong><br />
Identities Through the Adoption and Use of Social<br />
and Participative Media<br />
Enabling Disruptive Technologies for Higher<br />
Education Learning and Teaching<br />
Exploring the Potential of a Mobile Computer lab<br />
in a Developmental Context: The Teacher’s<br />
Perspective<br />
Collaborative eLearning in a Developing Country:<br />
A University Case Study in Uganda<br />
Applying the Multimedia Learning Theory in the<br />
Primary School: An Experimental Study About<br />
Learning Settings Using Digital Science Contents<br />
Designing a U-Learning Course Platform for the<br />
Identified Teacher Training Needs<br />
Nahla Aljojo, Carl Adams, Huda Saifuddin<br />
and Zainab Alsehaimi<br />
897<br />
Andy Coverdale 909<br />
Michael Flavin 917<br />
Fortunate Gunzo and Lorenzo Dalvit 925<br />
Evelyn Kigozi Kahiigi, Henrik Hansson,<br />
Mats Danielson, F.F Tusubira and Mikko<br />
Vesisenaho<br />
Fabio Serenelli, Enrico Ruggeri, Andrea<br />
Mangiatordi and Paolo Ferri<br />
932<br />
943<br />
Nazime Tuncay and Hüseyin Uzunboylu, 953<br />
Work In Progress Papers 971<br />
Someone to Talk to – Using Automated<br />
Characters to Support Simulated Learning<br />
Activities<br />
Liz Falconer and Manuel Frutos-Perez 973
Extreme Scaffolding in the Teaching and<br />
Learning of Programming Languages<br />
Benefits and Barriers: Applying eLearning in the<br />
Context of Organisational Change to Improve the<br />
Learning Experience for Mature, Part-Time<br />
Students<br />
Posters with Papers<br />
Investigating Student Engagement With an<br />
Electronically Delivered Simulation of<br />
Professional Practice<br />
Instrumental Distance Learning in Higher Music<br />
Education<br />
Reflections on <strong>Academic</strong> Blogging as a Vehicle<br />
for Professional Development<br />
A Framework for Understanding Online Learning<br />
Communities<br />
Trust in Distributed Personal Learning<br />
Environments: The Case Study of LePress PLE<br />
Breaking Down Barriers: Development of a Wiki<br />
Based Module to Enhance the International<br />
Learning Experience<br />
eLearning in German Higher Education:<br />
Technology Implementation as a Challenge for<br />
Organizational Change<br />
vii<br />
Dan-Adrian German 978<br />
Simon McGinnes<br />
982<br />
Olivia Billingham 989<br />
Karin Levinsen, Rikke Orngreen, Mie Buhl,<br />
Marianne Løkke Jakobsen and Jesper<br />
Andersen 1<br />
Peps Mccrea<br />
Sónia Sousa, David Ribeiro Lamas, José<br />
Braga de Vasconcelos and Ilya Shmorgun 1<br />
Sónia Sousa, David Ribeiro Lamas and<br />
Vladimir Tomberg<br />
Karen Strickland, Liz Adamson, Carolyn<br />
Blight and Wendy McInally<br />
Novita Yulianti, Michael Lund and Georg<br />
Müller-Christ<br />
993<br />
997<br />
1000<br />
1006<br />
1012<br />
1015
Conference Committee<br />
Conference Executive<br />
Sue Greener, Brighton Business School, University of Brighton, UK<br />
Asher Rospigliosi, Brighton Business School, University of Brighton, UK<br />
Andrea Benn, Brighton Business School, University of Brighton, UK<br />
Laurence Olver, Brighton Business School, University of Brighton, UK<br />
Stephanos Avakian, Brighton Business School, University of Brighton, UK<br />
Barbara Newland, Centre for Learning and Teaching, University of Brighton, UK<br />
Steve Kilgallon, School Of Environment and Technology, University of Brighton, UK<br />
Avril Loveless, School of Education, University of Brighton, UK<br />
Lyn Pemberton, School of Computing, Engineering and Maths., University of Brighton, UK<br />
Committee members<br />
The conference programme committee consists of key people in the e-<strong>learning</strong> community around the world.<br />
The following people have confirmed their participation:<br />
Ariffin Abdul Mutalib (Universiti Utara Malaysia, Malaysia); Siti aishah Abdullah (University Technology Mara,<br />
Kelantan, Malaysia); Tofan Cezarina Adina (Spiru Haret University, Romania); Wilfried Admiraal (Universiteit<br />
van Amsterdam, Netherlands); Shafqat Ali (University of Western Sydney, Australia); Abdallah Al-Zoubi<br />
(Princess Sumaya University for Technology, Jordan); Margarida Amaral (University of Porto, Portugal);<br />
Antonios Andreatos (Hellenic Air Force Academy, Greece); Alla Anohina (Riga Technical University, Latvia);<br />
Jane Ardus (Stevenson College,Edinburgh, UK); Mohamed Arteimi (7th of April University, Tripoli, Libya);<br />
William Ashraf (University of Sussex, UK); Bunyamin Atici (Firat University, Turkey); Anders Avdic (Orebro<br />
University, Sweden); Simon Bachelor (Gamos, Reading, UK); Joan Ballantine (University of Ulster, UK);<br />
Trevor Barker (University of Hertfordshire, UK); Josep-Maria Batalla (Universitat Oberta de Catalunya,<br />
Spain); Orlando Belo (University of Minho Campus de Gualtar, Portugal); David Benito (Public University of<br />
Navarre, Pamplona, Spain); Yongmei Bentley (University of Luton, UK); Daniel Biella (University of Duisburg-<br />
Essen, Germany); Radu Bilba (George Bacovia University,, Romania); Eric Bodger (University of<br />
Winchester, UK,); Stephen Bowman (Ravensbourne College of Design and Communication, UK); Willem-<br />
Paul Brinkman (Delft University of Technology, Netherlands); Ann Brown (CASS Business School, London,<br />
UK); Mark Brown (Massey University,Palmerston North, New Zealand); Giuseppe Cannavina (University of<br />
Sheffield, UK); Sven Carlsson (School of Economics and Management, Lund University, Sweden); James<br />
Carr (University of Newcastle, UK); Maggie Carson (Edinburgh University, UK); Antonio Cartelli (University of<br />
Cassino,, Italy); Rommert Casimir (Tilburg University, The Netherlands); Ivana Cechova (University of<br />
Defence, Czech Republic,); Maria Celentano (University of Lecce, Italy); Athina Chatzigavriil (LSE, London,);<br />
Satyadhyan Chickerur (M.S.Ramaiah Institute of Technology, Bagalore, India); Burhan China (PDSA,<br />
Somalia); Barbara Class (University of Geneva, Switzerland); Lynn Clouder (Coventry University, UK);<br />
Thomas Connolly (University of West of Scotland, UK); Ken Currie (Edinburgh University, UK); Valentina<br />
Dagiene (Institute of Mathematics and Informatics, Vilnius, Lithuania); Mark De Groot (Leeds Metropolitian<br />
University, UK); Antonio De Nicola (ENEA, Italy); Carmen De Pablos Heredero (Rey juan Carlos University,<br />
Spain); Rajiv Dharaskar (GH Raisoni College of Engineering, Nagpur, India); Vicenzo Di Lecce (Politecnico<br />
di Bari, Italy); Martina Doolan (University of Hertfordshire, UK); Christopher Douce (Institute of Educational<br />
Technology, Walton Hall, UK); Yanqing Duan (University of Luton, UK); Jane Eberle (Emporia State<br />
University, USA); Colin Egan (University of Hertfordshire, Hatfield, UK); Bulent Gursel Emiroglu (Eskisehir<br />
Yolu Baglica Mevkii, Turkey); Chew Esyin (University of Glamorgan, UK,); Ariwa Ezendu (London<br />
Metropolitan University, Uk); Bekim Fetaji (South East European University, Tetovo, Macedonia); Andrea<br />
Floros (Ionian University, Greece); Duncan Folley (Leeds Metropolitian University, England); Katie Goeman<br />
(Free University of Brussels (VUB), Belgium); Colin Gray (Edinburgh Napier University, Scotland); Susan<br />
Greener (University of Brighton, UK); David Guralnick (Kaleidoscope Learning, New York, USA); Richard<br />
Hall (De Monfort University, Leicester, UK); Patricia Harvey (Greenwich University, London, UK); Thanos<br />
Hatziapostolou (International faculty of the university of sheffield, Greece); Rose Heaney (University of East<br />
London, UK,); Alan Hilliard (University of Hertfordshire, Hatfield, UK); Uwe Hoppe (Bildungswerk der<br />
Sächsischen Wirtschaft gGmbH, Germany); Md. Fokhray Hossain (Daffodil International University,<br />
Bangladesh); Stefan Hrastinski (Uppsala University, Sweden); BALDE IDIATOU (NOBLE GROUP<br />
ORGANISED SOLUTIONS, GUINEA); Antonin Jancarik (Faculty of education, Charles University, Czech<br />
Republic); Amanda Jefferies (University of Hertfordshire, Hatfield, UK); Runa Jesmin (Global Heart Forum,<br />
UK); Aidan Johnston (Glasgow Caledonian University, UK); Paul Jones (University of Glamorgan, UK);<br />
Geraldine Jones (University of Bath, UK,); Jowati Juhary (National Defence University of Malaysia,<br />
viii
Malaysia); Tuomo Kakkonen (University of eastern Finland, Finland); Michail Kalogiannakis (School of<br />
Pegadogical and Technicological Education, ASPETE, Crete); Clifton Kandler (University of Greenwich,<br />
UK,); Jana Kapounova (University of Ostrava , Czech Republic); Andrea Kelz (University of Applied Sciences<br />
Burgenland,Campus Pinkafeld, Austria); Saba Khalil (Virtual University of Pakistan, Lahore, Pakistan);<br />
Jasna Kuljis (Brunel University, UK); Sunaina Kumar (Indira Gandhi National Open University, New Delhi,<br />
INDIA); Swapna Kumar (University of Florida, USA); venkata Durga kumar (Sunway University College,<br />
Malaysia,); Blair Kuntz (University of Toronto, Canada,); Eugenijus Kurilovas (Vilnius Gediminas technical<br />
university / institute of mathmatics and informatics of Vinius University, Lithuania); Eleni Kyza (Cyprus<br />
University of Technology, Lemesos, Cyprus); Maria Lambrou (University of the Aegean Business School,<br />
Greece); Andy Lapham (Thames Valley University, UK); Mona Laroussi (Institut National des Sciences<br />
Appliquées et de la Technologie, Tnis and Lille, Tunisia); Deepak Laxmi Narasimha (University of Malaya,<br />
Malaysia); Fotis Lazarinis (Applied Informatics in Management and Finance, Greece); Denise Leahy (Trinity<br />
College, Dublin, Ireland); Kate Lennon (Glasgow Caledonian University, UK); Mariana Lilley (University of<br />
Hertfordshire, UK); Jorgen Lindh (Jonkoping International Business School, Sweden); Gi-Zen Liu (National<br />
Cheng Kung University, Taiwan); Ying Liu (Cambridge University, UK); Jenny Lorimer (University of<br />
Hertfortshire, UK); Sam Lubbe (University of South Africa, South Africa); Nick Lund (Manchester<br />
Metropolitan University, England,); Alejandra Magana (Purdue University, United States of America, United<br />
States of America); Adnan Mahmood (University of Jinan, P.R.China,); Francis Maietta (Real Thinking<br />
Company, UK); Christina Mainka (Heidelberg University , Germany); Chittaranjan Mandal (School of IT,IIT<br />
Kharagpur, India); Augostino Marengo (University of Bari, Italy); Maria J Martinez-Arguelles (Universitat<br />
Oberta de Catalunya, Spain); Sephanos Mavromoustakos (Cyprus College, Cyprus); Erika Mechlova<br />
(University of Ostrava, Czech Republic); Cherifa Mehadji (University of Strasbourg, France); Rosina Merry<br />
(the school of Education The University of Waikatio, New Zealand); Linda Joy Mesh (Universita degli Studi di<br />
Siena, Italy); Jaroslava Mikulecka (University of Hradec Kralove, Czech Republic); Peter Mikulecky<br />
(University of Hradec Kralove, Czech Republic); Mike Mimirinis (Middlesex University, London, UK); Julia<br />
Mingullon (Universitat oberta de catalunya, Spain); Ali Moeini (University of Tehran, Iran); Johann Moller<br />
(University of South Africa (UNISA), South Africa,); Peter Monthienvichienchai (Insitute of Education,<br />
London, UK); Pam Moule (University of the West of England, Bristol, UK); Radouane Mrabet (ENSIA,<br />
Morocco); Minoru Nakayama (Tokoyo Institute of Technology, Japan); Julian Newman (Glasgow Caledonian<br />
University , UK); Chetsada Noknoi (Thaksin University, Songkhla, Thailand); Abel Nyamapfene (University of<br />
Exeter, United Kingdom); Sinead O’Neill (Waterford Institute of Technology, Ireland); Kamila Olsevicova<br />
(Univeristy of Hradec Kralove, Czech Republic); Rikke Orngreen (Aarhus University, Denmark); Jalil<br />
Othman, (University of Malaya ,Malaysia); Kutluk Ozguven (Dogus University, Turkey); Ecaterina Pacurar<br />
Giacomini (Louis Pasteur University, FRANCE); Alessandro Pagano (University of Bari, Italy); Vasileios<br />
Paliktzoglou (University of eastern Finland, Finland); Stefanie Panke (University of Ulm, Germany); George<br />
Papadopoulos (University of Cyprus, Cyprus); Iraklis Paraskakis (South East European Research Centre<br />
(SEERC) Research Centre of the University of Sheffiled, Thessaloniki, Greece); Vivien Paraskevi (TECFA -<br />
FPSE, Educational Technology Unit, University of Geneva, Switzerland, Switzerland); Angie Parker (Anthem<br />
College Online, USA,); Paul Peachey (University of Glamorgan, Treforest, UK); Arna Peretz (Ben Gurion<br />
Univeristy of the Negev, Omer, Israel); Christine Perry (University of the West of England, Bristol, UK);<br />
Donatella Persico (Istituto Tecnologie Didattiochje-Consiglio Nazionale Ricerche, Genova, Italy); Pit<br />
Pichappan (Annamalai University, India); Selwyn Piramuthu (University of Florida, Gainesville, USA); Michel<br />
Plaisent (University of Quebec in Montreal, Canada); Lubomir Popelinsky (Masaryk University, Czech<br />
Republic); Andy Pulman (Bournemouth University, UK); Muhammad Abdul Qadir (Mohammad Ali Jinnah<br />
University, Islamabad, Pakistan); Ricardo Queirós (ESEIG/KMILT & CRACS/INESC, Portugal); Susannah<br />
Quinsee (City University, London, UK); Abdul Rafay (Asia Pacific University College of Technology &<br />
Innovation, Malaysia); Liana Razmerita (Copenhagen Business School, Denmark); Christopher Reade<br />
(Kingston University, UK); Hugo Ribeiro (University of Porto, Portugal,); Vivien Rolfe (De Monfort University,<br />
Leicester, UK); Asher Rospigliosis (University of Brighton, UK); Florin Salajan (North Dakota State University<br />
, Canada); David Sammon (Univesity College Cork, Ireland); Gustavo Santos (University of Porto, Portugal);<br />
Venkat Sastry (Defence College of Management and Technology, Cranfield University, UK); Guy Saward<br />
(University of Hertfordshire, UK,); Brian Sayer (University of London, UK); Jeanne Schreurs (Hasselt<br />
University, Diepenbeek, belgium); Jane Secker (London School of Economics, UK); Angela Shapiro<br />
(Glasgow Caledonian University, Scotland); Aileen Sibbald (Napier University, Scotland, UK); Petia Sice<br />
(University of Northumbria, Newcastle-upon-Tyne, UK); Gurmeet Singh (The University of The South Pacific,<br />
Suva , Fiji, Fiji); Cees Th. Smit Sibinga (<strong>Academic</strong> insitute for the international development of transfusion<br />
medicine, The Neverlands); Alisdair Smithies (Manchester Medical School, UK); Keith Smyth (Napier<br />
University, Edinburgh, UK); Bent Soelberg (Copenhagen Business School, Denmark); Or Kan Soh<br />
(University Tunku Abdul Rahman (UTAR), Malaysia); Yeong-Tae Song (Towson University, Maryland, USA);<br />
Michael Sonntag (FIM, Johannes Kepler University, Linz, Austria); Rumen Stainov (University of Applied<br />
Sciences, Fulda, Germany); John Stav (Sor-Trondelag University College, Norway); Roxana Taddei<br />
(Université Clermont Ferrand 2, Montpellier, France); Yana Tainsh (University of Greenwich,, UK); Heiman<br />
Tali (The Open University, Israel); Bénédicte Talon (Université du Littoral, France); Marian Theron (False<br />
ix
Bay College, Tokai, South Africa); John Thompson (Buffalo State College, USA); Claudine Toffolon<br />
(Université du Mans - IUT de Laval, France); Eulalia Torras-Virgili (Open University of catalonia, Spain);<br />
Kathryn Trinder (Glasgow Caledonian University, UK); Christopher Turner (University of Winchester , UK);<br />
Karin Tweddell Levinsen (Danish University of Education, Denmark); Aimilia Tzanavari (University of Nicosia,<br />
Cyprus); Huseyin Uzunboylu (Near East University, CYPRUS); Linda Van Ryneveld (Tshwane University of<br />
Technology, Pretoria, South Africa); Carlos Vaz de Carvalho (Porto Polytechnic, Portugal); Andreas Veglis<br />
(Aristotle University of Thessaloniki, Greece); Bruno Warin (Université du Littoral, Calais, France); Fahad<br />
Waseem (University of Northumbria, Middlesbrough, UK); Garry Watkins (University of Central Lancashire,<br />
UK); Anne Wheeler (Aston University, UK); Steve Wheeler (Faculty of Education, University of Plymouth,<br />
UK); Nicola Whitton (Manchester Metropolitan University, UK); Roy Williams (University of Portsmouth, UK);<br />
Shirley Williams (University of Reading, UK); Rowena Yeats (University of Birmingham, UK); Panagiotis<br />
Zaharias (University of the Aegean, Greece); Mingming Zhou (Nanyang Technological University,<br />
Singapore); Chris Zielinski (External relations and Governing Bodies, World Health Organization, Geneva,<br />
Switzerland); Anna Zoakou (Ellinogermaniki Agogi, Greece);<br />
x
Preface<br />
These Proceedings represent the work of contributors to the 10th European Conference on e-Learning,<br />
ECEL 2011, hosted this year by Brighton Business School, University of Brighton, UK. The Conference Chair<br />
is Sue Greener, and the Programme Chair is Asher Rospigliosi, both from Brighton Business School, UK.<br />
The conference will open with a keynote address by Don Clark, former CEO of EPIC software and winner of<br />
the ‘Outstanding Achievement in e-<strong>learning</strong> Award’. Expect to be enlivened by his challenge: “Don't lecture<br />
me!, ..and why technology is only scalable solution”. Also on the first day we have Anne Boddington from the<br />
University of Brighton speaking on the topic of “Designing Education and Reshaping Learning”. The second<br />
day will be opened by Professor Gráinne Conole, Director of the Beyond Distance Research Alliance at the<br />
University of Leicester in the UK with a talk entitled “Trajectories of <strong>learning</strong> - new approaches and<br />
directions".<br />
With an initial submission of 239 abstracts, after the double blind, peer review process there are 108<br />
academic papers, 7 Phd Papers, 3 Work in Progress papers and 3 non academic papers in these<br />
Conference Proceedings. These papers reflect the truly global nature of research in the area with<br />
contributions from Australia, Austria, Belgium, Canada, China, Czech Republic, Denmark, Estonia, France,<br />
Germany, Greece, India, Iran, Ireland, Israel, Italy, Japan, Jordan, Lithuania, Malaysia, Nigeria, North<br />
Cyprus, Norway, Pakistan, Portugal, Romania, Saudi Arabia, Singapore, Slovakia, South Africa, Spain,<br />
Sweden, Taiwan, The Netherlands, Turkey, United Kingdom and the United States.<br />
A selection of papers – those agreed by a panel of reviewers and the editor will be published in a special<br />
conference edition of the EJEL (Electronic Journal of e-Learning www.ejel.org ).<br />
We know you will enjoy and be stimulated by this conference. We hope you will also revel in the uniqueness<br />
of Brighton.<br />
Sue Greener and Asher Rospigliosi<br />
November 2011<br />
xi
Biographies of Conference Chairs, Programme Chairs and<br />
Keynote Speakers<br />
Conference Chairs<br />
Dr Sue Greener is a University teacher: HRM, Business Context, Research Methods<br />
and Learning & Development and has received a Teaching Excellence award from the<br />
University of Brighton and is Programme Leader for the Foundation Degree in<br />
Business. Sue is also the Course Director: online final year undergraduate course with<br />
students in diverse world regions, her researcher interests are focused on e-<strong>learning</strong><br />
strategy, teacher development and reflective <strong>learning</strong>. Sue is the co-founder of the<br />
Business e-Learning Research Group and a member of the CROME research group<br />
on employment issues at Brighton Business School. Her Doctoral research focused<br />
on exploring students’ readiness for online <strong>learning</strong>. Sue holds a BA, MBA, EdD, FHEA and is a Chartered<br />
Fellow of CIPD.<br />
Programme Chair<br />
Asher Rospigliosi lecturers on e-commerce, management information systems, IS<br />
strategy, public sector IS and digital marketing at the University of Brighton. His<br />
research interests extend to e-<strong>learning</strong> and innovation in SMEs. Asher is a cofounder<br />
of the Business e-Learning Research Group and a member of the CROME<br />
research group on employment issues at Brighton Business School.<br />
Keynote Speakers<br />
Gráinne Conole is Professor of e-Learning at the Open University, with research<br />
interests in the use, integration and evaluation of Information and Communication<br />
Technologies and e-<strong>learning</strong> and impact on organisational change. She was<br />
previously chair of educational innovation at Southampton University and before that<br />
Director of the Institute for Learning and Research Technology at the University of<br />
Bristol. She has extensive research, development and project management<br />
experience across the educational and technical domains She serves on and chairs a<br />
number of national and international advisory boards, steering groups, committees<br />
and international conference programmes.<br />
Donald Clarke was CEO and one of the original founders of Epic Group plc, which<br />
established itself as the leading company in the UK e-<strong>learning</strong> market, floated on the<br />
Stock Market in 1996 and sold in 2005. Describing himself as ‘free from the tyranny of<br />
employment’, he is now a board member of Ufi LearnDirect (Government agency<br />
delivered e-<strong>learning</strong> to 2.8 million learners), Caspian Learning (<strong>learning</strong> games tool<br />
provider), LearningPool (content provider), Brighton Arts Festival, and a school<br />
governor. Donald has won many awards for the design and implementation of e<strong>learning</strong>,<br />
notably the ‘Outstanding Achievement in e-<strong>learning</strong> Award’.<br />
Anne Boddington is Dean of the Faculty of Arts at the University of Brighton.<br />
Educated and qualified as an Architect and subsequently as a Cultural Geographer,<br />
she currently leads a Faculty of .3,900 students and a portfolio that includes the Visual<br />
and Performing Arts, Architecture, Design, Media Studies, Literature, Languages and<br />
Humanities. She is also Co-director of the ADM HEA Subject Centre, which we are<br />
proud to host at the University. Anne is a Fellow of the Royal Society of Arts (RSA)<br />
and an affiliate member of the Royal Institute of British Architects (RIBA) and is an<br />
elected member of the executive for the Council for Higher Education in Art & Design<br />
(CHEAD). Her initial research interests were rooted in the design and development of<br />
the urban and cultural landscape and identity but have expanded to include the strategic design and<br />
development of <strong>learning</strong> and research space and its relationships to pedagogic practice and to educational<br />
strategies and governance.<br />
xii
Mini Track Chairs<br />
Dr Antonios Andreatos is a Professor at the Computer Engineering Division of the<br />
Hellenic Air Force Academy. He received the Diploma in Electrical Engineering from<br />
the Univ. of Patras, the M.S. in Computer Engineering from the Univ. of<br />
Massachusetts, the M.Ed in Adult Learning from the Hellenic Open Univ. and the<br />
Ph.D. in Computer Engineering from the National Technical Univ. of Athens.<br />
Research interests include e-Assessment, Active Learning methods, ICT & Web 2.0<br />
& Open Resources in Education, Didactics of Computer Engineering, etc. He has<br />
published over 60 papers in journals and conference proceedings and a book. He is<br />
also involved in the scientific committees of many conferences in his fields of interest.<br />
Orlando Belo is an associate professor in the Department of Informatics at Minho<br />
University, Portugal. His main research topics are related with data warehouse design,<br />
implementation and tuning, ETL services, database preferences, and distributed<br />
multidimensional structures processing. During the last few years he was involved with<br />
several projects in the decision support systems area designing and implementing<br />
computational platforms for specific applications like fraud detection and control in<br />
telecommunication systems, data quality evaluation, and ETL systems for industrial<br />
data warehousing systems. More recently, he was developing some research work<br />
establishing OLAP usage profiles and optimizing OLAP selection methods, applying some of the techniques<br />
studied on these areas over typical e-<strong>learning</strong> scenarios.<br />
Dr Colin Egan is a senior lecturer in the School of Computer Science, at the University<br />
of Hertfordshire. Colin has had an interest in accessibility and accessibility issues for a<br />
number of years and has presented his work to a range of International <strong>Conferences</strong>.<br />
Dr Amanda Jefferies is a Reader in Technology Enhanced<br />
Learning at the University of Hertfordshire. She is passionate about<br />
promoting a positive <strong>learning</strong> experience for students in HE through careful choice of<br />
technologies, to enhance their engagement with studies. She is well-known for<br />
innovative research into understanding the student experience through using student<br />
constructed reflective video and audio diaries, a technique she refined through her JISCsupported<br />
‘Learner Journeys’ STROLL project during 2007-2009. She has presented<br />
her research to international audiences across the UK, Europe and in North America.<br />
Dr Eugenijus Kurilovas is a Research Scientist in Vilnius University Institute of<br />
Mathematics and Informatics, an Associate Professor in Vilnius Gediminas Technical<br />
University, and a Head of International Networks Department in the Centre of<br />
Information Technologies in Education of the Ministry of Education and Science of<br />
Lithuania. He has published over 50 scientific papers, 2 books chapters and 2<br />
monographs, participated in many large scale international scientific projects. He is a<br />
member of 12 scientific committees of the international journals and conferences. His<br />
papers presented on ECEL-2008, 2009 and 2010 have been selected for publication in<br />
Electronic Journal of e-Learning.<br />
Dr Barbara Newland is a National Teaching Fellow with over 17 years experience of<br />
leading educational developments in Higher Education. She is a Principal Lecturer in the<br />
Centre for Learning and Teaching at the University of Brighton. Previously, Barbara was<br />
Blended Learning Co-ordinator at Glasgow Caledonian University, Manager of the<br />
Educational Development Services at Bournemouth University and the Learning<br />
Technology Team at Durham University.<br />
Dr Keith Smyth is a Senior Teaching Fellow and Senior Lecturer in Higher Education at<br />
Edinburgh Napier University, where he leads the MSc Blended and Online Education<br />
(www.napier.ac.uk/ed/boe). Keith is interested in how simple uses of technology can<br />
enhance <strong>learning</strong>, and where blended and online approaches can underpin community<br />
and work-based <strong>learning</strong>, broader skills development, and more inclusive educational<br />
practices.<br />
xiii
Biographies of contributing authors (in alphabetical<br />
order)<br />
Samuel Adu Gyamfi is currently a PhD Fellow at the centre for Communication, Media and Information<br />
technologies (CMI) at Aalborg University, Copenhagen. He is currently researching on developing the<br />
Personal Transferable Skills (PTS) of the university graduate through E-<strong>learning</strong>.<br />
Imran Ali is a Senior Learning technologist at Coventry University. He is passionate about the potential of<br />
technology in education. His background is in multimedia production and education. He currently supports<br />
staff across the Faculty of Health and Life Sciences in making effective use of technology for teaching,<br />
<strong>learning</strong>, assessment and research purposes.<br />
Nahla Aljojo worked as an IT manager in the economics and administration department at King Abdul Aziz<br />
University in Saudi Arabia. She has a master’s degree in computer systems and Information Technology<br />
from the Washington International University (USA) and is also a Ph.D. doctoral student at the School of<br />
Computing at the University of Portsmouth (UK).<br />
Hussein Al-Yaseen holds a PhD degree in Information Technology from the University of Brunel, UK. His<br />
areas of interest are in information systems evaluation, project management and the implementation of<br />
information systems in developing countries. Hussein is currently an associate professor at the Management<br />
Information Systems Department at Al-Ahliyya Amman University, Jordan.<br />
António Andrade PhD in Technologies and Information Systems MSc in Information and Management<br />
Director of the MSc in Information and Documentation.<br />
Roni Aviram is Chair of the Center for Futurism in Education, Department of Education, Ben-Gurion<br />
University. Aviram is interested in the impact of ICT on education and society, and in structuring theoretical<br />
and practical change processes in education. He has led R&D projects dedicated to designing virtual<br />
lifelong <strong>learning</strong> environments for enhancing human development and flourishing.<br />
Jonathan Barkand has a B.S. in Technology Education and an M.S. in Multimedia Technology from<br />
California University of Pennsylvania. He is currently completing his dissertation work at Duquesne<br />
University for a doctorate in Instructional Technology. His area of focus is online education and its effects on<br />
teachers and students.<br />
Trevor Barker is Principal Lecturer and University Fellow in Teaching and Learning in the Department of<br />
Computer Science at the University of Hertfordshire. He obtained a PhD for research into developing and<br />
evaluating adaptive multimedia computer systems in education. His recent research relates to the design<br />
and evaluation of adaptive educational systems and the affordances of virtual worlds such as Second Life<br />
for study and work.<br />
Josep Batalla has a Degree and a Phd in Economics from the University of Barcelona. He is a Professor of<br />
Economics and Business Studies at the Universitat Oberta de Catalunya (UOC), where he teaches courses<br />
in the field of applied economics. Resident scholar at the Internet Interdisciplinary Institute<br />
Olivia Billingham is a research fellow in the e-<strong>learning</strong> development unit at the University of the West of<br />
England. Her research interests include student perceptions and experiences of technology enhanced<br />
<strong>learning</strong>.<br />
Latefa Bin Fryan (BSc, MSc) is a Researcher in the School of Information Systems, Computing and<br />
Mathematics at Brunel University, UK, where she is pursuing her PhD in the area of e-Learning systems.<br />
She received M.Sc. (2009-2010) in Information Systems Management from Brunel University, UK. She<br />
has published papers in several international conferences.<br />
Sonya Campbell As senior service manager within Library and Information Services, Sonya supports the<br />
delivery of flexible <strong>learning</strong> environments and the development of services designed to maximise learner<br />
productivity and success. Current research includes the opportunities social media may bring to enable<br />
student engagement, <strong>learning</strong> and collaboration. Teaching themes are corporate strategy, effective<br />
management and customer service.<br />
xiv
Cornélia Castro is a PhD student in Educational Sciences: Educational Computing Master in Quality<br />
Control: Environment . Graduation in Pharmaceutical Sciences High School Chemistry and Physics teacher<br />
. Research interests: ICT integration in the classroom, Web 2.0 tools in Education, Open Educational<br />
Resources, Learning Objects, Repositories and Social Media in Education<br />
Martin Cápay works as a professor assistant at the Department of Informatics. He deals with the theory of<br />
teaching informatics subjects, mainly programming. He participates in the projects aimed at the usage of new<br />
competencies in teaching and also in the projects dealing with <strong>learning</strong> in virtual environment using e<strong>learning</strong><br />
courses.<br />
Tim Cappelli is an experienced Project Manager who has successfully managed and contributed to a range<br />
of Technology Enhanced Learning and change management projects. Tim has developed a rich<br />
understanding of the application of informal and collaborative <strong>learning</strong> through the use of social networking<br />
tools that reinforce the links between individual and organisational development.<br />
Ivana Cechova graduated from the Faculty of Arts at Masaryk University, Brno, with specialisations in<br />
pedagogy, English and Russian language and literature. She has worked as Head of Research and Deputy<br />
Head of the Language Department at the Faculty of Economics and Management. Currently she is a senior<br />
lecturer at the Language Training Centre of the University of Defence.<br />
Vivian Chan Yin Ha is a Senior Instructor of the Independent Learning Centre at the Chinese University of<br />
Hong Kong. Her research interests include language teaching, <strong>learning</strong> autonomy, modern and<br />
contemporary Chinese literature, and gender studies.<br />
Serdar Ciftci is a doctoral candidate studying in Department of Educational Technology, Institute of<br />
Educational Studies of Gazi University. He is also an expert working in Directorate of Computer Center of<br />
Gazi University. His researcher interests are focused on e-<strong>learning</strong>, metacognition, web-based <strong>learning</strong><br />
strategies.<br />
Chetz Colwell is a Project Officer in the Learning and Teaching Development team of the Institute of<br />
Educational Technology at the Open University. Her role is to support the university in making online<br />
teaching materials accessible to disabled students by conducting technical accessibility testing, and<br />
conducting student evaluations.<br />
David Comiskey lectures in Architectural Technology at the University of Ulster. He is a Fellow of the Higher<br />
Education Academy and his research interests include the use of video and screencasting technology to<br />
improve the student <strong>learning</strong> and feedback experience. He was recently funded by the Higher Education<br />
Academy to continue researching this area.<br />
Mary Crossan has worked at Coventry University for six years and has industry experience of over 16<br />
years. Her experience has been mainly in the financial area but with a high emphasis on managing people<br />
and resources for project delivery. Research interests are in the areas of improving student experience and<br />
work-based <strong>learning</strong> approaches.<br />
Marija Cubric is a Principal Lecturer at the UH Business School, where she teaches information systems,<br />
and project management related subjects. She is also a member of the UH Learning and Teaching Institute<br />
that promotes the use of blended <strong>learning</strong> practices across the University. Her research interests include<br />
educational technologies, and their influence on <strong>learning</strong> and teaching processes.<br />
Maya Khemlani David (Faculty of Languages and Lingusitics, University of Malaya) received the Linguapax<br />
Award in 2007. She is an Honorary Fellow of the Chartered Institute of Linguists, United Kingdom and an<br />
Honarary Member of the Foundation of Endangered Languages. Her areas of specialisation are crosscultural<br />
communication, discourse analysis and language maintenance and language shift.<br />
Christine Davies is a Senior Lecturer in Technology-Enhanced Learning at the University of Glamorgan.<br />
Originally a Biology teacher, she has taught in schools and FE, and continues to teach on OU science<br />
courses and on Wales-based post-graduate education courses. She has also worked for JISC RSC Wales<br />
as an e-Learning Advisor.<br />
Mark de Groot works for the Centre for Teaching and Learning team at Leeds Metropolitan University. He<br />
has more than 15 years’ experience initiating and supporting <strong>learning</strong> technology related staff development<br />
activities across and beyond the University.<br />
xv
Jiří Dlouhý is a specialist on cybernetics and works at the Environmental Center of the Charles University as<br />
head of the Environmental Education department. He is also a member of executive board of the European<br />
Environmental Bureau, chairman of the Society for Sustainable Living; and a founding member of the<br />
International Society of Information Specialists<br />
Elena Dominguez-Romero holds a Master’s degree in Innovative Teaching in Tertiary Education (Huelva,<br />
2008). She has participated in several Research Projects concerning Innovative Teaching (Universidad de<br />
Huelva: 2005, 2006a, 2006b; UCM: 2009, 2010). The results of this ECEL paper are derived from her<br />
participation in a current project funded by UCM (268-2010 UCM).<br />
Jon Dron is a member of the Technology Enhanced Knowledge Research Institute (TEKRI) at Athabasca<br />
University, Canada, an Honorary Faculty Fellow in the Faculty of Education & Sport, University of Brighton,<br />
UK, and a National Teaching Fellow of the UK Higher Education Academy.<br />
Michaela Drozdova graduated in 2009 with a Masters degree in Secondary Teaching Mathematics and<br />
Informatics approbation. In September 2009 she began teaching high school Teleinformatics, while<br />
commencing doctoral studies in the Department of Information and Communication Technology in Education<br />
at the University of Ostrava.<br />
Glenn Duckworth currently works at the University of the West of England in the role of e-Learning<br />
Development Officer. He is employed by the Faculty of Business and Law and is a member of the e-Learning<br />
Development Unit. He previously worked at the University of Huddersfield as Senior Lecturer, Researcher<br />
and Chief Psychology Technical Officer.<br />
Ibrahim Elbeltagi is a lecturer in information and knowledge management at the School of Management,<br />
University of Plymouth. His publications are related to electronic commerce, adoption of ICT, information<br />
systems in developing countries, social networking and knowledge management. He has more than 30<br />
journal and conferences papers published in national and international journals and conferences.<br />
Antonella Esposito. E<strong>learning</strong> practitioner since 1996, Antonella led the CTU (E-<strong>learning</strong> center) of the<br />
University of Milan for seven years. Currently she is a PhD candidate in the E-<strong>learning</strong> program, Open<br />
University of Catalonia and is completing a dissertation on digital scholarship for the MRes in Educational<br />
and Social Research, Institute of Education, University of London.<br />
Liz Falconer completed her MPhil at Manchester Polytechnic in 1992 and her PhD at Salford University in<br />
2001. In 2001 she moved to the University of Bath as Director of the Centre for Distance Education. She is<br />
now Professor of Technology Enhanced Learning at the University of the West of England.<br />
Gert Faustmann studied Information Technology at the Technical University Berlin. From 1992 to 2001 he<br />
was a software developer (Siemens AG), researcher (Fraunhofer Institute for Software and Systems<br />
Engineering) and consultant (debis - later T-Systems). He is now professor and course director for the<br />
division of computer science at Berlin School of Economics and Law.<br />
Sérgio André Ferreira has a Degree in Geography, specialization in Education Sciences. Master in<br />
Sciences of the Education, specialization in Education Computer Sciences. PhD student in Sciences of the<br />
Education, specialization in Education Computer Sciences (Grad Student). Research interests in Technology<br />
Enhanced Learning Environments. Geography teacher and Teacher Trainer in the areas of ICT and<br />
Geography.<br />
Rachel Fitzgerald, MSc is a Senior Lecturer (Information Science) at Northampton Business School,<br />
University of Northampton and a PhD student at the University of Lancaster. A former <strong>learning</strong> technologist,<br />
she holds CMALT certification and a Teaching Fellowship Award from the University of Northampton.<br />
Current research interests include Networked Learning and Distance Education Innovation.<br />
Michael Flavin is a research student at the Open University’s Institute for Educational Technology. He<br />
previously gained a PhD in nineteenth-century literature (Kent, 1999). He has taught at King’s College<br />
London and the Open University, and is currently Head of Learning and Teaching at the IFS School of<br />
Finance, a degree-awarding business school in London.<br />
Gabriele Frankl is head of the eLearning Service department at the Alpen-Adria-Universität Klagenfurt since<br />
2008. Her main research interests are eLearning and Blended Learning, Knowledge Management and winwin<br />
constellations as well as self-organisation processes. She has successfully implemented eLearning and<br />
Knowledge Management systems in the production industry.<br />
xvi
Manuel Frutos-Perez is the Leader of the E-<strong>learning</strong> Development Unit at the University of the West of<br />
England, Bristol, UK. He has worked for several universities in the UK, Spain and Austria. He has also<br />
written and contributed to numerous textbooks and e-<strong>learning</strong> interactive materials.<br />
Elaine Garcia is undertaking a part time PhD at the University of Plymouth focusing on the use of blogs<br />
within education. This is alongside employment as Head of Operations and Resources at Plymouth College<br />
of Art. Research interests include: Social Networking Sites, Social Media, Blogs, Social Learning and<br />
Knowledge Management.<br />
Danny Glick is an international e-<strong>learning</strong> consultant and an expert in the implementation of large-scale<br />
computer-based solutions. He has worked as a consultant for both public and private organizations. Glick is<br />
a research fellow at the Center for Futurism in Education at Ben-Gurion University, where he conducts<br />
research on mindful <strong>learning</strong> processes in Web-based <strong>learning</strong> environments.<br />
Andrea Gorra is a Senior Lecturer in the Business School at Leeds Metropolitan University (UK). Prior to<br />
this she has worked as a researcher in the area of <strong>learning</strong> technologies. Further research interests include<br />
grounded theory methodology, and the use of social software and mobile <strong>learning</strong> devices for assessment,<br />
<strong>learning</strong> and teaching.<br />
Rose Heaney is a <strong>learning</strong> technology advisor and Teaching Fellow at the University of East London with<br />
responsibility for facilitating the use of <strong>learning</strong> technologies in the schools of Health, Sport & Bioscience and<br />
Psychology. One of her research interests is the use of Second Life® in clinical education as well as for<br />
more general purposes.<br />
Carolina Hintzmann has a Degree in Economics from the Universitat de Barcelona, Spain. She is a teacher<br />
in the Economics and Business Sciences Department, Universitat Oberta de Catalunya (UOC), Barcelona,<br />
Spain<br />
Karen Hughes Miller, PhD, is Director for Graduate Medical Education (GME) Curriculum Design,<br />
Evaluation and Research, University of Louisville School of Medicine. Her work in adult education includes<br />
teacher education, military education, and currently medical education. She is interested in curriculum design<br />
and analysis and education research, and enjoys the challenges of teaching advanced content to very bright<br />
adult learners.<br />
Anne Jelfs is Manager of the Learning and Teaching Development team of the Institute of Educational<br />
Technology at the Open University. She manages a team that conducts evaluations of OU materials<br />
including usability testing. She also leads on a university-wide initiative to embed accessibility within the<br />
university's production and delivery systems.<br />
John Jessel is head of the PhD Programme in the Department of Educational Studies at Goldsmiths,<br />
University of London. His research activities focus on the social and cognitive processes that underlie<br />
<strong>learning</strong> and development both inside and outside the school setting and within the context of the use of<br />
digital technologies.<br />
Aidan Johnston, BSc. (Hons), PGDip, AHEA, is a Blended Learning Advisor with many years’ experience<br />
designing and embedding video and audio resources to enhance the teaching and <strong>learning</strong> experience of the<br />
learner. He has collaborated on projects involving the JISC, BBC Information & Archives, The National<br />
Science Foundation, The Higher Education Academy and REAP.<br />
Rosario Kane-Iturrioz is a Senior Lecturer in Spanish at Coventry University and also a member of the<br />
Chartered Institute of Linguists and the Higher Education Academy (UK). Her research interests are the<br />
integration of Information and Communication Technology (ICT) into language <strong>learning</strong> and assessment and<br />
the use of VLEs as an effective <strong>learning</strong> tool.<br />
Jana Kapounova is an associate professor at the Department of ICT, University of Ostrava in the Czech<br />
Republic. She teaches subjects as Educational Technology, ICT in Education, eLearning. She guarantees<br />
studies of ICT in Education in bachelor, master and Ph.D. degrees. Her research field is eLearning and<br />
evaluation of its quality.<br />
Shakeel Khoja is a Commonwealth <strong>Academic</strong> Fellow. He received his Ph.D. from the University of<br />
Southampton, UK, in 2001, and is working as a Professor at IBA, Karachi. Shakeel’s research interests<br />
xvii
include Learning Technologies, Web Technologies, and Internet programming. He has a professional career<br />
of over 15 years and has fifty research publications to his credit.<br />
Kaido Kikkas is an Associate Professor at Estonian Information Technology College and an Associate<br />
Professor of Social and Free Software at Tallinn University. His main focus of research and teaching has<br />
been a wide range of ethical and social issues of IT (information society, licensing, hacker culture etc).<br />
John Kleeman is the Founder and Chairman of Questionmark. John wrote the first version of the<br />
Questionmark assessment software system and founded Questionmark in 1988 to market, develop and<br />
support it. John has been heavily involved in assessment software development for over 20 years and has<br />
also participated in several standards initiatives including IMS QTI.<br />
John Knight started his career teaching EAP at universities in Turkey and Qatar, but developed an interest<br />
in computers and education in the late 90s and has worked in the field ever since. He now works in the<br />
Learning Development Unit, Bucks New University, where he has a particular interest in technologyenhanced<br />
<strong>learning</strong> and teaching.<br />
Katerina Kostolanyova has worked in the Faculty of Education, Institute of Information and Communication<br />
Technologies in Ostrava since 1999. She specializes in eLearning technology. Her further professional<br />
growth focuses on students’ <strong>learning</strong> styles in the e-Learning environment. She is an author and co-author of<br />
almost thirty professional articles and ten e-contents.<br />
Vivek Kumar is an Associate Professor in the School of Computing and Information Systems at Athabasca<br />
University researching in online <strong>learning</strong> technologies. Vivek's research centres around Technology-<br />
Enhanced Teaching, Learning, and Research that extends to mixed-initiative human-computer interaction,<br />
causal modelling, model tracing, automated instructional design, lifelong <strong>learning</strong>, cognitive modelling of selfregulated<br />
and co-regulated <strong>learning</strong>, semantics of online <strong>learning</strong> interactions, and competency modelling in<br />
portfolios.<br />
Eugenijus Kurilovas is Research Scientist in Vilnius University Institute of Mathematics and Informatics and<br />
Associate Professor in Vilnius Gediminas Technical University. He is a member of 12 scientific committees,<br />
has published more than 50 scientific papers, and participated in more than 10 EU-funded projects. His<br />
papers for ECEL 2008, 2009 and 2010 were published in Electronic Journal of e-Learning.<br />
Linda Leake, M. Ed., serves as a computer specialist, University of Louisville Blackboard technical<br />
support/trainer and instructional designer for the Delphi Center for Teaching and Learning. She received<br />
both her Bachelor of Science in Business Administration degree in 1994 and her Master of Education in<br />
Instructional Technology in 2003 from the University of Louisville.<br />
Jake Leith leads a Business and Professional Practice in Fashion & Textiles and Design & Craft at the<br />
University of Brighton, and has over 20 years’ experience in successfully running his own design practice<br />
and 13 years as a teacher-practitioner. Leith is currently President Elect of the Chartered Society of<br />
Designers.<br />
Birgy Lorenz is an eSafety trainer in Estonia. Her activities include being part of developing National<br />
Curricula ICT syllabus, writing articles about e-safety, project management in TurvaLan project what has<br />
recently been awarded by: Microsoft (2009): Innovative Teacher Award and European Schoolnet (2010): 1 st<br />
eLearning Award, in ‘Internet Safety’ category.<br />
Beulah Maas is a lecturer in social and health care leadership and management in the Faculty of Health and<br />
Life Sciences at Coventry University. She is an occupational psychologist and her key areas of teaching and<br />
research are developing and managing people and social entrepreneurship.<br />
Linda Martin is a senior lecturer and qualified social worker working in the Faculty of Health and Life<br />
Sciences at Coventry University. She teaches leadership and management in health and social care,<br />
including programmes which are entirely on line and has a particular interest in developing approaches to<br />
student engagement.<br />
Marie Martin is an education consultant in Northern Ireland specialising in e-<strong>learning</strong>, and is adjunct faculty<br />
of Duquesne University, Pittsburgh, USA, where she earned her doctorate in education in 2007. She has<br />
presented internationally and authored several publications on aspects of e-<strong>learning</strong>. In 2010 she was<br />
Rooney International Visiting Scholar in Robert Morris University, Pittsburgh.<br />
xviii
David Mathew works at the Centre for Learning Excellence at the University of Bedfordshire. In addition to<br />
his work as a Learning Technologist, working cross-faculty on online <strong>learning</strong> programmes, he co-edits The<br />
Journal of Pedagogic Development and teaches writing/publishing. He is also interested in psychoanalysis<br />
and creative writing, and published a novel in 2011.<br />
Peps Mccrea is a Senior Lecturer in Initial Teacher Education at the University of Brighton. He is interested<br />
in pedagogy and the role of the teacher in our increasingly digital age.<br />
Peter Mikulecky is a professor of Managerial Informatics at the Faculty of Informatics and Management at<br />
the University of Hradec Kralove, Czech Republic, since 1993. He has been the head of the Department of<br />
Information Technologies since 1994; recently he acts also as Director for Research and Director of<br />
Postgraduate Studies.<br />
Jonathan Moizer is a lecturer in business operations and strategy. His research interests include the use of<br />
computerised simulation for both decision-making and <strong>learning</strong>. He has particular interest in the use of<br />
system dynamics modeling as a simulation approach. In recent years Jonathan has organized three national<br />
workshops on using simulation games in teaching and <strong>learning</strong> for the Higher Education Academy.<br />
Kate Mottram works at Coventry University as a Graduate Teaching Assistant. Her research interests<br />
include general management and the theories surrounding quality in business organisational settings along<br />
with management in engineering.<br />
Peter Mozelius is working as a researcher and IT-Pedagogue at the Department of Computer and Systems<br />
Sciences, Stockholm University, Sweden. His research interests are in the fields of ICT4D, e-<strong>learning</strong>,<br />
software engineering and software visualization. During the last six years he has been working in aid projects<br />
in developing countries in southern Asia.<br />
Antoinette Muntjewerff is an Assistant Professor in General Legal Theory at the University of Amsterdam.<br />
Studied Educational Science and Law. PhD research involved theoretical and empirical studies into legal<br />
case solving and evaluation of an instructional environment for <strong>learning</strong> to solve legal cases PROSA. Her<br />
research is on modelling legal knowledge and legal reasoning for developing electronic materials for <strong>learning</strong><br />
the law.<br />
Phelim Murnion is a Senior Lecturer in Information Systems in the School of Business at Galway-Mayo<br />
Institute of Technology; lecturing on business information systems, data warehousing, and knowledge<br />
management. His research interests include Data Warehousing, Business Intelligence and Educational<br />
Technology. Phelim is a member of the Business Informatics (research) Group at Dublin City University.<br />
Chrissi Nerantzi is an <strong>Academic</strong> Developer at the University of Salford. She is involved in the development<br />
and delivery of the Postgraduate Certificate in <strong>Academic</strong> Practice (PGCAP) and CPD across the university.<br />
Chrissi’s current professional interests are social media for <strong>learning</strong>, creative teaching and <strong>learning</strong> in HE,<br />
online Problem-Based Learning, Open Educational Practice and mobile <strong>learning</strong>.<br />
Michaela Noakes is a Doctoral candidate and Research Assistant in Instructional Technology in the School<br />
of Education at Duquesne University. She received her MS-ISM, MBA and M. A. from Duquesne. She also<br />
is an Adjunct Faculty member at Duquesne University, Point Park University and Butler County Community<br />
College. She was recently inducted into Pi Lambda Theta, the International Honor Society for Educators.<br />
Abel Nyamapfene received his Ph.D. in Computing from the University of Surrey, Guildford, UK in 2006,<br />
and his MSc. in Communication Engineering and B.Sc. (Hon) in Electrical Engineering from the University of<br />
Zimbabwe in 1990 and 1997 respectively. He is currently the Assistant Director for Education (Engineering)<br />
at the University of Exeter, UK.<br />
Maruff Akinwale Oladejo is a faculty in the Department of Educational Foundations, Federal College of<br />
Education (Special), Oyo State, Nigeria. He is a member of the Editorial Advisory Board, JournalsBank<br />
Publishing Inc. (UK, Pakistan, India and West Africa).<br />
Fatemeh Orooji is a Ph.D. Candidate in the Department of Computer Engineering, Software Engineering<br />
group, at the University of Tehran, Iran. She is interested in web- based educational systems (WBES) and<br />
technology-enhanced <strong>learning</strong>. Her research activities focuses on Collaborative and Networked Learning,<br />
Educational Data Mining, Adaptive and Personalized Learning.<br />
xix
Kanari Paraskevi is a five year experienced school teacher of French Language in Greece with a master’s<br />
degree in French Literature and is a PhD student in the same field. She has carried out several cultural<br />
programs promoting education through art and theatrical research. She speaks four languages and has<br />
published various articles concerning French Literature.<br />
Lyn Pemberton is Reader in Human Computer Interaction at the University of Brighton. Lyn has been<br />
involved in numerous <strong>learning</strong> technology projects, mostly concerned with aspects of communication, writing<br />
and language <strong>learning</strong>. Her most recent projects have involved interactive television for <strong>learning</strong>, augmented<br />
reality and in particular mobile language <strong>learning</strong>.<br />
Katie Piatt has been working in the field of Learning Technologies for 10 years, with experience in<br />
development, implementation and evaluation. Her research focuses on methods of student engagement.<br />
Current projects include investigating the integration of social media with traditional environments and the<br />
introduction of game-based <strong>learning</strong> principles as part of the ALT special interest group.<br />
Andy Pulman works for the School of Health and Social Care (HSC) at Bournemouth University, where<br />
he manages and creates web resources and e-<strong>learning</strong> initiatives. He began studying for a part-time PhD in<br />
2008, concerning the role of Web 2.0 and Mobile technology to support people living with chronic illness and<br />
enhance quality of life.<br />
Thomas Putz is responsible for the initiation, the application and the coordination of national and<br />
international projects and acts as the contact point with funding organizations in the evolaris. He has been<br />
involved as coordinator and as specialist in eLearning and mLearning in many projects funded under Adapt,<br />
Grundtvig, Socrates, FP6, eContent+ and the LLP.<br />
Ricardo Queirós is an assistant professor at ESEIG/IPP which is responsible for Programming Languages<br />
courses. He is a Ph.D. student in the Faculty of Sciences of the University of Porto. His scientific activity is<br />
mostly related with e-Learning standards and Interoperability. He is member of CRACS/ INESC-Porto and a<br />
founding member of KMILT research group.<br />
Zahra Rahimi is a Master of Science Student in the Philosophy of Science group, at Amir Kabir University of<br />
Technology. She received a Bachelor degree in Computer Engineering- Software, from University of Tehran.<br />
She is interested in interdisciplinary topics, subjects such as use of software in education and philosophy of<br />
intellectual intelligence.<br />
Yambu Andrik Rampun recently completed his M.Sc Software Engineering (Department of Computer<br />
Science) at the University of Hertfordshire. He previously received his Diploma in Computer Science and<br />
B.Sc (Hons) degree Software Engineering from University Technology Mara (Department of Computer<br />
Science and Mathematics). He is interested in designing, development and evaluation of e-<strong>learning</strong><br />
applications as well as human computer interactions.<br />
Fauziah Redzuan is a junior lecturer at the University of Technology MARA, Malaysia. She holds a Master<br />
of Science in Information Technology from the University of Science Malaysia in the area of multimedia<br />
<strong>learning</strong>. Currently she is pursuing her study in the area of emotion and online <strong>learning</strong> using the Kansei<br />
Engineering technique.<br />
Bart Rienties is lecturer at the Centre for Educational and <strong>Academic</strong> Development at University of Surrey.<br />
His primary research interests are focussed on computer-supported collaborative <strong>learning</strong> and the role of the<br />
teacher to design effective blended and online <strong>learning</strong> courses.<br />
Manuel Rodrigues teaches Computer Science at Secondary School Martins Sarmento, Portugal, where he<br />
is responsible for the IT infrastructure of the school. He has a Computer Science degree and an MsC, both<br />
from Minho University, Portugal. Currently he’s a PhD Student at Vigo University, Spain, and working on<br />
recommendation and personalization systems for E-<strong>learning</strong> platforms such as Moodle.<br />
Andrée Roy is an engineer and professor of Management Information Systems at the Université de<br />
Moncton. Ms Roy is also an e-<strong>learning</strong> consultant and the owner of a business specialising in the field of e<strong>learning</strong><br />
called Web Training Solutions. Previously, Ms Roy worked as an industrial engineer and in the<br />
information technology and economic development sectors.<br />
Zuzana Šaffková is a senior lecturer at the Department of English, Technical University of Liberec, Czech<br />
Republic, where she specializes in the teaching of academic reading and writing to teacher trainees.<br />
xx
Research interests include the investigation of the effectiveness of e-<strong>learning</strong> in developing student<br />
academic capability, especially critical reading competence.<br />
Ahmed Salem is an Assistant Professor at the faculty of Engineering, King Abdul Aziz University at KAU. He<br />
graduated from CMU, PA, USA and he is interested in Active Learning, E-Learning and Excellence in<br />
engineering teaching. His contributions include a Design Helping Mechanism for Active Learning Courses<br />
that aid in designing <strong>learning</strong> experiences.<br />
Rowena Santiago is a tenured professor (emeritus) in Instructional Technology and founding/former director<br />
of the Teaching Resource Center, California State University, San Bernardino. Her research activities<br />
include learner characteristics in e-<strong>learning</strong> (US and Japan), the use of faculty <strong>learning</strong> communities for<br />
faculty development, and the use of student response systems for active <strong>learning</strong>.<br />
Vitor Santos is an invited Professor at Trás os Montes e Alto Douro University (UTAD), Minho University<br />
(UM) and Lusofona University (ULHT), teaching Information Systems, Compilers, and Artificial Intelligence<br />
courses in Computer Science and Informatics Engineering Degrees. Before that, he was the Microsoft<br />
Portugal <strong>Academic</strong> Computer Science Program Manager for 8 years.<br />
Marie Sams has worked in higher education and the NHS in a number of roles over the past fifteen years.<br />
This has included areas predominantly in marketing management and project management. Her research<br />
interests are in team creativity and peer <strong>learning</strong>.<br />
Maggi Savin-Baden is Professor of Higher Education Research at Coventry University, UK, and is director<br />
of the Learning Innovation Research Group. She has been researching <strong>learning</strong> for over 20 years and has<br />
just completed her 11th book. In her spare time she runs and tries out extreme sports<br />
Adriana Schiopoiu Burlea is Professor PhD at the University of Craiova and visiting professor at<br />
universities in France and Poland. She is Romanian ambassador for AGRH. She has written more than 20<br />
books and 130 articles in the field of Management. She is a member of the editorial boards of many<br />
prestigious journals and conferences.<br />
Fabio Serenelli is about to conclude the PhD course in "Information Communication Technology applied to<br />
knowledge society and <strong>learning</strong> processes" at the Milano Bicocca University where he collaborates as e-<br />
Learning Manager and Instructional Designer at the CPM (Multimedia Production Center). He adopts an<br />
interdisciplinary approach and his research interests are related to instructional formats for multimedia<br />
<strong>learning</strong>, infographics and User Experience Design, One to One Computing, PLE - Personal Learning<br />
Environment.<br />
Angela Shapiro MSc, Dip Ad Ed, TQFE has been a Lecturer at Glasgow Caledonian University for ten<br />
years. She teaches across all academic schools applying an advanced academic literacies (AAL) approach.<br />
This includes workshops across the university within the context of particular programmes, as well as<br />
distance and blended <strong>learning</strong> support to students.<br />
Cees Smit Sibinga is professor of International Development of Transfusion Medicine, University of<br />
Groningen. He founded the <strong>Academic</strong> Institute for International Development of Transfusion Medicine<br />
(IDTM), focusing on higher education in developing countries. Objective - to support leadership development.<br />
The first post-graduate course designed and implemented is a Masters in Management of Transfusion<br />
Medicine (MMTM).<br />
Dina Soeiro. Besides being a trainer of college teachers on E-<strong>learning</strong>, Dina supports and studies the use of<br />
Moodle at the College of Education of the Polytechnic Institute of Coimbra where she teaches. She is<br />
finalizing her PhD on ICT in Education at the University of Coimbra and she is a researcher at the CISUC.<br />
Mekala Soosay is a university teaching fellow and Technology-Enhanced Learning (TEL) team member with<br />
a passion for preparing and disseminating creative methods for assessment, <strong>learning</strong> and teaching. She<br />
also trains colleagues in applying TEL for their teaching practice. She has worked on a number of JISC and<br />
EU-funded projects, the current ones being PC3 and EuroPlot.<br />
Sonia Sousa holds a PhD in Education from Sheffield Hallam University, UK and an honors degree in<br />
Communication Engineering from Universidade Fernando Pessoa, Portugal. She is currently researching the<br />
influence of trust in online communities. R&D work includes Michigan State University's MIND Lab, USA,<br />
Universidade Fernando Pessoa Multimedia Research Center, Portugal, as well as Cape Verde and<br />
Mozambique.<br />
xxi
Iain Stewart has worked at Glasgow Caledonian University for more years than he cares to remember. He is<br />
primarily interested in the ways technology can be used to improve the <strong>learning</strong> experience of students. His<br />
other activities include working to improve FE to HE articulation and critical testing of the elasticity of<br />
deadlines.<br />
Caroline Stockman works at the Catholic University of Leuven as a lecturing assistant for the course of<br />
'Online Publishing', a key subject within the Master's degree of Cultural Studies. Her time is divided between<br />
Belgium and the UK, where she trains teaching staff in secondary and higher education in the use of<br />
educational technology.<br />
Amanda Sykes (PhD, DipAcPrac, FHEA) works with students of Science, Engineering and Veterinary<br />
Medicine at The University of Glasgow to enhance their <strong>learning</strong>. She is particularly interested in using<br />
emerging technologies to shift the focus from didactic lecturing towards scholarship that enables students to<br />
understand their own <strong>learning</strong> process as well as academic content.<br />
Fattaneh Taghiyareh is an Assistant Professor of Computer Engineering- Software & Information<br />
Technology, at the University of Tehran, where she has served since 2001. She received a Ph.D. in<br />
Computer Engineering from the Tokyo Institute of Technology in 2000. Her research interests include<br />
Human-Centered Computing applied to Learning Management Systems and based on Multi-Agent Systems.<br />
Nicolet Theunissen is senior research scientist ‘Training Innovations’ at TNO, The Netherlands. Her<br />
research is dedicated to the development and testing of training solutions for professionals. These <strong>learning</strong><br />
solutions imply both innovative <strong>learning</strong> technology and didactics for adult <strong>learning</strong>. Formerly, she worked at<br />
Utrecht University, Leiden University and at the Netherlands Institute of Health Services Research.<br />
Vladimir Tomberg works as a researcher in the Centre for Educational Technology, Tallinn University,<br />
Estonia. His main research topics are interoperability of online assessment tools and course coordination<br />
within blog-based Personal Learning Environments. He has stayed recently as a visiting researcher in Simon<br />
Fraser University at Vancouver, Canada and in RWTH at Aachen, Germany.<br />
Panos Vlachopoulos is Lecturer in the Centre for Learning Innovation and Professional Practice at Aston<br />
University. He leads the Postgraduate Certificate in Professional Practice in Higher Education and advises<br />
Aston University staff on the development of fully online distance <strong>learning</strong> programmes. His research<br />
interests include the online facilitation of student-centred <strong>learning</strong> and reflective practice.<br />
Chien-hwa Wang is a professor in the Department of Graphic Arts and Communications at National Taiwan<br />
Normal University. His recent research emphasis is Web 2.0 applications in Education. He also directed a<br />
government supported digital archiving project to preserve Taiwanese puppetry master Tien-lu Li’s historic<br />
works and collections .<br />
Cristina Wanzeller is professor at the Informatics Department, Technology and Management School, Viseu<br />
Polytechnic Institute (IPV), Portugal. Her teaching activities focus on Information Systems and Databases<br />
areas. She is a member of the IPV’s research centre CI&DETS. Her main scientific areas of research are<br />
Web Usage Mining, Case-Based Reasoning, e-Learning and Information Systems.<br />
Julie Watson is Principal Teaching Fellow in eLearning in Modern Languages at the University of<br />
Southampton. She manages eLanguages which produces and licenses the EAP and Study Skills Toolkits.<br />
She also led the development of the Prepare for Success website, works on projects involving emergent<br />
technologies, and develops online courses and resources.<br />
Phil Watten is from the University of Sussex, has been researching and developing <strong>learning</strong>-based<br />
technology and production methods since 2005. Using the Sussex-based Media Technology Lab, he has<br />
produced studio-based and portable production systems and is now working on fully automated approaches.<br />
Katherine Wimpenny, PhD, MSc, DipCOT, Cert Ed, is a Research Fellow at Coventry University within the<br />
Learning Innovation Research Group. She is involved in researching the impact of virtual worlds on teaching<br />
and <strong>learning</strong>. She has recently gained funding to conduct two qualitative research syntheses, enjoys<br />
supervising PhD students and professional background is occupational therapy.<br />
Koos Winnips is advisor in ICT and Education, with a particular interest in digital pedagogies: sorting out<br />
<strong>learning</strong> arrangements with the help of the electronic <strong>learning</strong> environment. He has worked at the University<br />
of Twente in the Netherlands and Glasgow Caledonian University as researcher, lecturer and advisor of ICT<br />
in Education.<br />
xxii
Novita Yulianti is a PhD student at the Faculty of Business Studies and Economics, University of Bremen,<br />
Germany. She received a Master of Science degree on International Technology Transfer Management. Her<br />
current research interests include change management, organizational management, information and<br />
technology management, innovation management.<br />
Li-Zhang Zhang received BA from Huazhong University of Science and Technology, China, MSc from the<br />
University of London, UK, and PhD from the University of Sydney, Australia. He currently teaches<br />
International Business and Economics at La Trobe University, Australia. His research interests include<br />
Asian-Australian Business and Trade, E-Commerce, Theory and Practice in English-Chinese Translation,<br />
and Online Education and Training.<br />
xxiii
Revisiting the Personal Transferable Skills Debate - an<br />
eLearning Pedagogical Perspective<br />
Samuel Adu Gyamfi 1 , Lene Tolstrup Sorenson 1 and Thomas Ryberg 2<br />
1<br />
CMI, Aalborg University, Copenhagen<br />
2<br />
Faculty of Humanities, Aalborg University<br />
sagy@plan.aau.dk<br />
Abstract: Personal Transferable Skills (PTS) are essential work skills which are not specific to any subject or<br />
profession, and which, though learned in one context may be successfully transferred to and applied in many<br />
other contexts. They are skills that enable people to acquire, structure, interpret and put to efficient use, their<br />
subject knowledge. There is a growing concern among educational providers and employers’ organisations on<br />
the lack of PTS of university graduates. This phenomenon has been amplified by the need of university<br />
graduates of the twenty-first century to possess skills and knowledge that can be effectively used in new domains<br />
and in different situations since there is also an increasing tendency for graduates to take jobs outside their<br />
academic field of study. There have been several attempts to promote PTS by university institutions in the<br />
traditional classroom settings where three broad approaches for developing PTS within the curriculum have been<br />
experimented with. To date, progress so far has been patchy. The paper is in two theoretical parts. The first part<br />
seeks to advance the theoretical framework of REALs as the better approach to teaching and <strong>learning</strong> in our<br />
universities. The second part of the paper argues that theoretically, communication theory (which draws on<br />
contemporary rhetorical theory) and social informatics theory provide important perspective for the application of<br />
eLearning based on REALs in the development of PTS for university graduates. The paper would contribute<br />
significantly to theoretical underpinnings of eLearning based on REALs for the development of PTS of university<br />
graduates.<br />
Keywords: personal transferable skills, eLearning, Rich Environment for Active Learning, intentional <strong>learning</strong>,<br />
co-operative <strong>learning</strong><br />
1. Introduction<br />
The main aims of university education are to inspire and enable individuals to develop their<br />
capabilities to the highest potential levels throughout life, so that they grow intellectually, are wellequipped<br />
for work, can contribute effectively to society and achieve personal fulfilment; and to serve<br />
the needs of an adaptable sustainable, knowledge-based economy at local regional and national<br />
levels (Dearing, 1997). Consequently, the graduates of our universities who would constitute the<br />
bedrock of the workers in this knowledge-based economy are expected to have more applied skills in<br />
terms of teamwork, communication skills, problem solving and lifelong <strong>learning</strong> (OECD, 2006). These<br />
skills have been variously referred to as core skills, personal skills, personal competences and<br />
personal transferable skills (Justice, Rice, & Warry, 2009; Bridges, 1993; Drummond, Nixon, &<br />
Wiltshire, 1998 & Nabi & Bagley, 1998). In this article, it has been referred to as personal transferable<br />
skills (PTS). Personal transferable skills are essential work skills which are not specific to any subject<br />
or profession and which, though learned in one context may be successfully transferred to and<br />
applied in many other contexts. They are skills that enable people to acquire, structure, interpret and<br />
put to efficient use, their subject knowledge (Gash & Reardon, 1988).<br />
There is growing concern among educational providers and employers’ organisations on the lack of<br />
PTS of university graduates (Drummond, Nixon, & Wiltshire, 1998 & Kemp & Seagraves, 1995). This<br />
phenomenon has been amplified by the need of university graduates of the twenty-first century to<br />
possess skills and knowledge that can be effectively used in new domains and in different situations,<br />
since there is also an increasing tendency for graduates to take jobs outside their academic field of<br />
study (Nabi & Bagley, 1998). There have been several attempts to promote PTS in university<br />
institutions in the traditional classroom settings where three broad approaches for developing PTS<br />
within the curriculum have been experimented with (Drummond, Nixon, & Wiltshire, 1998; Kemp &<br />
Seagraves, 1995 & Chadha, 2006). These approaches include the embedded or integrated<br />
development, parallel (or stand-alone) development and work placements or work-based projects.<br />
Kemp & Seagraves (1995) report on findings of an empirical study of transferable skills provision on<br />
five courses at Glasgow Caledonian University, which was carried out as part of a larger investigation<br />
into a novel form of work-based <strong>learning</strong> experience – Structured Industrial Practice Studies (SIPS),<br />
where the three approaches were used. The authors found out that a fairly high percentage of<br />
students at the end of the course did not feel they had received formal instruction or additional help in<br />
1
Samuel Adu Gyamfi et al.<br />
the development of these skills and consequently, concluded that there should be a radical rethinking<br />
of course structuring and its delivery if these skills are to be addressed seriously in higher education.<br />
Curry & Sherry (2004) report on a survey of 919 Irish graduate employers, students, recent graduates<br />
and academic staff of Trinity College of Dublin, Dublin City University and Waterford Institute of<br />
Technology to determine their attitudes to the importance of transferable skills and their perception of<br />
the current level of development of such skills in higher education programmes. Curry & Sherry (2004)<br />
found out that the two most important transferable skills for a graduate’s career were oral<br />
communication and time management. Again they found out that oral communication, research skills<br />
and written communication were being developed through academic programmes of the universities.<br />
Curry & Sherry (2004) concluded that transferable skills are highly valued by all stakeholders in higher<br />
education and considered an important factor in shaping graduates’ employability and in enhancing<br />
their personal development. Many transferable skills are and could be developed through academic<br />
programmes and that awareness must be created in higher education of the development of such<br />
skills.<br />
Justice, Rice & Warry (2009) report on a quasi-experimental study at McMaster University as part of a<br />
larger McMaster Innovative Learning Study which compared skill levels and academic performance<br />
over time of students who had taken an Inquiry course when they began university with students who<br />
had not taken the course. The Inquiry course was meant to prepare students to be more successful<br />
learners in their subsequent university courses and beyond. The authors found evidence that the<br />
Inquiry <strong>learning</strong> process was transferred from the course. Additionally, the students who had taken<br />
the course were able to use the skills acquired in new situations. The authors therefore, concluded<br />
that the transfer of core skills occurs under particular <strong>learning</strong> conditions that can be fostered through<br />
course design and enhanced through specific pedagogical objectives.<br />
It can therefore, be deduced from the literature that PTS could be developed in the traditional<br />
classroom. However, as Justice, Rice & Warry (2009) put it, “there are critical components to curricula<br />
design that enhance knowledge and skills transfer to other contexts – a process integral to <strong>learning</strong> to<br />
learn. Transfer does not just happen – it is not a natural bi-product of education; rather it is a process<br />
that needs to be consciously fostered through course design, and nurtured in the classroom”. Again<br />
the literature establishes the fact that for PTS development, the established models of good practice<br />
depends on opportunities to practice these skills with support and guidance which encourages and<br />
informs constructive reflection and the definition of strategies for improvement (Drummond, Nixon, &<br />
Wiltshire, 1998). Consequently, a new teaching and <strong>learning</strong> environment has to be created for<br />
effective PTS development of the graduate of the twenty-first century.<br />
2. Theoretical considerations<br />
Two possible answers that readily come to mind when the question of what constitutes <strong>learning</strong> is put<br />
are ‘the change in behaviour’ and ‘acquisition of knowledge’. However, in this write-up, we share<br />
Visser (2008) definition of <strong>learning</strong> as the “disposition of human beings, and of the social entities to<br />
which they pertain, to engage in continuous dialogue with the human, social, biological and physical<br />
environment, so as to generate intelligent behaviour to interact constructively with change”. For if<br />
<strong>learning</strong> should apply to the most comprehensive level of human endeavours then it should involve<br />
the development and maintenance of a lifelong disposition to dialogue with one’s environment for the<br />
purpose of constructively interacting with change in that environment (Visser, 2008).<br />
Research shows that at the university, what and how students learn should be the main responsibility<br />
of university academics ( (Laurillard, 2002). According to Laurillard (2002) the familiar methods that<br />
support <strong>learning</strong> by which academics offer tuition in our universities are through:<br />
Acquisition – and therefore academics offer lectures and reading;<br />
Practice – and therefore academics set exercises and problems;<br />
Discussion – and therefore academics conduct seminars and tutorials;<br />
Discovery – and therefore academics arrange fieldtrips and practicals.<br />
However, due to the impacts of ‘information society’, both the nature of knowledge and how<br />
knowledge is acquired should change (Nunes & McPherson, 2003). Indeed, <strong>learning</strong> in our higher<br />
education system has often focused on producing inert knowledge and abstract concepts which are of<br />
no use if the learner does not have the understanding to apply them in appropriate settings<br />
2
Samuel Adu Gyamfi et al.<br />
(Grabinger & Dunlap,1995 and Nunes & McPherson, 2003). CTGV, (1993) and Grabinger & Dunlap<br />
(1995) outline the following flaws in approaches to teaching that lead to the development of inert<br />
knowledge in higher education:<br />
<strong>Academic</strong>s tend to fill students with facts, and leave no time for dealing with topics in depth.<br />
<strong>Academic</strong>s expect students to remember dates, formulae, algorithms, quotations and whole<br />
poems, yet show little practical use for that knowledge despite the fact that the students have<br />
difficulty in transferring the knowledge.<br />
<strong>Academic</strong>s rely on decontextualized instructional strategies where the focus is on abstract basic<br />
skills, concepts and technical definitions. <strong>Academic</strong>s believe that decontextualized skills have<br />
broad applicability and are unaffected by the activities or environments in which they are acquired<br />
and used. When this happens students do not learn when to apply those skills or within what<br />
kinds of contexts they work.<br />
Students are given arbitrary, uninteresting, and unrealistic problems to solve when it comes to<br />
providing them with practice. This emphasises the point that academics mistakenly believe that<br />
they must emphasise decontextualized skills that are applicable everywhere.<br />
Additionally, students are treated passively, rarely giving the opportunity to take responsibility for<br />
their own <strong>learning</strong>, to explore ideas of their own choosing, to collaborate with one another or with<br />
academics, or to make valuable contributions to the <strong>learning</strong> of others. Hence, the students do not<br />
learn the skills necessary to become life-long learners and daily problem solvers.<br />
Furthermore, students are not evaluated in authentic ways. After teaching in decontextualized<br />
ways academics tend to use paper-pencil tests to measure the quantity of knowledge learned<br />
without examining the quality of students’ thinking and problem-solving performance.<br />
3. New <strong>learning</strong> environment for the development of PTS<br />
To be able to overcome these flaws in our current teaching and <strong>learning</strong> in higher education which<br />
produce inert knowledge, our students should operate in <strong>learning</strong> environments which prepare them<br />
for the complexities of the professional world. These <strong>learning</strong> environments should utilize instructional<br />
activities that reflect the problem-solving and challenge-meeting processes that professionals use on<br />
their jobs (Grabinger, Dunlap, & Duffield, 1997). And as Grabinger, Dunlap, & Duffield (1997) put it<br />
our higher education students need a <strong>learning</strong> environment “that places the students in the driver’s<br />
seat of the <strong>learning</strong> process – involving them in the planning, controlling and directing of <strong>learning</strong><br />
activities and the application and assessment of <strong>learning</strong> processes and outcomes. Such a <strong>learning</strong><br />
environment is what Grabinger & Dunlap (1995) refer to as rich environments for active <strong>learning</strong><br />
(REALs).<br />
Grabinger & Dunlap (1995) and Grabinger, Dunlap, & Duffield (1997) have outlined five key attributes<br />
of REALs as follows:<br />
Student responsibility and initiative – this attribute explains that REALs are “student-centred<br />
<strong>learning</strong> environments which place a major emphasis on developing intentional <strong>learning</strong> and lifelong<br />
<strong>learning</strong> skills. These skills include the abilities to construct higher-order questions to guide<br />
<strong>learning</strong>, reflect on consequences and implications of actions, and monitor and modify their<br />
personal cognitive activity. This attribute is critical because students cannot actively construct and<br />
evolve their knowledge structures without taking responsibility and initiative for their <strong>learning</strong>”.<br />
According to Grabinger & Dunlap (1995) and Scardamalia, et al. (1991) research points to<br />
passive and immature learners possessing certain characteristics that prevent them from<br />
becoming skillful problem-solvers. These characteristics which include their inability to organize<br />
their mental activities around topics ultimately result in their inability to transfer their knowledge to<br />
new problems because they have learned content and strategies in a decontextualised context.<br />
To be able to develop intentional <strong>learning</strong> and thereby promote transferability of knowledge to<br />
other context, REALs suggest reciprocal teaching strategies. According to Grabinger & Dunlap<br />
(1995), reciprocal teaching promotes social interactions, scaffolding and contextualized, holistic<br />
activities that have relevance for learners.<br />
Generative <strong>learning</strong> activities – According to Grabinger & Dunlap (1995) generative <strong>learning</strong><br />
activities require that students engage in argumentation and reflection as they try to use and then<br />
refine their existing knowledge as they attempt to make sense of alternate points of view.<br />
Students, in a generative <strong>learning</strong> situation apply the information they learn. Students are<br />
therefore, involved in creating solutions to authentic problems through the development and<br />
3
Samuel Adu Gyamfi et al.<br />
completion of projects. Generative <strong>learning</strong> activities call for the use of cognitive apprenticeship<br />
form of instruction (Grabinger & Dunlap (1995). According to Grabinger & Dunlap (1995) in<br />
cognitive apprenticeship the goal of the teacher is to help students generalize and transfer their<br />
<strong>learning</strong> through conditionalised and indexed knowledge. This is done through the presentation of<br />
a range of tasks that encourage reflection and identification of common transferable elements<br />
across tasks.<br />
Authentic <strong>learning</strong> contexts – according to Grabinger & Dunlap (1995) authenticity is important to<br />
REALs for three reasons. First realistic problems hold more relevance to students’ needs and<br />
experiences because they can relate what they are <strong>learning</strong> to problems and goals they see every<br />
day. Secondly, because the situations students encounter during <strong>learning</strong> are authentic, they<br />
develop deeper and richer knowledge structures, leading to a higher likelihood of transfer to novel<br />
situations. Finally, because complex problems require a team approach that provides natural<br />
opportunities for learners to test and refine their ideas and to help each other understand the<br />
content, it encourages collaboration and negotiation. REALs therefore, propose anchored<br />
instructional strategies to achieve authenticity. According to Grabinger & Dunlap (1995) because<br />
anchored instructional strategies encourage students to work together to solve complex problems<br />
they are exposed to multiple perspectives in an environment that gives them the opportunity to<br />
test out their ideas, solutions and processes and consequently overcome the problem of inert<br />
knowledge.<br />
Authentic assessment strategies – Authentic assessment strategies refer to the measurement of<br />
intellectual accomplishments that are worthwhile, significant and meaningful as compared to<br />
standardized tests inherent in conventional schooling (Grabinger & Dunlap (1995). Accordingly<br />
Grabinger & Dunlap (1995), quotes Wiggins (1989) as explaining that authentic tests show the<br />
intellectual design features that emphasize complexity, stress depth more than breath, include illstructured<br />
tasks or problems, and require students to contextualize content knowledge. When<br />
applying authentic assessment to student <strong>learning</strong> and achievement, a teacher applies criteria<br />
related to “construction of knowledge, disciplined inquiry, and the value of achievement beyond<br />
the school.<br />
Co-operative support – according to Grabinger & Dunlap (1995), “working in groups helps<br />
students refine their knowledge through argumentation, structured controversy and the sharing<br />
and testing of ideas and perspectives”. “Co-operative <strong>learning</strong> and problem-solving groups also<br />
address students’ needs for scaffolding during unfamiliar <strong>learning</strong> and problem-solving activities;<br />
therefore, with the support of others in the group, students are more likely to achieve goals they<br />
may not have been able to meet on their own (Dunlap, 1999). Grabinger & Dunlap (1995) suggest<br />
problem-based <strong>learning</strong> as a REAL strategy for achieving co-operative <strong>learning</strong>. In problem-based<br />
<strong>learning</strong> students work with problems in a manner that fosters reasoning and knowledge<br />
application appropriate to their levels of <strong>learning</strong>. The skills and knowledge acquired by this study<br />
are applied back to the problem to evaluate the effectiveness of <strong>learning</strong> and to reinforce <strong>learning</strong><br />
Grabinger & Dunlap (1995).<br />
The five key principles of REALs amply demonstrate their achievement in promoting the development<br />
of PTS in higher education institutions as outlined in the literature. Indeed Grabinger (1999) has<br />
looked at some research conducted with various implementations of REALs which showed positive<br />
results.<br />
Paradoxically, research has shown that lecturers in our higher educational institutions are challenged<br />
and de-motivated on a day-to-day basis with the practicalities of teaching increasing numbers of<br />
students with a decreasing unit of resources in institutions, especially where competition for funding is<br />
fierce and where there is pressure from external reviews of research and teaching performance, Kitts<br />
& Hancock (1999). Consequently, students often find it difficult to get seating spaces in the lecture<br />
halls let alone hear what the lecturer is saying. Libraries are overcrowded, books are out of date and<br />
journal holdings lag years behind due to budgetary constraints. Therefore, the acquisition function in<br />
higher educational institutions is unduly hampered in most developing economies.<br />
Excessive student numbers have also resulted in a situation whereby academics could hardly set<br />
meaningful exercises let alone mark and present feedback to students. There is also the pressure on<br />
the academics to meet research targets in order to qualify for funding which has compounded the<br />
problem of meeting students’ needs. Therefore, students hardly had time to practice what they learn.<br />
Additionally, small group tutorials and seminars necessary for discussion exercises are hardly feasible<br />
4
Samuel Adu Gyamfi et al.<br />
in many departments. Furthermore, because of the large student numbers with its attendant budget<br />
constraints, hardly do departments organise fieldtrips and practical sessions which are key ingredients<br />
for discovery <strong>learning</strong>. This situation rather promotes teacher-centred methods of practice instead of<br />
student-centred <strong>learning</strong> approach which facilitates effective PTS development (Drummond et al,<br />
1998).<br />
4. The argument for eLearning strategies<br />
Grabinger (1999) has outlined some issues that need investigation in REALs such as individual<br />
differences, learner control, scaffolding and support, <strong>learning</strong> and assessment. These are issues<br />
which it is argued theoretically, could be supported by eLearning and consequently promote the<br />
development of PTS. In order to justify the use of eLearning to facilitate the acquisition of PTS in<br />
university education, it is prudent to look at the theories that are useful for examining and<br />
understanding eLearning. In this article, much of the work on the theoretical framework of eLearning<br />
relies heavily on the work of Andrews & Haythornthwaite (2007)<br />
For the electronic aspect of eLearning Andrews & Haythornthwaite (2007) argue for the<br />
communication theory (which draws on the contemporary rhetorical theory) and social informatics as<br />
providing important perspectives. Contemporary rhetoric, according to Andrews & Haythornthwaite<br />
(2007) is ‘concerned with the relationship among three key elements namely, the speaker/writer, the<br />
audience and the subject matter. This communicative triangle enables exploration and definition of<br />
the purpose of the communicative act, as well as the possibility of investigation of the means by which<br />
the communication takes place’. Consequently, they argue that the rhetorical theory can be used to<br />
analyse communication and predict the patterns and means of communication that might be<br />
necessary in a particular situation. This theory is therefore in alignment with eLearning since<br />
eLearning takes place in a particular situation because it is contextualised.<br />
This rhetoric theory has been depicted in the adopted model in figure 1. Here <strong>learning</strong> is considered<br />
as a “dialogic and dialectical exchange, not only between the learner and the teacher, but also<br />
between the learner and the body of knowledge that is being explored” (Andrews & Haythornthwaite,<br />
2007) . The model therefore places the learner in a more powerful, active position in relation to the<br />
social dynamics of <strong>learning</strong> who can even critique the teacher’s mediation of existing knowledge<br />
(Andrews & Haythornthwaite, 2007). This supports REALs co-operative <strong>learning</strong> strategies where<br />
students work together to learn and are responsible for one another’s <strong>learning</strong> as well as their own.<br />
Working together in groups helps students refine their knowledge through argumentation, structured<br />
controversy and reciprocal teaching (Grabinger, 1999). Accordingly, co-operative <strong>learning</strong> and<br />
problem-solving activities encourage generative <strong>learning</strong> (Grabinger, 1999) which eventually leads to<br />
the development of students’ personal transferable skills.<br />
Figure 1: Model of communication (source: Andrew and Haythornthwaite, 2007)<br />
For social informatics theory, Andrews & Haythornthwaite (2007) quotes the definition of Kling,<br />
Rosenbaum, & Sawyer (2005) to buttress their point of view. “Social informatics refers to the<br />
interdisciplinary study of the design, uses, and consequences of ICTs that takes into account their<br />
5
Samuel Adu Gyamfi et al.<br />
interaction with institutional and cultural contexts” (Kling, Rosenbaum, & Sawyer, 2005) Quoting<br />
extensively from many researchers and fields that have shaped social informatics including studies<br />
about adoption and diffusion of innovations, social construction and social shaping of technology,<br />
activity theory, social networks and communities of practice, Andrews & Haythornthwaite (2007)<br />
assert that “social informatics provides another theoretical foundation for addressing eLearning<br />
particularly where it intersects with computing use by groups, organisations, communities and<br />
societies”.<br />
Additionally, Andrews & Haythornthwaite, (2007) have extended the principles of social informatics<br />
into the <strong>learning</strong> situation where they have adopted the term ‘educational informatics’. Quoting Levy,<br />
et al. (2003) they have defined educational informatics as “the study of the application of digital<br />
technologies and techniques to the use and communication of information in <strong>learning</strong> and education”.<br />
It has therefore created the opportunity for an isolated student of old to be able to carry on authentic<br />
<strong>learning</strong> with other students through class discussion boards, and emails, whispering, moulding and<br />
forming the communication dialogue they prefer” (Andrews & Haythornthwaite, 2007). Consequently,<br />
not only does the model advance the position of a student in a better relation to a community of<br />
learners and body of knowledge, but it also enables reflection to become an integrated part of the<br />
actual dialogic interaction between the participants while in the <strong>learning</strong> process (Andrews &<br />
Haythornthwaite, 2007). Provision of digital technology tools in the model can therefore be used to<br />
support cognitive processes, share the cognitive load by providing support for lower-level cognitive<br />
skills, allow learners to engage in <strong>learning</strong> activities normally out of reach and allow learners to<br />
generate test hypotheses Lajoie (1993) and consequently promote the development of PTS.<br />
5. Conclusion<br />
It has been established that for personal transferable skills development in higher education to be<br />
achieved in the traditional classrooms a new <strong>learning</strong> environment that encourages knowledge and<br />
skills transfer to other contexts must be encouraged. This paper has argued for the use of REALs. It<br />
has been established that REALs promote intentional <strong>learning</strong> by encouraging the growth of students’<br />
responsibility, initiative, decision-making and intentional <strong>learning</strong>, a constructivist <strong>learning</strong> theory that<br />
promotes the development of PTS. REALs apply dynamic, generative <strong>learning</strong> activities that promote<br />
high level thinking process (that is, analysis, synthesis, problem-solving, experimentation, creativity<br />
and examination of topics from multiple perspectives to help students integrate new and old<br />
knowledge and thereby create rich and complex knowledge structures, including creating and<br />
providing access to resources. REALs also utilise authentic <strong>learning</strong> contexts to promote study and<br />
investigation, including contextualising <strong>learning</strong> and thereby promote PTS development. Again,<br />
REALs encourage collaboration to cultivate an atmosphere supportive of knowledge building<br />
communities and reinforce reflection by embedding opportunities to reflect on the <strong>learning</strong> process as<br />
well as on the content acquired to promote both <strong>learning</strong> and meta-cognitive skills development<br />
necessary for the development of PTS. However, challenges in our current higher educational<br />
institutions demand new <strong>learning</strong> environments. The paper has consequently made a case for the use<br />
of communicative and social informatics theories in an eLearning environment to facilitate the<br />
development of PTS in higher education.<br />
References<br />
Andrews, R., & Haythornthwaite, C. (2007). Introduction to eLearning Research. In R. Andrews, & C.<br />
Haythornthwaite (Eds.), Sage Handbook of eLearning Research (pp. 1-52). London: Sage Publications.<br />
Bostock, S. J. (1998). Constructivism in Mass Higher Education: A Case Study. British Journal of Educational<br />
Technology , 29 (3), 225-240.<br />
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn: Brain, Mind, Experience and School.<br />
Washington, D.C.: National Academy Press.<br />
Bridges, D. (1995). Transferable Skills: A Philosophical Perspective. Studies in Higher Education , 20 (3), 43-51.<br />
Chadha, D. (2006). A Curriculum Model for Transferable Skills Development. Journal of the Higher Education<br />
Academy Engineering Subject Centre , 1 (1), 1-10.<br />
CTGV, (. a. (1993). Integrated Media: Toward a Theoretical Framework for Utilizing their Potential. Journal of<br />
Special Education Technology , 12 (2), 76-89.<br />
Cullen, J., Hadjivassiliou, K., Hamilton, E., Kelleher, J., Sommerlad, E., & Stern, E. (2002). Review of Current<br />
Pedagogic Research and Practice in the Fields of Post-compulsory Education and Lifelong Learning.<br />
London: The Tavistock Institute.<br />
Cunningham, D. J., Duffy, T. M., & Knuth, R. A. (2000). The Textbook of the Future. Bloomington: Centre for<br />
Research on Learning and Technology.<br />
6
Samuel Adu Gyamfi et al.<br />
Curry, P., & Sherry, R. (2004). The Hidden Value of Higher Education Learning: Transferable Skills and their<br />
Importance for Graduates of Modern Language Programmes. A Survey of Students, <strong>Academic</strong> Staff,<br />
Graduates and Employers. Dublin: Transferable Skills Project.<br />
Dearing, R. (1997). Higher Education in the Learning Society. London: HCIHE Publications.<br />
Drummond, I., Nixon, I., & Wiltshire, J. (1998). Personal Transferable Skills in Higher Education: The Problems of<br />
Implementing Good Practice. Quality Assurance in Education , 6 (1), 19-27.<br />
Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the Design and Delivery of Instruction.<br />
In D. H. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology. NY:<br />
Macmillan Library Reference USA.<br />
Dunlap, J. C. (1999). Rich Environments for Active Learning on the Web; Guidelines and Examples. WebNet 99:<br />
World Conferene on the WWW and Internet. Honolulu, Hawaii: EDRS.<br />
Gash, S., & Reardon, D. F. (1988). Personal transferable skills for the modern information professional: a<br />
discussion paper. Journal of Information Science , 15, 285-292.<br />
Grabinger, S. R. (1999). Rich Environments for Active Learning. In D. H. Jonassen (Ed.), Handbook of Research<br />
for Educational Communications and Technology (pp. 665-692). New York: Macmillan.<br />
Grabinger, S. R., & Dunlap, J. C. (1995). Rich Environments for Active Learning: A Definition. Association for<br />
Learning Technology Journal , 3 (2), 5-34.<br />
Grabinger, S. R., Dunlap, J. C., & Duffield, J. A. (1997). Rich Environments for Active Learning in Action:<br />
Problem-based Learning. ALT-J , 5 (2), 5-17.<br />
Jonassen, D. H. (1994). Thinking Technology: Toward a Constructivist Design Model. Educational Technology ,<br />
34 (3), 34-37.<br />
Justice, C., Rice, J., & Warry, W. (2009). Developing Useful and Transferable Skills: Course Design to Prepare<br />
Students for a Life of Learning. Georgia Southern University.<br />
Kemp, I. J., & Seagraves, L. (1995). Transferable Skills - Can Higher Education Deliver? Studies in Higher<br />
Education , 20 (3), 315-328.<br />
Kitts, S. A., & Hancock, J. T. (1999). Putting Theory into Practice: The Creation of REALs in the Context of<br />
Today's Universities. Association of Learning Technology Journal (ALT-J) , 7 (2), 4-14.<br />
Kling, R., Rosenbaum, H., & Sawyer, S. (2005). Understanding and Communicating Social Informatics. Medford,<br />
NJ: Infomation Today.<br />
Lajoie, S. P. (1993). Computer Environments as Cognitive Tools for Enhancing Learning. In S. Lajoie, & S. Derry<br />
(Eds.), Computers as Cognitive Tools (pp. 261-188). Hilldfslr, NJ: Erlbaum.<br />
Laurillard, D. (2002). Rethinking University Teaching: A Framework for the Effective Use of Learning<br />
Technologies (2nd ed.). Abingdon: RoutledgeFalmer.<br />
Lebow, D. (1993). Constructivist Values for Instructional Systems Design: Five Principles Toward a New Mindset.<br />
Educational Technology Research and Development , 41 (3), 4-16.<br />
Levy, P., Ford, N., Foster, J., Madden, A., Miller, D., Nunes, M. B., et al. (2003). Educational Informatics: An<br />
Emerging Research Agenda. Journal of Information Science , 29 (4), 298-310.<br />
Nabi, G. R., & Bagley, D. (1998). Graduates' Perceptions of Transferable Personal Skills and Future Career<br />
Preparation in the UK. Career Development International , 3 (1), 31-39.<br />
Nunes, M. B., & McPherson, M. (2003). Constructivism vs. Objectivism: Where is Difference for Designers of<br />
eLearning Environments? The 3rd IEEE International Conference on Advanced Learning Technologies<br />
(ICALT'03). IEEE COMPUTER SOCIETY.<br />
OECD. (2006). Year Report. Organisation of Economic Co-operation and Development.<br />
Scardamalia, M., Bereiter, C., McLean, R. S., Swallow, J., & Woodruff, E. (1991). Higher levels of Agency for<br />
Children in Knowledge building: A Challenge for the Design of New Knowledge Media. The Journal of the<br />
Learning Sciences , 1 (1), 37-68.<br />
Visser, J. (2008). Constructive Interaction with Change: Implications for Learners and the Environment in which<br />
They Learn. In J. Visser, & M. Visser-Valfrey (Eds.), Learners in a Changing Learning Landscape:<br />
Reflections from a Dialogue on New Roles and Expections (Vol. 12, pp. 11-35). USA: Springer.<br />
Wiggins, G. (1989). A True Test: Toward more Authentic and Equitable Assessment. Phi Delta Kappan (70), 703<br />
713.<br />
7
Survey of Teachers’ use of Computer/Internet in Secondary<br />
Schools in South West Nigeria<br />
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
Kwara State University, Malete-Nigeria<br />
babatunde.alege@kwasu.edu.ng<br />
stephen.afolabi@kwasu.edu.ng<br />
Abstract: The Federal Government of Nigeria developed a National Policy for Information Technology at the<br />
beginning of the new millennium in order to keep pace with the rest of the world and to meet the challenges of a<br />
rapidly changing global environment. Nine years later the Technical Working Committee Group on Information<br />
and Communication Technology submitted its report. A major propelling philosophy is to have a modern ICTdriven<br />
educational system for the effective delivery of educational services at all levels. To what extent this has<br />
been achieved remains to be investigated. The study was a descriptive survey carried out to investigate the<br />
teachers’ literacy profiles, attitudes towards computers, general integration of Information and Communication<br />
Technology and also determine the hindrances experienced by them on the integration of ICT in their teaching.<br />
Five hundred and sixty two teachers from private and public schools in southwest Nigeria were randomly<br />
selected. Their basic statistics include 58% females and 42% males with varying educational qualifications and<br />
working experiences. The questionnaire used consisted of five parts based on teachers’ general information,<br />
computer literacy, attitude, integration of ICT in their teaching and hindrances. Research questions were<br />
generated. Data collected were analysed using simple percentage. The results indicated among others that 87%<br />
of the teachers irrespective of their gender, school proprietorship and experience do not use computers in their<br />
teaching. The teachers’ tendency not to use computers was determined by their lack of expertise in using ICT,<br />
non-availability of infrastructure, equipment and software, laziness, technophobia, lack of incentives and support,<br />
and their general attitude. It was recommended among others: that computer training programmes are organized<br />
for all the teachers in Nigeria, adequate infrastructure and equipment should also be provided in each school,<br />
teachers should be well remunerated to serve as incentives and Nigeria should address ICT issue with all<br />
seriousness it deserves within the context of her education policy and implementation.<br />
Keywords: Information and Communication Technologies, Technology Integration, Computer/Internet,<br />
Secondary schools, Southwest Nigeria, Technophobia<br />
1. Introduction<br />
The end of the last millennium witnessed rapid technological advances and deep changes in several<br />
aspects of human activity which, are taken together as indicative of a shift into knowledge era. Such<br />
discussion according to Sringer, (2008) has stimulated much discussion about the role and processes<br />
of education and about the role of Information and communication technology (ICT) in teaching and<br />
<strong>learning</strong> in this new era. Presently the world is experiencing an ever-increasing use of computers in<br />
diverse areas of human endeavour. (Chika, 2004). Numerous blueprints on education reforms and on<br />
ICT in education and other sectors have also been set out by various governments since the mid-<br />
1990s. In Nigeria, Information and Communication Technology (ICT) had been widely used in the<br />
commercial and Business sectors for quite some time, particularly when Banks and other business<br />
organizations began to automate their offices with desktop computers and other accessories for<br />
financial networking transactions (Wodi,2009). The rapid rate at which ICTs have evolved since the<br />
mid 20 th century, their convergence and pervasiveness have given them an active central role in<br />
development and globalization (Nwagu, 2006; Adomi, 2010). It has been argued that ICT is the<br />
bedrock for National survival and development in a rapidly changing environment.<br />
As the world is witnessing rapid development in science and technology with the capability of<br />
changing our life for better, there is an overwhelming awareness that there are great potentials that<br />
the computer can offer when properly used in our schools. Bradly, 1996 noted that the adoption and<br />
integration of ICT in the education sector is on the increase. A major consensus among educators is<br />
that teachers should keep abreast of modern technology to integrate ICT in their teaching (Tahir and<br />
Okebukola, 1995). The teacher of the 21 st century is expected to rise up to the challenges occasioned<br />
by the new technology. Over the years, Nigeria had made several attempts to improve her education<br />
system and make it more accessible to Nigerians and compete favourably in a competitive global<br />
economic and social environment. Many reforms and strategic plans were introduced to address vital<br />
educational issues for the achievement of set goals.<br />
8
2. Statement of the Problem<br />
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
Some fifty years after independence, the education sector in Nigeria is still grappling with many<br />
challenges. These include acute underfunding, poor facilities, shortage of qualified teachers<br />
especially in core subject areas, poor teaching and <strong>learning</strong> environment, decayed infrastructure, and<br />
very low research activities in higher institutions. There is also the problem of access, mass failure in<br />
senior secondary school certificate examination, examination malpractices and cultism. Recent<br />
studies revealed that about 10 million children who are supposed to be in school were not there while<br />
current national enrolment ratio is about 57% in public schools. Between 2003 and now, there have<br />
been several new national and global initiatives. The Jomtien World Conference on Education for all<br />
(EFA) in 1990 was sponsored by the World Bank, the United Nations Educational, Scientific and<br />
Cultural Organisation (UNESCO), the United Nations Development Programme (UNDP) and the<br />
United Nations Children Emergency Fund (UNICEF) to bring education to everyone irrespective of<br />
race, colour, creed, physical disability or socio-economic status. Similarly the Millennium Development<br />
Goals (MDGs), drawn from the actions and targets contained in the Millennium Declaration is to be<br />
achieved by the end of 2015. In the past, Nigeria had made several attempts at making education<br />
accessible to Nigerians. The old western region started free education during Chief Obafemi<br />
Awolowo’s regime. In 1976, the Universal Primary Education Scheme was launched. Among the<br />
reasons adduced to its near collapse are poor management, poor infrastructure, high rate of illiteracy<br />
as a result of limited access to, poor retention in, and poor quality of basic education. As a result of<br />
this, the Federal Government re-launched the UBE programme in September 1999 as a radical<br />
intervention strategy for ensuring access to quality basic education (UBEC, 2008). A major challenge<br />
of the reform for the education sector therefore is the need to reform the curriculum so as to bring it in<br />
line with the strategic objectives of NEEDS and MDGs. To this end, the country developed a national<br />
Policy for Information Technology that would among others, have a modern ICT-driven educational<br />
system for the effective delivery of educational services at all levels. UNESCO (2004) identified a<br />
number of frameworks for setting ICT for education programmes. These include: policy and vision of<br />
ICT use in schools, technology and infrastructure, curriculum, pedagogy and content development,<br />
professional development, monitoring and supervision. (Adeosun, 2010). To what extent Nigeria has<br />
been able to meet these criteria remains to be investigated. This study therefore determines teachers’<br />
use of Computer/Internet in teaching and <strong>learning</strong> in secondary schools and examines the<br />
implementation of education policy in terms of the availability of ICT tools in schools and their general<br />
integration for the improvement of teaching and <strong>learning</strong>.<br />
2.1 Purpose of the study<br />
The study was undertaken to investigate teachers, literacy profiles, attitudes towards computers,<br />
general integration of Information and Communication Technology and also determine the hindrances<br />
experienced by them on the integration of ICT in their teaching.<br />
The study would specifically:<br />
Find out the attitude profile of the teachers in Southwest Nigeria towards the integration of ICT in<br />
their teaching practice.<br />
Find out whether teachers use Computer/Internet in their teaching.<br />
Find out the competency profile of teachers with respect to the use of various computer and<br />
communication environments.<br />
Determine the nature of hindrances experienced by teachers on the pedagogical integration of<br />
ICT.<br />
2.2 Scope of the study<br />
The focus of this research is on computer/Internet use by teachers in Southwest Nigeria. The<br />
study surveyed teachers’ use of Computer/Internet in Southwest Nigeria comprising of Ekiti, Ogun,<br />
Ondo, Osun, Oyo and Lagos states.<br />
2.3 Significance of the study<br />
This study is significant in the sense that it would evaluate teachers’ use of Computer/Internet in<br />
Nigerian secondary schools.<br />
9
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
The present crop of teachers would benefit from the study as it would serve as eye opener for<br />
them on the need to teach using information technology and also help them gain appreciable<br />
insight into the power of these technologies worldwide.<br />
The study hopes to be of benefit to curriculum developers and those charged with responsibility of<br />
implementing various educational policies in Nigeria.<br />
The authors may also benefit from the study as it may have shed light on some aspects that<br />
needed to be reviewed.<br />
The study would have implications for the improvement of the quality of education in Nigeria as a<br />
nation.<br />
3. Research questions<br />
The following questions shall guide the study:<br />
What are the attitudes of teachers toward the use of Computer/Internet as teaching tools in<br />
Southwest Nigeria?<br />
What are the competency levels of teachers on the use of Computer/Internet as teaching tools in<br />
Southwest Nigeria?<br />
To what extent do teachers in Southwest Nigeria use Computer/Internet in the instructional<br />
process?<br />
What are the barriers to ICT integration in teaching-<strong>learning</strong> process in Southwest Nigeria?<br />
3.1 Current Situation<br />
The presence and potentials of information and communication technologies in almost every human<br />
activity is overwhelming. Globalization and advances in ICT have brought about phenomenal<br />
improvements and great opportunities for developing countries to participate meaningfully in the<br />
global digital economy (Nigeria Vision 2020, 2009). The use of ICT promotes development and<br />
improves services in any organisation. It brings changes in today's business environment. In<br />
academic environment, it speeds up information delivery, facilitates teaching, <strong>learning</strong> and research.<br />
In spite of the above observation about the potentials, and benefits of using ICT, the level of<br />
awareness and use in Nigeria appears to be very minimal. (Haliso, 2011). The literature dealing with<br />
technology and pedagogy attests to the powerful impact ICT can have on the teaching and <strong>learning</strong><br />
process (John, 2010). Research indicates that level of collaboration and communication are<br />
enhanced by the use of computers for knowledge building and thinking skills (Howe et al., 1997;<br />
McFarlane, 1997).<br />
Nigeria started implementing its ICT policy in April 2001 with the establishment of National Information<br />
Technology Development Agency (NITDA) with a vision ‘to make Nigeria IT capable country in Africa<br />
and a key player in the information, using IT as the engine for sustainable development, and global<br />
competitiveness (Federal Ministry of Science & Technology, 2001). The strategic Action Plan provides<br />
for a period of 5 years for the key sectors- health, education infrastructure, human resource<br />
development, agriculture, legal/regulations, private sector/industry, media/community- as part of an<br />
integrated approach to achieving national development within the context of the Millennium<br />
Development Goals (MDGs), the Federal Government 7-point Agenda, the National Economic<br />
Empowerment Development Programme (NEEDS) and a host of other socio-economic development<br />
programmes and initiatives. According to Adeosun, (2010) the mission statement of NITDA<br />
recognized the need to use IT for education (p.iii). In addition, the general objectives (3 out of 31)<br />
focused on integrating ICT into the mainstream education and training, with a strategy to “restructure<br />
educational systems at all levels to respond effectively to the challenges and imagined impact of the<br />
information. Yusuf (2005) noted that although the mission, general objectives and strategies<br />
recognized the importance of ICT in education, the document has no sectored application to<br />
education and issues relating to education are subsumed under human resources development. ).<br />
Adeosun (2010) also pointed out that Nigeria has no particularly articulated policy for ICT in education<br />
in spite of her strong commitment to the promotion of ICT in her educational and economic goals. The<br />
National Information Technology Development Agency (NITDA) developed some standards for<br />
computer uses in schools, including students- computer ratio, in which schools are required to<br />
10
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
establish computer laboratories and classrooms equipped with interactive white boards and other<br />
teacher-aided <strong>learning</strong> tools.<br />
The education sector has been affected by ICTs worldwide. Consequently, the processes of teaching,<br />
<strong>learning</strong> and research have also been affected (Yusuf, 2005). The Federal Government of Nigeria, in<br />
the National Policy on Education (2004), recognizes the prominent role of ICTs in the modern world,<br />
and has integrated ICTs into education in Nigeria. To actualize this goal, the document states that<br />
government will provide basic infrastructure and training at the primary school. In the new 9-year<br />
Basic Education curriculum, Computer Studies/ICT has been made compulsory at the lower, middle<br />
and upper levels of education. The government also intended to provide necessary infrastructure and<br />
training for the integration of ICTs in the secondary school system. Adomi and Kpangban (2010)<br />
noted that this was not the first attempt the Nigerian government made to introduce computer<br />
education in schools. In 1988, the Nigerian government enacted a policy on computer education. The<br />
plan was to establish pilot schools and diffuse computer education innovation first to all secondary<br />
schools, and then to primary schools. Unfortunately, the project did not really take off beyond the<br />
distribution and installation of personal computers (Okebukola, 1997; cited by Aduwa-Ogiegbaen and<br />
Iyamu, 2005). Nkerenwem (1996) identified recent effort made by the Government towards the ICT<br />
Development which includes the following:<br />
Launching of the National Telecommunications policy (September, 2000)<br />
Development of a comprehensive science and Technology policy. (2001)<br />
Development and launching of the National Information Technology Policy (2001)<br />
Establishment of the National Information Development Agency (NITDA) (2001)<br />
Launching of the Nigerian Satellite Systems Programme by the National space Research and<br />
Development Agency (NASRDA) (2001)<br />
Development of the National Infrastructure Backbone (NIIB)<br />
Development and launching of the Mobile Internet Units, these are buses equipped with computer<br />
systems and other accessories with a VSAT installed in the buses for Internet access. Others are:<br />
SchoolNet Nigeria<br />
Computer-in-School project<br />
One-Laptop-Per-Child (OLPC) project<br />
Interactive Radio program; and<br />
NEPAD e-school Initiative<br />
In terms of ICT and teacher development, the National Policy on Education (2004) stipulates that<br />
teacher education shall continue to take cognizance of changes in methodology and the curriculum<br />
and teachers shall be regularly exposed to innovations in their profession. Similarly the National<br />
Policy on Teacher Education (FME, 2007) developed a vision “to produce quality, highly skilled,<br />
knowledgeable and creative teachers based on explicit performance standards through pre-service<br />
and in-service programs to raise a generation of students who can compete globally’ (p.6). The goal is<br />
to ‘ensure teachers are trained and recruited to teach world-class standards and continue to develop<br />
their competence over their entire career’ (p.6). ICT was identified as one of the conditions for the<br />
achievement of the goal, as ‘the training of teachers on strategies of collaboration, reflection on<br />
enforcement of ICT practices and action research’ (p.5). According to Hennessy et al (2010), bringing<br />
ICT into the classroom can have a considerable impact on the practice of teachers, in particular when<br />
ICT is conceptualised as a tool that supports a real change in the pedagogical approach. Not only do<br />
the teachers need to change their roles and class organisation, they also need to invest energy in<br />
themselves and their students in preparing, introducing and managing new <strong>learning</strong> arrangements.<br />
Okebukola (1997) however pointed out that the chalkboard and textbook still continue to dominate<br />
classroom activities as computer is still not part of classroom technology in more than 90 percent of<br />
Nigerian public schools. Many barriers have been attributed to lack of use of technology in the<br />
teaching and <strong>learning</strong> process. In a survey conducted by Adomi and Kpangban (2010) on the<br />
application of ICT in Nigerian schools, it was reported that limited/poor information infrastructure;<br />
lack/inadequate ICT facilities in schools; poor ICT policy/project implementation strategies and<br />
frequent electricity interruptions ranked very high as causes of low ICT application.<br />
11
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
Goshit, (2006) predicted that ICT application and use will prove beneficial in improving Nigeria's<br />
educational system and giving students a better education. A technologically-advanced workforce will<br />
lead to ICT growth in Nigeria, with the potential to improve military technology and<br />
telecommunications, media communications, and skilled ICT professionals who will be well-equipped<br />
to solve IT problems in Nigeria and other parts of the world.<br />
4. Methodology<br />
The research design used for the study is a descriptive survey design. Five hundred and sixty two<br />
(562) Secondary school teachers from public and private schools were selected by using<br />
proportionate random sampling from the six states that comprised the Southwest zone of Nigeria. Five<br />
hundred and fifty (550) teachers eventually participated in the study. Their basic statistics include 58%<br />
females and 42% males with varying educational qualifications and working experiences. The<br />
instrument used was questionnaire. Two types were used. The first one was adopted from Computer<br />
Attitude Questionnaire developed by Knezek & Miyashita, 1994, consisting of twenty two items which<br />
was used to produce a measure of the teachers’ attitude to computer/Internet. The second was<br />
adapted from a survey developed by Damiraslan and Usluel (2005) which had been used in earlier<br />
research studies and had an established content validity. It consisted of four sections to collect<br />
information on demography, computer application, frequency of use and barriers to integration of<br />
computer applications as an instructional tool. The instrument was administered on the respondents<br />
by the researchers during teacher training sessions and workshops. This enabled them to collect<br />
nearly all the questionnaires.<br />
5. Data analysis<br />
Collected surveys were analyzed using simple percentages.<br />
Table 1: Demographic analysis of Teachers’ Experience<br />
Computer<br />
Experience<br />
1-5 years 6-10 years 11-15 years 16-20 years >20 years<br />
42.6% 25.7% 23.3% 10.8.% 7.6%<br />
Background information of the teachers included computer experiences of participants. (58%).<br />
percent of the teachers responding to the survey were females. Their teaching experience varied from<br />
1-5 years to over 15 years. (Table 1.) Only 7.6% of the teachers indicated that they had experience<br />
with computers more than 20 years, while 42.6% of teachers indicated 1-5years of computer<br />
experience.<br />
5.1 Research Question 1<br />
What are the attitudes of teachers toward the use of Computer/Internet as teaching tools in Southwest<br />
Nigeria?<br />
Table 2: Teachers’ Attitude towards the use of Computer/Internet as teaching tools<br />
Computer/Internet Attitude YES% NO%<br />
1 I enjoy doing things on a computer. 21 79<br />
2 I am tired of using a computer. 23.5 76.5<br />
3 I will be able to get a good job if I learn how to use a computer. 45.3 54.7<br />
4 I concentrate on a computer when I use one. 46.2 53.8<br />
5 I enjoy computer games very much. 26 74<br />
6 I would work harder if I could use computers more often. 45.1 54.9<br />
7 I think that it takes a long time to finish when I use a computer. 34.9 65.1<br />
8 I know that computers can give me opportunities to learn many new things. 87.4 12.6<br />
9 I can learn many things when I use a computer. 46.8 53.2<br />
10 I enjoy lessons on the computer. 67.5 32.5<br />
11 I believe that the more often I use computers, the more I will enjoy school. 78 22<br />
12 I believe that it is very important for me to learn how to use a computer. 84 16<br />
12
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
13 I think that computers are very easy to use. 27.8 72.2<br />
14 I would like to study in the library rather than using a computer. 45 55<br />
15 I feel comfortable working with a computer. 38.6 61.4<br />
16 I get a sinking feeling when I think of trying to use a computer. 80.8 19.2<br />
17 Working with a computer makes me nervous. 74.3 25.7<br />
18 Using a computer is very frustrating. 76 24<br />
19 I will do as little work with computers as possible. 68.8 31.2<br />
20 Computers are difficult to use. 68 32<br />
21 Computers do not scare me at all. 69.4 30.6<br />
22 I can learn more from books than from a computer. 80.8 19.2<br />
Adapted from Part 1<br />
Adapted from Computer Attitude Questionnaire, Knezek & Miyashita, 1994<br />
Table 2 revealed that the teachers display favourable dispositions toward computers when 87.4% of<br />
the respondents admitted that computer can give opportunities to learn many new things.<br />
Interestingly, 80.8% of the respondents who reported that they get a sinking feeling when they think of<br />
trying to use a computer also believe they could learn more from books than from a computer. The<br />
outcome of this study corresponds with the finding of Namlu and Ceyhan (2002) that individuals who<br />
perceive themselves as computer introverts do have a higher level of computer anxiety and thus tend<br />
to exhibit unwelcoming attitude towards computer use. Meanwhile,<br />
5.2 Research Question 2<br />
What are the competency levels of teachers on the use of Computer/Internet as teaching tools in<br />
Southwest Nigeria?<br />
Table 3: Competency levels of teachers on the use of Computer/Internet as teaching tools<br />
Very high High Average Low Very low<br />
Proficiency in: % % % % %<br />
Word processing 3.6 5.8 43.6 38 9<br />
Spread sheets 2.8 3.5 20.3 38.8 34.6<br />
Database programs 0.8 3.6 26 69.6 45.8<br />
Graphics and drawing programs 6,9 11.2 34.8 23 24.1<br />
Desktop publishing 8.4 12.6 40 20.2 18.8<br />
Presentation programs 0.6 1.5 12.9 57 28<br />
Educational CDs 18 24.5 30.6 15.5 11.4<br />
Email 9.6 12.2 32.8 36 9.4<br />
Internet 9.8 12.8 40 24.8 12.6<br />
Table 3 shows the teachers’ competency levels on the use of Computer/Internet. The table indicates<br />
that teachers’ proficiency in the use of ICT tools is generally low. Only 9.4% of the teachers are highly<br />
proficient in word processing, 43.6% are average while 47% are lower.<br />
5.3 Research Question 3<br />
To what extent do teachers in Southwest Nigeria use Computer/Internet in the instructional process?<br />
Table 4: Computer/Internet utilization<br />
I use computer-based technology in my class for: Very Often Rarely Not at all<br />
Teaching 33 (6%) 38 (6.9%) 479 (87.1%)<br />
Assignment 52 (9.5%) 165 (30%) 333 (60.5%)<br />
Word processing 35(6.4%) 158 (28.7%) 357 (64.9%)<br />
Tutorials 22 (4%) 26 (4.7%) 502 (91.3%)<br />
Keyboarding 37 (6.7%) 133 (24.2%) 380 (69.1%)<br />
Information access 76 (13.8%) 220 (40%) 254 (46.2%)<br />
Games 43 (7.8%) 69 (12.5%) 438 (79.7%)<br />
Multimedia applications 36 (6.5%) 115 (20.9%) 439 (79.8%)<br />
Graphics 87(15.8%) 154 (28%) 309 (56.2%)<br />
13
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
I use Internet for: Very often Rarely Not at all<br />
e-mail 148 (26.9%) 114 (20.7%) 288(52.4%)<br />
Chat 36 (6.5%) 87 (15.8%) 429 (78%)<br />
Downloading instructional materials 58 (10.1. %) 165 (30%) 328 (59.9%)<br />
Giving assignments 69 (12.5%) 182 (33.1%) 299 (54.4%)<br />
News and bulletin 76 (13.8%) 86 (15.6%) 388 (70.6%)<br />
Information access via Web browsers (e.g. Google) 102 (18.5%) 69 (12.5%) 379 (68.9%)<br />
Games 43 (7.8%) 136 (24.7%) 371 (67.5%)<br />
File transfer 25 (4.5%) 200 (36.4%) 325 (59.1%)<br />
In Table 4, the results show that only 6% of the teachers use computer-based technology for their<br />
teaching very often while 87.1% do not use it at all. There is a clear indication that lack of access<br />
have accounted for the low utilization of computers in many Nigerian secondary schools.<br />
5.4 Research Question 4<br />
What are the barriers to ICT integration in teaching-<strong>learning</strong> process in Southwest Nigeria?<br />
Table 5<br />
Causes of low level/barriers to ICT integration No. Percent<br />
<strong>Limited</strong>/poor information infrastructure 440 80<br />
Lack of/inadequate inadequate ICT facilities in schools 460 85<br />
Frequent electricity interruption 275 50<br />
Non integration into the school curriculum 143 26<br />
Poor ICT policy/project implementation strategy 400 73<br />
Inadequate ICT manpower in the schools 505 92<br />
High cost of ICT facilities/components 380 69<br />
<strong>Limited</strong> school budget 275 50<br />
Lack of/limited ICT skills among teachers 500 91<br />
Lack of/poor perception of ICTs among teachers and administrators 330 60<br />
Inadequate educational software 375 68<br />
Technophobia 405 74<br />
Laziness 330 60<br />
Poor management on the parts of school administrators and government 50 9<br />
Lack of maintenance culture 35 7<br />
Lack of interest in ICT application/use on the part of the school 220 40<br />
N=550<br />
Adapted from Adomi & Kpangban Causes of low level of ICT application in Nigerian high schools<br />
Table 5 shows the teachers’ responses to the barriers experienced by them for not integrating ICT in<br />
the teaching-<strong>learning</strong> process. Top on the list is inadequate ICT manpower in the schools as reported<br />
by 92% of the teachers. Lack of/limited ICT skills among teachers came second with 91%. The<br />
facilities in schools are not adequate as reported by 85 of the teachers. Only 9% reported that poor<br />
management on the parts of school administrators and government is responsible for the barriers.<br />
6. Discussion<br />
As Chika (2004) pointed that the world is experiencing an ever-increasing use of computers in diverse<br />
area of human endeavour. There is evidence of this in Nigeria too, as governments at both the<br />
Federal and State levels vote much money to supply computer equipments into secondary schools.<br />
Yet, the proficiency of the teachers has been revealed by the study to be at its low ebb. However, it is<br />
evident, that computer is now in Nigerian schools to affirm the position of Nwagu (2006) and Adomi<br />
(2010) that the rate at which the countries of the world accept the use of computer is a major factor for<br />
development.<br />
The findings of this survey have shown that there is still a lot to be done in terms of human<br />
development, provision of facilities and monitoring of schools to ensure strict compliance with<br />
educational policies and practices.<br />
Nigeria realised the positive impact that Information and Communication technology can have on<br />
teaching and <strong>learning</strong> and have made several efforts to integrate it into the primary and secondary<br />
14
Babatunde Alabi Alege and Stephen Olufemi Afolabi<br />
school systems. There remains a concerted effort to remove all barriers to effective use of these<br />
technologies in Nigerian schools.<br />
Recommendations<br />
All the schools should be provided with computers and adequate infrastructure to facilitate the good<br />
intention of the government.<br />
Teachers should be adequately trained and retrained to improve on their ICT capabilities while<br />
teachers that are computer literate should be posted to every school to impart ICT skills to students.<br />
All other staff should be encouraged and given adequate incentives to improve on their ICT literacy<br />
levels.<br />
Nigeria should ensure that teachers are well remunerated like their counterparts in other sectors. The<br />
government should remove all forms of discriminations in form federal, state and local government<br />
workers/teachers so that all teachers are placed on equal pedestal...<br />
References<br />
Abolade A.O. and Yusuf M.O. (2005) Information and Communication Technologies (ICTs) and the Nigerian<br />
Teacher Education Programme 1<br />
African Journal of Educational Studies, Vol.3 No.1, p. 1<br />
Adebowale, O.F, Adediwura, A.A & Bada, T. A. (2009) Correlates of Computer Attitude among Secondary<br />
School Students in Lagos State, Nigeria International Journal of Computing and ICT Research, Vol. 3, No.<br />
2, pp. 20 - 30. http://www.ijcir.org/volume3-number2/article3.pdf<br />
Adeosun, O. (2010) Quality Basic Education Development in Nigeria: Imperative for Use of ICT<br />
http://digitalcommons.unl.edu/libphilprac/345<br />
Adomi, E.E. & Anie, S.O. (2006) An Assessment of Computer Literacy Skills of Professionals in Nigerian<br />
University Libraries. Library Hi Tech News Vol.23 No 2.pp.10-14<br />
Anderson, N. & Baskin, C. (2002). Can we leave it to chance? New <strong>learning</strong> technologies and the problem of<br />
professional competence. International Education Journal, 3 (3), 126 – 137 (Electronic Version), retrieved<br />
January 24 th , 2004, from http://iej.cjb.net/<br />
Esharenana E., Adomi and Kpangbany, E. (2010) Application of ICTs in Nigerian<br />
Secondary Schools Library Philosophy and Practice (e-journal) CICE Hiroshima University, Journal of<br />
International Cooperation in Education, Vol.13 No.2 (2010) pp.193 ~ 211<br />
Federal Republic of Nigeria (2001). Nigerian national policy for information technology. Abuja:<br />
Federal Republic of Nigeria (2004). National Policy on Education (4 th ed.). Lagos: NERDC.<br />
Goshit, T. (2006). Nigeria's need for ICT: SP. 259 technology and policy in Africa. Available:<br />
http://ocw.mit.edu/NR/rdonlyres/Special-Programs/SP-259Spring-2006/891209EE-E63B-4617-<br />
BA9D7635A63C754B/0/goshit.pdf<br />
Hennessy, S, Harrison, D & Wamakote, L. (2010) Teacher Factors Influencing Classroom Use of ICT in Sub-<br />
Saharan Africa Itupale Online Journal of African Studies, 2<br />
Knezek, G & Miyashita (1994) Survey of Teachers’ Attitude Toward Computers: Texas Center for Educational<br />
Technology<br />
National Information Development Technology (2008) ICT 4D Strategic plan for Economic Sector of Nigeria<br />
NITDA (2003). Use IT: National Information Technology Development Agency, Abuja Nigeria. Available:<br />
http://www.nitda.gov.ng/use_it.htm.<br />
NITDA. Retrieved 23 rd of January, 2004, from http://www.nitda.gov.ng/docs/policy/ngitpolicy.pdf<br />
Nigeria Vision 2020 (2009) Report of the Vision 2020 National Technical Working Group on Information and<br />
Communication Technology.<br />
Nwagwu, W.E. (2006). Integrating ICTs into the globalization of the poor developing countries. Information<br />
Development 22 (3): 167-179.<br />
Ocak, M.A., and Akdemir O. (2008) An Investigation Of Primary School Science Teachers’ Use Of<br />
Computer Applications The Turkish Online Journal of Educational Technology – TOJET October 2008 ISSN:<br />
1303-6521 volume 7 Issue 4 Article 6<br />
Oduroye, A.P. (n.d.) Challenges of <strong>learning</strong> and teaching with computers. Available:<br />
http://www.itnetwork.org.uk/56.htm<br />
Okebukola, P. (1997). Old, new, and current technology in education. UNESCO Africa 14 (15): 7-18.<br />
Okebukola, P. (2004). E-<strong>learning</strong> in varsities, others underway, NUC boss lists strategies. The Guardian (12<br />
October): 35, 39.<br />
Osei Tutu Agyeman2007) ICT for Education in Nigeria<br />
Sam E. O. Aduwa-Ogiegbaen, and Uwameiye,R. (2006) Internet Usage Among University<br />
Lecturers in Southern Nigeria: International Journal of Information and Communication Technology Education,<br />
Vol 2, Issue 1 edited by Lawrence Tomei<br />
15
Issues and Challenges in Implementing eLearning Projects<br />
in Higher Education: The Case of Jordan<br />
Hussein Al-Yaseen, Saheer Al-Jaghoub and Nidal Al-Salhi<br />
Al-Ahliyya Amman University, Amman, Jordan<br />
hyaseen@ammanu.edu.jo<br />
saljaghoub@ammanu.edu.jo<br />
nalsalhi@ammanu.edu.jo<br />
Abstract: Jordanian universities have started to adopt eLearning projects aiming to improve the effectiveness<br />
and efficiency of the educational process for both instructors and learners. This paper presents the results of a<br />
preliminary study that aims to identify the main issues and challenges facing these universities in implementing<br />
their eLearning projects. The data were collected using a survey to investigate the main factors affecting<br />
eLearning projects in Jordanian universities. The results of this study showed that the most important issues and<br />
challenges faced by Jordanian universities are human, cultural, regulatory, support, technical and financial<br />
respectively. These findings are hoped to be useful for researchers and policy makers as they provide some<br />
insights to universities in Jordan and other developing countries about the main issues and challenges that may<br />
be faced when planning for and developing their own eLearning systems.<br />
Keywords: eLearning, challenges, higher education, developing countries, Jordan<br />
1. Introduction<br />
eLearning (electronic <strong>learning</strong>/ distance <strong>learning</strong>) concept is argued to be one of the most important<br />
recent developments in information systems (Wang, 2003). It is a <strong>learning</strong> trend that challenges the<br />
traditional concepts of education. eLearning has been viewed as identical with web-based <strong>learning</strong>,<br />
Internet-based training, advanced distributed <strong>learning</strong>, web-based instruction, online <strong>learning</strong> and<br />
open <strong>learning</strong> (Khan, 2001). It is about the delivery of course content through electronic media, such<br />
as Internet, Intranets, Extranets, satellite broadcast, audio/video tape, interactive TV, and CD-ROM<br />
(Urdan & Weggen, 2000). eLearning system (models) can be asynchronous (real time) or<br />
asynchronous (anytime and/or anywhere). Synchronous <strong>learning</strong> can supports live; oral; visual<br />
communications between instructors and students. Synchronous <strong>learning</strong> can be attained by using<br />
the audio systems to support oral communications or by using of interactive keypad devices to<br />
support the exchange of data and voice or by using of the videoconferencing technologies. In<br />
asynchronous <strong>learning</strong> the communication between instructors and students occurs at a different time<br />
and can be achieved using text material; recorded video and recorded audio (Holden and Westfall,<br />
2010).<br />
eLearning is argued to have many benefits that can be delivered, such as: cost savings and resource<br />
efficiency, flexibility and accessibility, ease of renew and update, joint/collaborative <strong>learning</strong>,<br />
scalability, student achievement, inclusion, and participation (Alexander, 2001; Marengo and<br />
Marengo, 2005; Brady et al., 2010; Paechter et al., 2010). These benefits have caused many<br />
universities in both developed and developing countries to move towards implementing eLearning<br />
projects. For example, the UK market for eLearning remains the largest European market (Patterson<br />
et al., 2010). The US market for self-paced eLearning products and services reached $18.2 billion in<br />
2010; the revenues generated by eLearning are expected to double every year in the United States,<br />
while in the Asian Pacific region, it is worth US$233 million (Ambient Insight Report, 2010).<br />
A number of developing countries have started implementing a number of eLearning projects<br />
(Omidinia et al., 2010), but the level of these projects vary among countries because of the<br />
differences in a number of factors such as accessibility, income, and ICT infrastructure (Dadzie,<br />
2009). However, the ease of access to education provided by ICTs makes it a viable option to provide<br />
better education to people who may have been otherwise deprived from such opportunities (Casal,<br />
2007). For developing countries, better access to education provided by eLearning constructed an<br />
important means to achieve socio-economic development and fight poverty (Khan and Williams,<br />
2006).<br />
For Jordan in particular, there is a strong interest in education, given the country’s lack of natural<br />
resources and its dependence on human resources. As John Chambers, CEO of Cisco Systems said:<br />
“No country in the world has done what Jordan did in such a short period of time... Jordan is<br />
16
Hussein Al-Yaseen et al.<br />
improving by using technology to implement strategies and to achieve dreams”. Because of the<br />
promised benefits of eLearning and the increasing need in Jordan for new opportunities to provide<br />
higher education for students, a number of Jordanian universities have started implementing<br />
eLearning projects. However, investing in such projects requires resources, is faced by many<br />
challenges and is influenced by a number of factors. Therefore, this paper aims to identify the main<br />
challenges facing universities undergoing such projects.<br />
The paper is structured as follows: the introduction is followed by a brief review of the issues and<br />
challenges in implementing eLearning projects in higher education. In Section 3 The higher education<br />
system in Jordan is briefly presented. Section 4 discusses the research approach, followed by a<br />
discussion of the main findings in Section 5. Finally, in Section 6 the main conclusions and<br />
recommendations of the paper are discussed.<br />
2. Issues and challenges in implementing eLearning projects in higher<br />
education<br />
eLearning may offer many benefits, yet its adoption by universities is faced by a number challenges<br />
and obstacles that need to be identified and tackled in order to realise the promised benefits. Previous<br />
studies have identified six main challenges in implementing such projects, which are briefly presented<br />
below.<br />
The first challenge is Legislation of eLearning in higher education: which refers to the extent by which<br />
the legislations set by the ministry of higher education support the adoption of eLearning because<br />
rules and regulations can act as enablers or inhibitors for eLearning (Rezaei Mood, 2006). Second,<br />
awareness of using eLearning content and its efficiency and effectiveness amongst learner groups<br />
(students): this is argued to be an important factor that will affect learners’ rate of adoption (Tham and<br />
Werner, 2002; Tyan, 2003 and Rezaei Mood, 2006). eLearning is argued to be a student-centred way<br />
of delivering education and therefore, if the main parties involved, especially students are not fully<br />
aware of the benefits of online <strong>learning</strong> the success of such programmes becomes questionable.<br />
Third, management support of developing and adopting eLearning in higher education: the<br />
acceptance of eLearning methodology by decision makers in higher education increases and<br />
supports the success of eLearning projects (Murphy and Terry, 1998; Omidi Najafabadi et al., 2008).<br />
Generally speaking, for any project to be successful, management support is essential and eLearning<br />
projects are no exception especially that implementing such programmes requires many changes in<br />
the traditional methods of teaching and <strong>learning</strong>, training for educators and students and resources.<br />
Therefore, management support is necessary for providing such requirements. Fourth, connectivity<br />
and speed of downloading eLearning content: having a successful eLearning programme requires<br />
efficient internet connections and high rates of internet diffusion to enable students to realise the<br />
benefits of online education, which will contribute to a high rate of adoption (Zhang et al., 2002;<br />
Cantoni et al., 2004). Fifth, quality of e-content and availability of expertise that are responsible for<br />
developing a good e-content combined with sufficient financial resources will lead to low or high<br />
quality eLearning content which in effect will impact the learner’s rate of adoption (Andersson, 2008).<br />
Sixth, language barrier: the widespread use of English language in eLearning content is one of the<br />
major issues that could affect the rate of eLearning adoption especially in non-English speaking<br />
countries (Rezaei, 2006; Andersson, 2008). Therefore, universities in such countries have to create<br />
their own content for students.<br />
3. Higher education in Jordan<br />
Before moving on to discuss eLearning in Jordan, it may be useful to briefly present the country’s<br />
higher education system. The Ministry of Education (MoE) is responsible for the pre-primary, primary<br />
and secondary levels of education, whilst, the post-secondary education is the responsibility of the<br />
Ministry of Higher Education and Scientific Research (MoHESR). This Ministry includes the Higher<br />
Education Council and the Accreditation Council. The MoHESR has outlined a National Strategy for<br />
Higher Education for the years 2007-2012 (MoHE, 2011).<br />
Access to higher education is open to holders of the General Secondary Education Certificate (or its<br />
equivalent) who can then choose between private community colleges, public community colleges or<br />
universities (public and private). The credit-hour system, which entitles students to select courses<br />
according to a study plan, is implemented at universities. The higher education system of the country<br />
has evolved considerably in the past years, but still a lot needs to be done to keep up with a rapidly<br />
growing knowledge based economy that Jordan aims to be transformed into. In the years between<br />
17
Hussein Al-Yaseen et al.<br />
2000/2001 and 2006/2007, Jordan has seen an increased demand for higher education with<br />
enrolments growing at an annual rate of 14 percent from 77,841 to 218,900 students (MoHE, 2011).<br />
The first Jordanian university, the University of Jordan, was established in 1962. Today there are ten<br />
public universities. In the past, higher education was the exclusive duty of government, but the role of<br />
the state in this sector decreased with the establishment of the first private university, Amman Al-<br />
Ahliyya University, in 1990. The emergence of private universities was a necessary development to<br />
face the problem of the lack of university seats and to solve the problem of increasing demand.<br />
Privatization of higher education has increased gradually and today there are seventeen private<br />
universities. Foreign universities also became common in higher education. (e.g., German Jordanian<br />
University, established in 2005, Arab Open University established in 2002 to provide continuous<br />
education for all ages, and New York Institute of Technology).<br />
The use of ICTs as a catalyst for development coupled with the importance of education for<br />
Jordanians has led to the start of a number of eLearning initiatives both at school and university<br />
levels. The benefits of eLearning in addition to the need to meet the increasing demand for higher<br />
education in Jordan has motivated most Jordanian universities to invest in some sort of an eLearning<br />
system in order to support “blended <strong>learning</strong>” or pure “on-line <strong>learning</strong>” modules. However, these<br />
projects have faced a number of challenges. Our aim in this paper is to identify some of the issues<br />
and challenges facing Jordanian universities which have adopted eLearning projects.<br />
4. Research approach<br />
This research follows the quantitative (survey) approach at this preliminary stage of work. A<br />
questionnaire was developed based on interviews with some experts and previous literature. The<br />
questionnaire was then revised with the help of experts with significant experience in eLearning<br />
including academics from Jordanian universities. The questionnaire included fixed-choice questions<br />
about the main issues and challenges of eLearning in higher education. A 5 point Likert scale ranging<br />
from 1 as strongly disagree to 5 as strongly agree was used for the measurement; and 0 as not<br />
applicable. Data collection for this research took place in May 2011.The questionnaire was sent by<br />
email to the people who are directly involved in developing and implementing eLearning projects in all<br />
the public and private universities in Jordan.<br />
The data was analyzed using a combination of the parametric statistical methods: Descriptive<br />
Analysis and Factor Analysis (Pett et al., 2003).<br />
A factor analysis technique was employed in order to identify possible categories. Factor analysis was<br />
performed in three steps (following Berthold and Hand, 2003):<br />
A matrix of correlation coefficients for all possible pairings of the variables was generated.<br />
Factors were then extracted from the correlation matrix using principal factors analysis.<br />
The factors were rotated to maximize the relationships between the variables and some of the<br />
factors and minimize association with others using Varimax Kaiser Normalization, which<br />
maintained independence among the mathematical factors. The Eigenvalues determined which<br />
factors remained in the analysis. Following Kaiser’s criterion, factors with an Eigenvalue of less<br />
than 1 were excluded. A Screen plot provides a graphic image of the Eigenvalue for each<br />
component extracted.<br />
5. Discussion of findings<br />
The questionnaire was distributed by e-mail to all public and private universities in Jordan (10 public<br />
and 17 private universities). From 10 questionnaires sent to the public universities, 6 useable<br />
questionnaires have been returned with a response rate of 60%, and from 17 questionnaires sent to<br />
the private universities, 12 useable questionnaires have been returned with a response rate of 70.5%.<br />
This makes the total response rate from all Jordanian universities 66.7%; which is considered to be<br />
above expectation given that the generally accepted average responses to non-incentive based<br />
questionnaires are around 23%. This response rate indicated that the respondents in both<br />
public/private universities found the topic interesting and relevant, and more interesting in the private<br />
universities where the competition rate is higher.<br />
The results of descriptive statistics showed that most universities in Jordan have used an open-source<br />
<strong>learning</strong> management system such as (Moodle or Blackboard). 40% of the public universities have an<br />
18
Hussein Al-Yaseen et al.<br />
independent eLearning centre that is responsible for developing and managing eLearning projects;<br />
while 70% of the private universities have their eLearning centre.<br />
Based on factor analysis of the questionnaire, the most important issues and challenges in<br />
implementing eLearning projects in higher education in the Jordanian universities are discussed<br />
below: (see Appendix 1 for a detailed description of these factors referred to as ICIe-LHE).<br />
Human: the findings showed that the human factor was the most important issue and challenge that<br />
hinder the success of implementing eLearning projects at private and public universities. Human<br />
factor is highly correlated with seven variables as factor analysis showed. The most two important<br />
variables were the perceived threat of eLearning to instructors’ career and the lack of sufficient<br />
experts in eLearning. The less important variables for this factor were lack of training for both<br />
educator and learner and lack of relationship between instructors and students. This may be due to<br />
the fact that some of the instructors do not have sufficient computer skills and/or see the use of<br />
technology as a way by which they will be replaced.<br />
Cultural: this factor is highly correlated with five variables. The most two important variables were not<br />
taking the eLearning project seriously form both developers and learners and the problem related to<br />
the users’ culture toward eLearning, while the least two variables were the problems related to group<br />
working and the negative preconceptions of users towards eLearning. Such factor could be related to<br />
the traditional face to face methods of teaching used at universities. The soft skills in general are a<br />
problem facing many graduates that need further training on such skills. For some faculty members, it<br />
may be difficult to change the way they were teaching for many years, as eLearning is a new trend<br />
that requires new teaching cultures (Uhomoibhi, 2006) and for some students changing the way they<br />
were taught was also difficult, which has also been the case for students elsewhere (O'Donoghue et<br />
al., 2003)<br />
Regulations and management: this factor is highly correlated with five variables. The most two<br />
important variables were lack of strategic vision in developing eLearning projects and the unfamiliarity<br />
of top management with eLearning applications. However, the least two important variables for this<br />
factor were lack of intellectual property right and lack of network security regulations. The lack of<br />
vision could be related to the fact that some universities actually started an eLearning project because<br />
it is a new trend rather than being actually related to the organisations’ vision. In terms of regulations,<br />
intellectual property right laws have been enforced in Jordan since 2002. However, the major obstacle<br />
is that according to the laws and regulations of the Ministry of Higher Education (MoHE) offering<br />
online degrees is still not permitted as a certain percentage of any course has to be offered face to<br />
face.<br />
Support: this factor is highly correlated with four variables. Lack of appropriate support services for<br />
users and limited of lack of incentives for using eLearning system were found the most important<br />
variables for this factor. Lack of appropriate support services for maintaining eLearning equipments<br />
and problem related to the course starting were found the least important variable to this factor. These<br />
challenges underline the importance of having an incentive system to encourage instructors and<br />
students to adopt eLearning, especially that it requires change in the teaching methods which is time<br />
consuming and requires effort especially at the start-up phase<br />
Technical: was found to be one of the least important challenges for in implementing eLearning<br />
projects at the universities. The technical factor is highly correlated with eight variables. The most two<br />
important variables were lack or limited access to appropriate contents and the low bandwidth, while<br />
lack of hardware and lack of appropriate infrastructure were found to be the least important variables.<br />
Appropriate content may be related to the language barrier, which makes it necessary for universities<br />
to create their own content in subject areas that are taught in Arabic. Infrastructure is not a major<br />
issue given the general enhancements in technological infrastructure in Jordan since the start of the<br />
ICT related initiatives in the country in 1999.<br />
Financial: is also one of the least important challenges. This factor is highly correlated to seven<br />
variables. The most two important variables were lack of budget for setting up eLearning and the cost<br />
of upgrading the e-content, while, the least two important variables were cost of upgrading the system<br />
and cost of maintaining the system. Given that some <strong>learning</strong> management systems are open source<br />
(such as Moodle) contributes to the fact that the financial factor is not a major issue. The start-up<br />
19
Hussein Al-Yaseen et al.<br />
phase involves investments considering that many universities established an eLearning centre as<br />
part of their projects. However, this may not be a major obstacle especially in private universities.<br />
6. Conclusion<br />
In Jordan, eLearning adoption level is still at its early stages. In order to achieve eLearning benefits in<br />
higher education it is important for universities and government to work together to resolve the issues<br />
and challenges discussed above. By analyzing the findings of this paper, we can propose some<br />
recommendations that may be useful to universities and decision makers in implementing eLearning<br />
projects.<br />
eLearning approach to use: Jordanian universities are still at the early stages of transition between<br />
the traditional and eLearning modes. It is proposed that the blended approach is more suitable (faceto-face<br />
and online interactions). Using blended approach may be a successful delivery mode and<br />
could contribute to overcoming many of the issues and challenges such as improving the relationship<br />
between learners and students, increasing of the understanding level of the advantages of eLearning,<br />
increasing level of awareness about knowledge and skills needed for using eLearning, and change<br />
management. Furthermore, blended <strong>learning</strong> is allowed by current legislations of the MoHE while fully<br />
online courses are not.<br />
Awareness: Generally there is still a lack of awareness amongst the main stakeholders (instructors<br />
and students) of the effectiveness of eLearning. Many instructors feel that the traditional <strong>learning</strong><br />
mode is better. Workshops and training courses need to be set up to increase the awareness level of<br />
top management, instructors and students of eLearning. Such awareness will overcome some of the<br />
obstacles such as feeling that eLearning will threat instructors’ role, problems related to the culture of<br />
eLearning users, not taking eLearning seriously by users, the negative preconceptions of users and<br />
universities towards eLearning.<br />
Top management support: Decreasing the unfamiliarity of decision makers in higher education<br />
institutions about eLearning advantages will contribute to increasing the rate of adoption between<br />
users and decrease the effect of some problems related to culture of developers, educators and<br />
learners towards eLearning systems.<br />
Legislations of eLearning in higher education: Legislations of eLearning in higher education in<br />
Jordan restrict adoption level of eLearning systems in Jordanian universities as the fully online<br />
certificates are still unrecognized by the Ministry of Higher Education (MoHE) regardless of the quality<br />
and source of these certificates. It is proposed to establish a national committee, which represents the<br />
various stakeholders such as representatives from the ministry of higher education, ministry of<br />
education, ministry of information and communication technology, and representatives from both<br />
private and public universities. The role of this committee is to review the legislation of distance<br />
<strong>learning</strong> in developed and developing countries; develop regulations and standards for eLearning<br />
systems; establish accreditation policies for online certificates; increase the awareness level of<br />
eLearning in higher education; encourage and support eLearning in higher education; monitor and<br />
control eLearning practices in higher education; train and develop eLearning content experts;<br />
establish of standards of quality e-content; encourage eLearning knowledge sharing among higher<br />
education institutes in Jordan.<br />
Finally, eLearning is hoped to play an important role in improving the quality of education and provide<br />
<strong>learning</strong> opportunities to people who may otherwise be deprived from such opportunities. In order to<br />
improve the effectiveness and efficiency of eLearning practices in Jordan, ministries related to<br />
education need to move towards a higher level of eLearning development which will require<br />
establishing a strategic vision in developing eLearning and more effort especially technically and<br />
financially. Issues like human, legislation, culture appear to be the major challenges of adopting<br />
eLearning projects in higher education institutes in Jordan and need attention in the deployment of<br />
distance <strong>learning</strong>. Furthermore, in order to realize the full benefits of eLearning, the decision makers<br />
in the government need to play an active role in supporting and promoting eLearning practices and<br />
initiatives adopted by higher education institutes in Jordan. The research findings may be useful as<br />
they provide a clear insight to other developing countries about the main issues and challenges that<br />
face the adoption of eLearning projects in Jordanian universities when planning for and developing<br />
their eLearning systems.<br />
20
Hussein Al-Yaseen et al.<br />
7. Appendix 1: Issues and challenges of implementing eLearning projects in<br />
higher education in Jordan<br />
Factor Code/Variable Code Description Eigenvalue<br />
ICIe-LHE1 Lack of budget for setting up eLearning<br />
0.967<br />
ICIe-LHE2 High cost of buying hardware 0.982<br />
Financial<br />
ICIe-LHE3<br />
ICIe-LHE4<br />
High cost of buying software<br />
High cost of access to internet<br />
0.991<br />
0.986<br />
ICIe-LHE5 Cost of maintaining the system 0.950<br />
Technical<br />
Support<br />
Regulations &<br />
Management<br />
Culture<br />
Human<br />
References<br />
ICIe-LHE6 Expense of upgrading the system 0.942<br />
ICIe-LHE7 Expense of upgrading the content 0.972<br />
ICIe-LHE8 Lack of appropriate infrastructure 0.870<br />
ICIe-LHE9 Lack of hardware 0.855<br />
ICIe-LHE10 Lack of software 0.886<br />
ICIe-LHE11 Low bandwidth 0.898<br />
ICIe-LHE12 <strong>Limited</strong> access to the educational<br />
software<br />
0.889<br />
ICIe-LHE13 Inequality in access to equipments 0.884<br />
ICIe-LHE14 Speed of changes in computer<br />
technology<br />
0.842<br />
ICIe-LHE15 Lack or limited access to appropriate<br />
contents<br />
0.899<br />
ICIe-LHE16 Lack of appropriate support services for<br />
user<br />
0.880<br />
ICIe-LHE17 <strong>Limited</strong> or lack of incentives (for using<br />
eLearning system)<br />
0.902<br />
ICIe-LHE18 Lack of appropriate support services for<br />
maintains equipments<br />
0.932<br />
ICIe-LHE19 Problem related to the establishing<br />
courses<br />
0.926<br />
ICIe-LHE20 Lack of strategic vision in development<br />
eLearning<br />
0.936<br />
ICIe-LHE21 Unfamiliarity of top manager with<br />
eLearning applications<br />
0.951<br />
ICIe-LHE22 Lack of intellectual property right 0.951<br />
ICIe-LHE23 Lack of appropriate rules for using<br />
eLearning<br />
0.941<br />
ICIe-LHE24 Lack of network security 0.912<br />
ICIe-LHE25 Negative preconceptions of user towards<br />
eLearning<br />
0.977<br />
ICIe-LHE26 Negative preconceptions of<br />
organizations towards eLearning<br />
0.978<br />
ICIe-LHE27 Problem related to group working 0.982<br />
ICIe-LHE28 Not taking eLearning seriously by people 0.946<br />
ICIe-LHE29 Problem related to the culture of users 0.919<br />
ICIe-LHE30 Threat to instructor 0.922<br />
ICIe-LHE31 Require new knowledge and skills 0.973<br />
ICIe-LHE32 Lack of adequate experts in eLearning 0.869<br />
ICIe-LHE33 Not understanding about advantages<br />
and disadvantages of eLearning<br />
0.894<br />
ICIe-LHE34 Lack of preparatory training for educator 0.865<br />
ICIe-LHE35 Lack of preparatory training for learner 0.973<br />
ICIe-LHE36 Lack of relationship between instructors<br />
and students<br />
0.974<br />
Alexander, S. (2001) eLearning developments and experiences, Journal of Education and Training, Vol. 43, No.<br />
4, Pp. 240-8.<br />
Ambient Insight Report, 2010 [On-line], available: http://www.ambientinsight.com/Reports/eLearning.aspx<br />
Andersson, A. (2008) Seven major challenges for eLearning in developing countries: Case study eBIT, Sri Lanka,<br />
International Journal of Education and Development using ICT, Vol. 4, No. 3. Available online at:<br />
http://ijedict.dec.uwi.edu/viewarticle.php?id=472.<br />
21
Hussein Al-Yaseen et al.<br />
Berthold, M. and Hand, D. J. (2003) Intelligent Data Analysis, 2nd Ed., Springer-Verlag, Berlin.<br />
Brady, K., Holcomb, L., and Smith, B. (2010) The Use of Alternative Social Networking Sites in Higher<br />
Educational Settings: A Case Study of the ELearning Benefits of Ning in Education, Journal of Interactive<br />
Online Learning, Vol. 9, No. 2, Pp. 151-170.<br />
Cantoni, V., Cellarer, M. and Porta, M. (2004) Perspective and challenges in eLearning: towards natural<br />
interaction paradigms, Journal of visual languages and computing, Vol. 15, Pp. 335-345.<br />
Casal, C. (2007) ICT for education and development, Info, Vol. 9, No. 4, Pp. 3-9.<br />
Dadzie, P. (2009) ELearning and E-Library Services at the University of Ghana: prospects and challenges,<br />
Information Development, Vol. 25, No. 3, Pp. 207-217<br />
Holden, J. and Westfall, P. (2010), An Instructional Media Selection Guide for Distance Learning – Implications<br />
for Blended Learning Featuring an Introduction to Virtual Worlds, United States Distance Learning<br />
Association, 2nd edition.<br />
Khan, B. H. (2001). A framework for web-based <strong>learning</strong>, Engelwood CliVs, NJ: Educational Technology<br />
Publications.<br />
Khan, H., Williams, J. (2006) Poverty Alleviation through Access to Education: Can ELearning Deliver.<br />
U21Global. Graduate School for Global Leaders.<br />
[Online].http://www.u21global.com/PartnerAdmin/ViewContent?module<br />
Marengo, A. and Marengo, V. (2005) Measuring the Economic Benefits of ELearning: A Proposal for a New Index<br />
for <strong>Academic</strong> Environments, Journal of Information Technology Education, Vol. 4, Pp. 329-346.<br />
Murphy, T.H. and Terry, H.R. Jr. (1998) Opportunities and obstacles for distance education in agricultural<br />
education, J. Agric. Edu., Vol. 39, No. 1, [online], http://pubs.aged.tamu.edu/jae/pdf/vol39/39-01- 28.pdf.<br />
Najafabadi, M., Hosseini, J. and Mirdamadi, M. (2009) A Case Study of Requirements and Challenges of<br />
Information and Communication Technology System to Train Private Agricultural Insurance Brokers,<br />
American-Eurasian J. Agric. & Environ. Sci., Vol. 6, No. 2, Pp. 152-159.<br />
O'Donoghue, J., Singh, G. and Handy, D. (2003) Higher education – IT as a catalyst for change, On the Horizon,<br />
Vol. 11, No. 3, Pp. 23-28.<br />
Omidi Najafabadi, M., Farajollah Hosseini, J., Mirdamadi, M. and Moghadasi, R. (2008) Designing an Efficient<br />
Information and Communication Technology (ICT) System to Train Private Agricultural Insurance Brokers in<br />
Iran, Australian Journal Basic and Application Science, Vol. 2, No. 41, Pp. 1041-1051.<br />
Omidinia, S., Masrom, M. and Selamat, H. (2010) Review of ELearning and ICT Infrastructure in Developing<br />
Countries (Case Study of Iran), American Journal of Economics and Business Administration, Vol. 3, No. 1,<br />
Pp. 120-125.<br />
Patterson, D., Jung, G., Broadhead, G., and Halton, R. (2010) The UK eLearning market, [On-line], Available:<br />
http://www.eLearningcentre.co.uk/resource.<br />
Paechter, M., Maier, B., and Machera, D. (2010) Students’ expectations of, and experiences in eLearning: Their<br />
relation to <strong>learning</strong> achievements and course satisfaction, Computers and Education, Vol. 54, No. 1, Pp.<br />
222-229.<br />
Pett, M. A., Lackey, N. R. and Sullivan, J. J. (2003) Making sense of factor analysis: the use of factor analysis for<br />
instrument development in health care research, Sage Publications, London<br />
Rezaei Mood, F. (2006) Challenges Assessment and ways of implementing eLearning in Iran, Master Degree<br />
Thesis, College of Technology, Sharif University of Technology, Iran.<br />
Tham, C. and J. Werner (2002) Designing and Evaluating ELearning in Higher Education: a Review and<br />
Recommendations, Proceedings of the 2002 annual Conference of the Midwest Academy of Management,<br />
Indianapolis, [online]:http://cobacourses.creighton.edu/MAM/2002/papers/Tham.doc.<br />
Tyan, K.J. (2003) Diffusion barriers to eLearning in corporate Taiwan: a factor analysis of practitioners’<br />
perspective. Phd dissertation, Indian University.<br />
Urdan, T. A., & Weggen, C. C. (2000). Corporate eLearning: Exploring a new frontier WR Hambrecht + Co. [Online].<br />
Available:http://www.wrhambrecht.com/ research/coverage/e<strong>learning</strong>/ir/ ir_explore.pdf .<br />
Wang, Y. (2003). Assessment of learner satisfaction with asynchronous electronic <strong>learning</strong> systems, Information<br />
& Management, Vol. 41, No. 1, Pp. 75–86.<br />
World Bank (2009) Higher Education Reform for the Knowledge Economy Project, Project Appraisal Document,<br />
World Bank. Washington, DC".<br />
Zhang, W., J. Niu and G. Jiang (2002) Web-Based Education at Conventional Universities in China: A Case<br />
Study, International Review of Research in Open and Distance Learning, Vol. 2, No. 2, [online],<br />
www.irrodl.org/index. php/ irrodl/article/download/63/130.<br />
22
The use of Open Educational Resources in Intra-<br />
Organisational eLearning and Continuing Education<br />
Antonios Andreatos<br />
Div. of Computer Engineering & Information Science, Hellenic Air Force<br />
Academy, Dekeleia Air Force Base, Dekeleia, Greece<br />
aandreatos@gmail.com<br />
Abstract: Sustained <strong>learning</strong> is a crucial factor of an organisation’s ability to survive and effectively compete in<br />
the globalisation era worldwide. Continually shrinking half-life periods of knowledge and fast changing globalised<br />
workplaces, force employees working in knowledge-intensive companies to acquire new knowledge and skills for<br />
life. The evolution of Web 2.0, the advantages and boom of New Media, the massive production of open<br />
educational resources and globalisation, are bringing revolutionary changes to all forms of education: formal,<br />
non-formal and informal. Thus, the need for continuing education and lifelong <strong>learning</strong> is obvious. Various<br />
surveys performed worldwide show that university graduates are not well prepared for today's working<br />
environment. Most important skills missing are related to the use of ICT and adoption of emerging technologies,<br />
teamwork/ collaboration, creativity, innovation and change management, diversity and Leadership and<br />
fundamentals of enterpreneurship. Thus, there is also a need for education of newcomers. The purpose of this<br />
work is to examine the use of open educational resources in intra-organisational <strong>learning</strong> and to examine the<br />
properties of Learning Objects such as their metadata, that will facilitate this task. More precisely, this paper<br />
initially argues about the advantages of using open educational resources in eLearning and continuing education.<br />
Prerequisites for selecting and combining external Learning Objects will be examined. Ways for externalising the<br />
organisation's social capital in reusable Learning Objects appropriate for intra-organisational <strong>learning</strong> will be<br />
proposed. The issue of certification of informally acquired knowledge and skills will be mentioned. The use of<br />
open educational resources may enrich existing social capital and facilitate intra-organisational eLearning and<br />
continuing education in a cost-effective way. For better results, a formal characterisation scheme of LOs, in the<br />
form of a posteriori metadata, will greatly facilitate their organisation, search and retrieval process.<br />
Keywords: open educational resources, <strong>learning</strong> objects, continuing education, metadata, intra-organisational<br />
<strong>learning</strong>, social capital<br />
1. Introduction<br />
With financial conditions changing constantly and affecting every aspect of business activities,<br />
declining resources and growing competition, modern businesses are faced with multiple problems on<br />
all fronts. On the one hand they should be able to do more with less resources, and the other, they<br />
have to plan and achieve goals faster, more efficiently and of higher quality than in the past. As a<br />
consequence, work environments and conditions are continually changing.<br />
Continually shrinking half-life periods of knowledge and ever changing workplaces, force employees<br />
working in knowledge-intensive companies to acquire new knowledge and skills for life. These<br />
professionals have to continually update their experiences and knowledge profile (Fonstad & Lanvin,<br />
2010). Thus, sustained <strong>learning</strong> is a crucial factor of an organisation’s ability to survive and effectively<br />
compete in the globalisation era (Burke et al., 2006).<br />
In their effort to survive, organisations turn to ICT. Technologists invent and propose new solutions<br />
promising to facilitate enterprises to reduce their working cost and increase their revenues, such as<br />
Cloud Computing and Virtualization. The problem is that all these developments are ... new! Who will<br />
design the plans, who will implement them and who will manage them? The answer is of course “the<br />
business' people”, who should be trained not only effectively but also very quickly in their new tasks.<br />
1.1 The missing skills - what surveys show<br />
Various studies (Fonstad & Lanvin, 2010; Azami et al., 2009; Workforce Readiness Report, 2006;)<br />
concerning three different regions of the World (Europe, Malaysia and USA respectively), state that<br />
graduates are not well prepared for today's working environment.<br />
According to Fonstad & Lanvin (2010), there is a shortage of key ICT skills in Europe. Most missing<br />
skills are basic and professional skills as well as cross-sectoral skills (such as principles of economy<br />
and entrepreneurship). It is anticipated that by 2015 the unfilled vacancies of ICT professionals will be<br />
between 1.7% and 13%.<br />
23
Antonios Andreatos<br />
Figure 1: Estimated skills shortage in Europe for the next years (source: Fonstad & Lanvin (2010))<br />
Azami et al. (2009, p. 27, 29) have found out that the skills considered most important by employers<br />
were related to teamwork, communication and problem-solving ability. Other missing skills were those<br />
related to lifelong <strong>learning</strong>, application of basic knowledge, as well as, understanding professional,<br />
social and ethical responsibilities.<br />
The American Workforce Readiness report was prepared collaboratively by the Conference Board,<br />
Corporate Voices for Working Families, the Partnership for the 21st Century Skills and the Society for<br />
Human Resource Management. They conducted an in-depth study on the readiness of new entrants<br />
into the United States workforce by level of educational attainment from the corporate perspective.<br />
The four organisations surveyed over 400 employers across the U.S. These employers articulated the<br />
skill sets that new entrants need in order to succeed in the workplace. Among the most important<br />
skills cited by employers were: professionalism/ work ethic, oral and written communications,<br />
teamwork/ collaboration and critical thinking/ problem solving.<br />
Employer respondents determined as the top six skills expected to increase in importance over the<br />
next five years the following (Workforce Readiness Report, 2006, p. 49) are related to: Critical<br />
Thinking, Problem Solving, Information Technology adoption, Teamwork/Collaboration, Creativity,<br />
Innovation management, Diversity and Leadership.<br />
Employer respondents identified as the “most critical” for future graduates entering the U.S. workforce<br />
in the next five years the following:<br />
Use entrepreneurial skills to enhance workplace productivity and career options<br />
Understand economic issues and the role of business in the U.S. and global economy<br />
Demonstrate understanding of global markets and cultural effects of globalisation<br />
Use non-English languages as a tool for understanding other nations, markets and cultures.<br />
The findings of the three surveys agree that, apart from knowledge related to their profession,<br />
graduating engineers and IT professionals lack: a) communication, cooperation and team work skills;<br />
b) skills related to lifelong <strong>learning</strong> and continuing education; c) basic knowledge and a general<br />
understanding of organisational operations, d) media literacy and adoption of current and emerging<br />
ICT technologies in their job and e) social and ethical responsibilities.<br />
24
Antonios Andreatos<br />
Hence, enterprises should strengthen personal development of their employees in non-formal<br />
<strong>learning</strong> courses developed especially for their staff (Andreatos, 2011). Fonstad & Lanvin (2010) have<br />
found that organisations invest in at the following alternatives for developing the competences of their<br />
employees:<br />
In-house training provided by a university;<br />
In-house training provided by a consulting firm;<br />
In-house training provided by employees;<br />
External business degree programs;<br />
External ICT degree programs;<br />
External university courses;<br />
A combination of the above options.<br />
Professional development goes hand in hand with organisational development; therefore,<br />
organisations ought to foster employees' continuing education in various ways. Lifelong <strong>learning</strong> is<br />
needed, with courses focusing on the skills most needed by the organisation, as well as, specific<br />
inter-disciplinary modules related to entrepreneurship, e-business, lifelong <strong>learning</strong>, face to face and<br />
distance cooperation, communicative and ICT -related skills. The certification of informally acquired<br />
knowledge and skills by official formal education degrees such as master's or doctoral titles or even<br />
by informal titles awarded by the company (such as “technical expert”), constitute ways to promote<br />
and foster creativity.<br />
2. Learning at the workplace<br />
Several schemes for the continuing education of employees have been proposed (Ley et al., 2008;<br />
Fonstad & Lanvin, 2010; Andreatos, 2011) with varying cost (Workforce Readiness Report, 2006, p.<br />
46). Courses could be offered either face-to-face or from distance by universities or specialised<br />
continuing education organisations, either public or private, or either by the enterprises themselves<br />
(e.g. the Human Resources Dept). However, such programmes are often difficult or even impossible<br />
for small enterprises to run, especially in crisis periods; hence, other alternatives should also be<br />
examined, such as in-house training provided by experienced or specialised employees (Fonstad &<br />
Lanvin, 2010, p. 21) and self-directed <strong>learning</strong>.<br />
Other reasons calling for <strong>learning</strong> at the workplace are: the direct application of knowledge to current,<br />
real job activities hence maximisation of the <strong>learning</strong> transfer, the maintainance of knowledge work,<br />
and hence, the increase of productivity (Haskell, 2001; Ley et al., 2008). Aligning <strong>learning</strong> to<br />
organisational goals and task requirements is an important factor posing challenges for traditional<br />
personnel development instruments and trainings. How this alignment can be addressed within<br />
knowledge work remains an open issue (Elkjaer, 2000).<br />
2.1 Cost reduction of lifelong <strong>learning</strong><br />
In-house continuing education saves direct costs such as fees as well as indirect costs related to<br />
employees' absence. However, in-house continuing education programmes still have a cost related to<br />
tutor compensation, cost of educational materials, <strong>learning</strong> need analysis etc.<br />
In order to reduce cost, organisations may take advantage of their asset such as social capital and<br />
communities of practice.<br />
Social capital refers to knowledge artifacts and resources are stored within organisational databases.<br />
A more formal definition given by Nahapiet and Ghoshal (1998: p. 243) is the following: social capital<br />
is “the sum of the actual and potential resources embedded within, available through and derived from<br />
the network of relationships possessed by an individual or social unit”.<br />
A community of practice (abbreviated as CoP) is an important intra-organisational structure that has<br />
been proven to effectively promote informal <strong>learning</strong> (Wenger, 1998; Andreatos, 2009a). A CoP is a<br />
human network enabling professionals to communicate either face-to-face or virtually (vCoPs) and<br />
25
Antonios Andreatos<br />
exchange applied knowledge related to their profession. People engaged in CoPs share some<br />
commonly accepted background of beliefs and a common pool of resources (social capital of the<br />
CoP); therefore, they are continually educated in their profession informally. Hence CoPs are very<br />
important since they not only keep their members up-to-date, but they also produce applied<br />
knowledge and professional skills. For this reason, it is essential for their members to externalise this<br />
knowledge and diffuse it within the organisation, as well as, to incorporate it to the organisation's<br />
social capital databases. A modern as well as convenient way to externalise knowledge is to use web<br />
2.0 platforms and technologies, especially New Media (Andreatos, 2011).<br />
An important issue is to capture, formalise and externalise the knowledge generated within the<br />
organisation for intra-organisational <strong>learning</strong> and on-the-job training. Perhaps this is the best way to<br />
cultivate and proliferate the skills most needed by the organisation to its personnel. One way to do<br />
this is to have technical experts and other employees mastering specific techniques or processes, to<br />
externalise their experience in Learning Objects and New Media products. These will constitute or will<br />
contribute to the organisation's social capital (Andreatos, 2009a; Nahapiet and Ghoshal, 1998; Lesser<br />
& Prusak, 1999; Andreatos, 2011). In fact, technical experts are talented workers incorporating a<br />
great part of the organisational social capital. The encoded and written social capital in digital<br />
products can be used as the primary source of educational materials on which intra-organisational<br />
<strong>learning</strong> may be based. Social capital can (and should) be used to train newcomers (Andreatos,<br />
2009b).<br />
The cost of educational materials such as collectively authored books by experts, dealing with modern<br />
hot topics (like those related to ICT) or books especially written for distance education, which is an<br />
attractive formal continuing education option for working adults (Holmberg et al., 1982; Race, 1989;<br />
West, 1996; Mena) is not negligible. An alternative is to use open educational resources, but will they<br />
fit the needs? Several questions arise, concerning their quality, interoperability, coverage of <strong>learning</strong><br />
needs, etc.<br />
Another scheme for intra-organisational <strong>learning</strong> on issues on which the organisation does not<br />
produce knowledge but they are considered important is the formal or informal continuing education.<br />
Informal continuing education may heavily rely on LO's available through repositories or New Media<br />
channels (such as YouTube, slideshow, podcast sources, Internet educational TV channels, etc.)<br />
3. In-flow of new knowledge<br />
Just as a CoP can externalise its own knowledge, it can as well internalise new knowledge produced<br />
by other sources such as universities or individuals or even other CoPs, in the form of New Media.<br />
Free digital products which have an educational nature (also called Learning objects) are known as<br />
OER, standing for Open Educational Resources. Traditionally, OER are reusable materials; i.e., they<br />
may be used in various educational contexts, for various purposes (Andreatos& Katsoulis, 2011).<br />
3.1 Selecting OER to fulfill specific needs<br />
Although this is an open issue, we shall attempt to specify a set of criteria for selecting OER to fulfill<br />
specific needs.<br />
Quality;<br />
Relevance to programme targets;<br />
Matching with existing social capital, as well as, organisational culture;<br />
Matching with other materials already approved;<br />
Conformance to the programme orientation (e.g. theoretical vs. practical, etc.).<br />
Organisations offering continuing education courses to enterprises (such as universities, firms or<br />
consultants) may preserve a pool of LOs covering a wide range of needs and <strong>learning</strong> styles, and<br />
then select specific LOs to build programmes tailored to specific needs.<br />
The persons responsible for selecting OER for a specific programme have several options:<br />
One is to adopt an existing open course, for instance, from a recognised university. Several<br />
universities have opened their lectures as OER, for instance Harvard, MIT (“MIT OpenCourseWare”),<br />
26
Antonios Andreatos<br />
Stanford (“Stanford Engineering Everywhere”) and Yale (“Open Yale Courses”) - see<br />
http://www.jimmyr.com/blog/ 1_Top_10_Universities_With_Free_Courses_Online.php<br />
There are several advantages associated with this option: guaranteed quality, leading to known<br />
expected results; saving of time, possibility of certification by the specific university or its affiliates,<br />
prestige, etc. If a specific course (e.g. communication, cooperation and team work skills or Innovation<br />
management) makes part of a specific master's programme (e.g., MBA), then there is an additional<br />
motivation for employees to attend the course: enrolling in the programme and taking additional<br />
courses, even from distance, will allow them to earn a formal degree, possibly leading to a better<br />
position. Professional development goes hand in hand with organisational development; therefore,<br />
organisations ought to offer continuing education motivation to their people for mutual advantage<br />
(Fonstad & Lanvin, 2010). Table 1 below summarises the most important options with their pros and<br />
cons.<br />
Table 1: The most important options in designing a continuing education in-house programme<br />
No Option Pros Cons<br />
1 Adopt an existing<br />
course, for instance,<br />
from a recognised<br />
university<br />
2 Make your own<br />
course<br />
3 Adopt an existing<br />
course and enhance it<br />
with selected LOs<br />
Assured quality and specific level (grad., postgrad,<br />
etc.); Ready; time saving option;<br />
Guaranteed acceptance;<br />
No certification problems; May easily lead to<br />
certification by external universities or firms<br />
Fully adapted to organisation's needs; also may<br />
adapted to various <strong>learning</strong> styles; economic<br />
under conditions; certification is questionable<br />
May by adapted to <strong>learning</strong> styles, personalised<br />
needs & organisation social capital; possible<br />
certification by external universities or firms<br />
Cannot be adapted to specific<br />
personalised needs & <strong>learning</strong><br />
styles; may not fully comply<br />
with organisation social capital<br />
or needs; cost in case of<br />
external certification<br />
Time consuming; unscertain<br />
recognition<br />
Moderate time consuming;<br />
Moderate difficulty in<br />
assessment;<br />
The most important reasons for adopting an existing course from a recognised university are:<br />
ascertained quality, ready-to-use, easier way to certification (such as a Mastrer's degree, perhapswith<br />
additional study).<br />
In case the organisation selects to make its own course, the person responsible to select the LOs has<br />
two options: either carefully examine (i.e., read, watch or listen to) the OER exhaustively; or to select<br />
LOs by examining their metadata.<br />
3.2 LO metadata<br />
Learning Objects are described by metadata, i.e., information required to fully or adequately describe<br />
their content. Typical metadata information may be author's name, institution, file size, description,<br />
location, time of creation, language, culture etc. This information is important for the recall of <strong>learning</strong><br />
objects, their appropriateness regarding specific uses tasks and their quality (Andreatos & Katsoulis,<br />
2011).<br />
From another perspective, metadata can be either a priori or a posteriori. A priori metadata are<br />
created in advance by the authors of <strong>learning</strong> objects or professional indexers. A posteriori<br />
metadata, in contrast, are created after usage by the users themselves or by automatic means<br />
(Andreatos & Katsoulis, 2011).<br />
However, professional LOs are described by high quality metadata information related to their content<br />
or even the specific <strong>learning</strong> needs they intend to cover. Such a scheme greatly facilitates search and<br />
data mining processes. As an example, consider the articles embedded in the “Help” of office suites.<br />
When a user needs help, he/she types a specific keyword in the help textbox and immediately the<br />
system finds, recalls and presents a set of articles, as well as, related topics (Figure 2).<br />
Ley et al. (2008) describe a smart technology-enhanced work-integrated platform facilitating in-house,<br />
just-ιn-time <strong>learning</strong>, developed in the APOSDLE project. They model the typical IT-based workplace<br />
as a set of three conceptual inter-related spaces: the work space, the <strong>learning</strong> space and the<br />
27
Antonios Andreatos<br />
knowledge space, called the Workplace Learning Context. LOs stored in the knowledge space,<br />
encoding the organisation's knowledge, are assigned a number of concepts from a domain-specific<br />
ontology in the form of metadata. The system keeps a historical <strong>learning</strong> profile for each employee<br />
based on tasks performed in the past; then it uses this profile to find and propose LOs related to<br />
competencies needed in order to successfully carry out future assigned tasks (Ley et al., 2008: p.<br />
2103).<br />
Figure 2: Search results from the “Help” of OpenOffice suite are based on metadata<br />
From the above examples it comes out that a proper metadata assignment and process scheme can<br />
greatly facilitate the search of proper LOs, as well as, the construction of proper courses tailored to<br />
specific or personalised needs.<br />
But when it comes to OER from various sources, by various authors, made to cover different needs,<br />
with minimum and incompatible metadata, there is a problem. Τhere is a great difficulty to classify,<br />
search and recover LOs according to specific criteria; moreover, there is no uniformity, no quality<br />
guarantee, nothing. The following three figures show metadata concerning LOs about Stakeholder<br />
Analysis from three different repositories.<br />
As Andreatos & Katsoulis (2011) have noticed, there is also a lot of good quality educational LOs in<br />
New Media formats (flash content, presentations, webcasts, podcasts etc.), stored in general-purpose<br />
New Media sites such as YouTube. All resources stored in YouTube are accompanied by metadata<br />
information, as well as, statistics such as number of views, “likes” and “dislikes” and users' comments.<br />
Another important feature of YouTube is the projection of related videos. It is noticeable that famous<br />
universities and university professors maintain their own New Media channels (Figure 6).<br />
28
Figure 3: LO metadata from EDNA repository<br />
Figure 4: LO metadata from Ideas repository<br />
Antonios Andreatos<br />
29
Figure 5: Video metadata from YouTube<br />
Antonios Andreatos<br />
Figure 6: Yale maintains educational channels in YouTube and iTunes<br />
From the above figures it comes out that there is high incompatibility of metadata, making their<br />
automatic process difficult. This means that in most cases the selection of LOs from various sources<br />
will have to be done by a human: somebody will have to exhaustively examine the LOs and manually<br />
assign the missing metadata according to a formal scheme, to make them machine-usable.<br />
Of course, other options exist, such as examining part of the LOs exhaustively, while choose some<br />
others (for instance, those coming from trusted sources) only by checking their metadata. Another<br />
solution is to rate sources or authors by quality and then easily accept LOs from these sources or<br />
authors.<br />
One form of metadata added by users is social tags or folksonomies. Various schemes for automatic<br />
metadata generation using combinations of author indexing, expert indexing, peer review, automatic<br />
metadata generation and/or collaborative social tagging have been proposed (Andreatos & Katsoulis,<br />
2011).<br />
As far as quality is concerned, many repositories use a peer process to assess their LOs. New Media<br />
channels on the other hand use statistics, “likes” and “dislikes” and users' comments, which provide<br />
some quality indication.<br />
Recently some experimental tools facilitating the search of OER have been proposed (Andreatos &<br />
Katsoulis, 2011). However, the author believes that the research in this area is still in its infancy.<br />
30
4. Propositions<br />
Antonios Andreatos<br />
It has been ascertained that:<br />
CoPs promote informal <strong>learning</strong> and work as a mechanism of professional continuing education.<br />
Organisations are interested in promoting employees' personal development (Andreatos, 2011).<br />
There is a high diversity in employees' personal educational needs, contrary to that of students.<br />
CoPs produce social capital which may be encoded in New Media products.<br />
A huge plethora of OER covering all areas of knowledge is available in the Internet.<br />
There are a lot of university open courses available.<br />
There are a lot of New Media LOs, appropriate for education.<br />
Continuing education is needed in order to make organisations antagonistic in today's globalised<br />
environment (Fonstad and Lanvin, 2010; Andreatos 2011).<br />
It is of highest importance for an organisation to know its knowledge status. Questions like the<br />
following have to be answered:<br />
Is the organisation producing any knowledge? Of what kind ?<br />
What percentage of the produced knowledge is currently stored within the organisational databases?<br />
In what format?<br />
Does the organisation need to assimilate new knowledge? and of what kind ?<br />
What is the existing knowledge gap at various levels (organisational, departmental, CoP, individual)?<br />
Which are the missing skills?<br />
Is there a <strong>learning</strong> needs analysis? Learning needs analyses can also identify inter- and intradepartmental<br />
needs, common missed skills etc. Can uniform groups be formed?<br />
What are the various continuing education programme options and what is their cost?<br />
Who will design them and how will they assure their quality?<br />
Will there be a certification?<br />
How shall we assess the quality of the certification?<br />
Is this certification widely accepted?<br />
Can our staff be certified by an external, universally accepted authority such as a University? At what<br />
cost?<br />
Thus, organisations have to adopt sustained <strong>learning</strong> policies exploiting their staff (e.g., technical<br />
experts, CoPs), their social capital, as well as, OER freely available in the Internet.<br />
5. Conclusion<br />
In this paper the following points were discussed:<br />
Sustained <strong>learning</strong> is a crucial factor of an organisation’s ability to survive and effectively compete<br />
in the globalisation era worldwide.<br />
Organisations are <strong>learning</strong> entities producing their own knowledge encoded in their social capital,<br />
part of which is stored in their databases.<br />
This social capital is a very important asset because it summarises the organisation's experience.<br />
31
Antonios Andreatos<br />
The organisational social capital can be encoded in LOs stored in its databases.<br />
It has to be continually enhanced and enriched, reflecting the company's effort to assimilate new<br />
ICT and new knowledge.<br />
The organisational social capital plus selected external LOs may form the basis of the educational<br />
materials used to train newcomers and continually educate knowledge workers.<br />
After proper <strong>learning</strong> needs analysis, an organisation investing in knowledge and ICT needs to design<br />
and adopt a sustained continuing education plan. Several options appear (Fonstad & Lanvin, 2010),<br />
ranging from formal post-graduate university education to informal in-house training designed and<br />
offered by the organisation itself. In most cases, cost reduction policies make the use of proper OER<br />
an attractive choice.<br />
In order for an organisation to design its own in-house continuing education programme, a set of<br />
scientific, pedagogical and quality and cost criteria have to be met. Fundamental prerequisites<br />
include: the quality credentials of the course designers; the quality of the educational materials;<br />
coverage of specific <strong>learning</strong> needs; correctness and efficiency. Additional issues regard the<br />
effectiveness, efficiency and certification of such programmes.<br />
After collecting a set of appropriate LOs meeting predefined criteria, selected LOs may be stored in<br />
the organisation's databases to enhance the existing social capital. The use of a formal<br />
characterisation of LOs in the form of a posteriori metadata will greatly facilitate their organisation,<br />
search and retrieval process.<br />
In conclusion, the use of open educational resources may enrich existing social capital and facilitate<br />
intra-organisational eLearning and continuing education in a cost-effective way.<br />
References<br />
Andreatos, A. (2009a) “On the Definition and Impact of Virtual Communities of Practice”, International Journal of<br />
Virtual Communities and Social Networking, 1(4), pp 73-88.<br />
Andreatos, A. (2009b) “Informal <strong>learning</strong> in FOSS communities; the Greek case”. In A. Lionarakis (Ed.),<br />
Proceedings of the 5th International Conference on Open and Distance Learning. 27 -29 Nov. 2009,<br />
Athens, Greece.<br />
Andreatos, A. & Katsoulis, S. (2011) “Enriching a course syllabus with open educational resources”. In<br />
Proceedings of ICICTE 2011, International Conference on Information Communication Technologies in<br />
Education. Rhodes, Greece, July 7 - 9, 2011. http://www.icicte.org/.<br />
Andreatos, A. (2011) “Educating the 21st century's Engineers and IT Professionals” Chapter in Book “New Media<br />
Communication Skills for Engineers and IT Professionals: Trans-National and Trans-Cultural Demands”, IGI<br />
Global (forthcoming).<br />
Azami, Z., Yuzainee, Y., Mohd Zaidi, O., Azah, M. and Norhamidi, M. (2009) “Employers’ Perceptions and<br />
Expectation toward Engineering Graduates: A Study Case”. Proceedings of the 6th WSEAS International<br />
Conference on Engineering Education, pp 23-29.<br />
Burke, S. C. et al., (2006) ”What types of leadership behaviors are functional in teams? A meta-analysis”,<br />
Leadership Quarterly, Vol. 17, 2006, pp 288-307.<br />
Elkjaer, B. (2000) "Learning and getting to know: the case of knowledge workers", Human Resource<br />
Development International, Vol. 3 No. 3, pp 343-359.<br />
Fonstad, N. O. & Lanvin, B. (2010) Strengthening e-Skills for Innovation in Europe - Building on partnerships<br />
between academia, industry and government for better e-competences curricula. European Commission<br />
INSEAD eLab, [online], http://www.insead.edu/facultyresearch/centres/ecompetences/library/documents/<br />
INSEADECNewSkillsforInnovationsReport.pdf.<br />
Haskell, R.E. (2001) Transfer of Learning: Cognition and Instruction, Elsevier <strong>Academic</strong> Press.<br />
Holmberg B. et al., (1982) Zur Effizienz des gelenkten didaktischen Gespräches, Fernuniversität (ZIFF), Hagen.<br />
Lesser E. and Prusak, L. (1999) Communities of practice, social capital and organizational knowledge. White<br />
paper, [online], http://www.providersedge.com/docs/km_articles/Cop_-_Social_Capital_-_Org_K.pdf.<br />
Mena M., “New Pedagogical Approaches to Improve Production of Materials in Distance Education” [online],<br />
http://cade.icaap.org/vol8.3/10b_mena-english.html.<br />
Nahapiet, J. and Ghoshal, S. (1998) “Social Capital, Intellectual Capital and the organizational Advantage”,<br />
Academy of Management Review, 23(2), 242-266.<br />
Nonaka, I. (2008) The knowledge-creating company. Boston: Harvard Business Press.<br />
Race, Ph. (1989) The Open Learning Handbook – Selecting, designing and supporting Open Learning materials.<br />
London: Kogan Page.<br />
Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity. Cambridge University Press.<br />
32
Antonios Andreatos<br />
West R., (1996) “Concepts of text in distance education”, In Motteram G., Walsh G., West R. (eds), Proceedings<br />
of 2nd symposium on Distance Education for Language Teachers, University of Manchester, Manchester<br />
1996, pp 62-72.<br />
Workforce Readiness Report (2006) “Are they really ready to work? Employers’ perspectives on the basic<br />
knowledge and applied skills of new entrants to the 21st century U.S. workforce”. Publisher: The<br />
Conference Board, Corporate Voices for Working Families, Partnership for 21st Century Skills and the<br />
Society for Human Resource Management. [online], http://www.p21.org/documents/<br />
FINAL_REPORT_PDF09-29-06.pdf.<br />
33
Constructing a Survey Instrument for Assessing<br />
Characteristics of Effective Online Teachers<br />
Jonathan Barkand<br />
Duquesne University, Pittsburgh, USA<br />
Jbarkand@gmail.com<br />
Abstract: This paper will report on a proposed research project, which seeks to create and validate an<br />
instrument that can be used to determine the effectiveness of online teachers. This study is the first step in a<br />
comprehensive research project that is intended to establish the reliability and validity of a survey for online<br />
teachers. A significant body of research has been conducted in identifying and quantifying characteristics of<br />
effective online teachers. However, to date, a standardized survey instrument has not been formally accepted or<br />
approved to make a qualitative assessment of which characteristics are considered to be most essential for<br />
success in an online environment. As education continues to make the shift to a completely online environment,<br />
there will be a need to design an assessment system to determine effective and ineffective online teachers.<br />
Paralleling this transition is the need for the training and education of teachers to be more effective in an online<br />
environment. Certificates in Online Teaching and Learning (OTL) are being offered at universities at the graduate<br />
level. These programs are designed to prepare teachers for teaching and <strong>learning</strong> online. The quality of OTL<br />
programs would be greatly increased with a survey instrument to be used with current and graduated students.<br />
The survey results will be categorized based on the frameworks used by the National Council for Accreditation of<br />
Teacher Education (NCATE). Data will be sorted into the three NCATE categories of Knowledge, Skills, and<br />
Professional Dispositions. This categorization will create an easy to use characteristic breakdown for effective<br />
online teachers. The use of these results is not limited to self-improvement, but could be used by supervisors and<br />
school administration. Based on the literature and results, an assessment matrix will be developed that will allow<br />
teachers from any discipline to see a measurement of their effectiveness as an online teacher.<br />
Keywords: online teaching, eLearning teachers, teacher characteristics, effective teaching survey instrument,<br />
quality online teaching<br />
1. Introduction<br />
Teachers are a key component of students <strong>learning</strong> in the education environment. There are many<br />
state and national standards for effective teachers in a traditional classroom, though there are very<br />
few, if any, standards that relate directly to effective online teachers. An online teacher will be<br />
accessible to hundreds of students, and the students should be <strong>learning</strong> from a highly qualified and<br />
effective online teacher. The transition for students from a traditional education setting to a fully online<br />
setting has been increasing each year. An instrument that can reliably measure teacher self<br />
perceptions and also student perceptions of effective online teachers will be necessary if we are to<br />
continue to improve the quality of online instruction.<br />
The expanding enrollments in online courses are occurring in both higher education and K-12<br />
institutions. In fall 2009, the enrollment of higher education students who take at least one online class<br />
increased 21% over fall 2008 (Allen & Seaman, 2010). Since 2002, there has been a compounded<br />
annual growth rate of 19%, bringing the total enrollment to 5.6 million students in 2009 (Allen &<br />
Seaman, 2010). A growth rate of this level will eventually peak, but until then, more students are<br />
taking online courses each year. The enrollments in basic education K-12 institutions are more difficult<br />
to predict, although certain estimates can be extrapolated from previous data. For example, there<br />
were an estimated 1,030,000 students engaged in online courses in 2008 according to a survey of<br />
school district administrators (Picciano & Seaman, 2009). The number of K-12 students is lower than<br />
higher education students, which is likely due to the fact that K-12 education is years behind higher<br />
educations’ shift to an online environment. An interesting fact is that the 2008 survey figure<br />
represented a 47% growth over two years since 2006. Both K-12 and higher education trends appear<br />
to be following similar growth patterns. Given these growth patterns, online education should already<br />
be a part of both K-12 and higher education institutions education plan ( Kim & Bonk, 2006).<br />
A survey instrument which can be used to determine the effectiveness of online teachers in this ever<br />
increasing online <strong>learning</strong> environment will be necessary for improving the quality of online teaching<br />
and <strong>learning</strong>. Based on literature and published standards, a survey instrument will be developed<br />
which can be used for K-12 or higher education online environments. The National Council for<br />
Accreditation of Teacher Education (NCATE) framework was used to categorize each of the<br />
standards into the three categories of knowledge, skills, and professional dispositions. Each question<br />
34
Jonathan Barkand<br />
on the survey was aligned with a specific standard. A conceptual framework was necessary to tie<br />
different standards, suggestions, and criteria together into a standardized survey instrument. The<br />
NCATE framework was chosen because of its wide-spread use for developing high quality teacher<br />
education programs on a national level within the United States.<br />
Requiring candidates to meet state standards for teacher certification is the most common way to<br />
obtain licensure. Current teacher licensure testing is based on preparing candidates for a traditional<br />
education setting. Most states require teacher candidates to meet a combination of passing scores on<br />
the Praxis examinations and having successfully completed the requirements in an approved program<br />
of study. Most online schools hire candidates who have met the traditional state guidelines for teacher<br />
certification. The certification process is still trying to catch up to the specific requirements for<br />
successful online education in the K-12 environment. In the absence of specific state standards,<br />
several colleges and universities have started offering certificates in Online Teaching and Learning<br />
(OTL) in an effort to fill the gap. Professional organizations such as the North American Council for<br />
Online Learning (NACOL) in 2006 have promulgated prototype standards in order to provide<br />
consistent standards which can be used on a national level.<br />
Teacher preparation will be a critical component in preparing teachers for an online environment.<br />
Without modeling effective online teaching strategies and methods, most of the 86,000 new teachers<br />
who enter the online profession each year, begin without having specific online teaching skills in their<br />
professional repertoire (NEA, n.d.). Understanding the characteristics that make teachers effective in<br />
an online environment will be an important component in shaping teacher preparation programs.<br />
2. Standards<br />
After a comprehensive review of the literature, NACOL standards and indicators were used as the<br />
basis for the proposed survey instrument (NACOL 2006). The NACOL standards are based on the<br />
Southern Regional Education Board (SREB) Standards for Quality Online Teaching. The standards<br />
were developed by knowledgeable and experienced resource persons from K-12 and postsecondary<br />
education from national and regional organizations, SREB state departments of education, colleges,<br />
and universities (SREB, 2006).<br />
NACOL adopted the work of SREB. NACOL added two additional standards from the Ohio<br />
Department of Education’s Ohio Standards for the Teaching Profession and the Electronic Classroom<br />
of Tomorrow’s Teacher Evaluation Rubric based on the results of the review (NACOL, 2008). The<br />
indicators for each standard were also modified slightly.<br />
SREB Standards:<br />
The teacher meets the professional teaching standards established by a state licensing agency or<br />
the teacher has academic credentials in the field in which he or she is teaching.<br />
The teacher has the prerequisite technology skills to teach online.<br />
The teacher plans, designs and incorporates strategies to encourage active <strong>learning</strong>, interaction,<br />
participation and collaboration in the online environment.<br />
The teacher provides online leadership in a manner that promotes student success through<br />
regular feedback, prompt response and clear expectations.<br />
The teacher models, guides and encourages legal, ethical, safe, and healthy behavior related to<br />
technology use.<br />
The teacher has experienced online <strong>learning</strong> from the perspective of a student.<br />
The teacher understands and is responsive to students with special needs in the online<br />
classroom.<br />
The teacher demonstrates competencies in creating and implementing assessments in online<br />
<strong>learning</strong> environments in ways that assure validity and reliability of instruments and procedures.<br />
The teacher develops and delivers assessments, projects, and assignments that meet standardsbased<br />
<strong>learning</strong> goals and assesses <strong>learning</strong> progress by measuring student achievement of<br />
<strong>learning</strong> goals.<br />
The teacher demonstrates competencies in using data and findings from assessments and other<br />
data sources to modify instructional methods and content and to guide student <strong>learning</strong>.<br />
35
Jonathan Barkand<br />
The teacher demonstrates frequent and effective strategies that enable both teacher and students<br />
to complete self-assessments and pre-assessments.<br />
NACOL Additional Standards:<br />
The teacher collaborates with colleagues.<br />
The teacher arranges media and content to help students and teachers transfer knowledge most<br />
effectively in the online environment. (Optional)<br />
3. NCATE categorization<br />
The NCATE Professional Standards for the Accreditation of Teacher Preparation Institutions was<br />
used to categorize the online teacher standards collected. NCATE is officially recognized by the U.S.<br />
Department of Education as an accrediting body for institutions that prepare teachers and other<br />
professional persons for work in preschool, elementary, and secondary schools (NCATE, 2008).<br />
NCATE Standard 1 related to Candidate Knowledge, Skills, and Professional Dispositions was<br />
specifically used for categorization. The optional NACOL additional standard was related more to<br />
instruction design rather than teacher characteristics and was not used. The SREB/NACOL standards<br />
have had a naming structure applied for alignment purposes for the survey.<br />
1. Content and Pedagogical Knowledge for Teacher Candidates:<br />
1.1. The teacher meets the professional teaching standards established by a state licensing agency or<br />
the teacher has academic credentials in the field in which he or she is teaching.<br />
1.2. The teacher has the prerequisite technology skills to teach online.<br />
2. Pedagogical and Professional Skills for Teacher Candidates:<br />
2.1. The teacher plans, designs and incorporates strategies to encourage active <strong>learning</strong>, interaction,<br />
participation, and collaboration in the online environment.<br />
2.2. The teacher provides online leadership in a manner that promotes student success through<br />
regular feedback, prompt response, and clear expectations.<br />
2.3. The teacher has experienced online <strong>learning</strong> from the perspective of a student.<br />
2.4. The teacher understands and is responsive to students with special needs in the online<br />
classroom.<br />
2.5. The teacher demonstrates competencies in creating and implementing assessments in online<br />
<strong>learning</strong> environments in ways that assure validity and reliability of instruments and procedures.<br />
2.6. The teacher develops and delivers assessments, projects, and assignments that meet standardsbased<br />
<strong>learning</strong> goals and assesses <strong>learning</strong> progress by measuring student achievement of <strong>learning</strong><br />
goals.<br />
2.7. The teacher demonstrates competencies in using data and findings from assessments and other<br />
data sources to modify instructional methods and content and to guide student <strong>learning</strong>.<br />
2.8. The teacher demonstrates frequent and effective strategies that enable both teacher and<br />
students to complete self-assessments and pre-assessments.<br />
3. Professional Dispositions for Teacher Candidates:<br />
3.1. The teacher models, guides and encourages legal, ethical, safe, and healthy behavior related to<br />
technology use.<br />
3.2. The teacher collaborates with colleagues.<br />
36
4. Indicators and survey questions<br />
Jonathan Barkand<br />
The following Indicators were part of NACOL Standards for Quality Online teaching. Not all indicators<br />
were used to create the assessments. An indicator for each standard was chosen based on the ability<br />
to accurately measure the self-assessment. Some questions cover more than one indicator. Some<br />
standards contained multiple methods of assessment and required more than one question to<br />
accurately cover the scope of the standard. The actual question was created by the researcher and<br />
the question type is added to show the form of measurement.<br />
Indicator 1.1: Meets the state’s professional teaching standards or academic credentials in the field.<br />
Question 1.1: Are you a state certified teacher in your content area?<br />
Type: Dichotomous (Yes or No)<br />
Indicator 1.2: Demonstrates the ability to effectively use word-processing, spreadsheet, and<br />
presentation software.<br />
Question 1.2: How would you rate your technology skills in Microsoft Office and Internet systems<br />
(browsing, emailing)?<br />
Type: Five-Point Likert Scale<br />
Indicator 1.2: Utilizes synchronous and asynchronous tools (e.g., discussion boards, chat tools,<br />
electronic whiteboards) effectively.<br />
Question 1.2: How would you rate your technology skills with teaching synchronous and<br />
asynchronous tools (discussion boards, chat/tutor tools, electronic whiteboard)?<br />
Type: Five-Point Likert Scale<br />
Indicator 2.1: Facilitates and monitors appropriate interaction among students.<br />
Indicator 2.1: Promotes <strong>learning</strong> through group interaction.<br />
Question 2.1: How many hours per week are spent on discussion boards and facilitating group<br />
interaction?<br />
Type: Nominal with ranges (Less than an hour, 1-2 hours, 2-4 hours, 4-8 hours, 8+ hours)<br />
Indicator 2.2: Models effective communication skills and maintains records of applicable<br />
communications with students.<br />
Question 2.2: What is your average response time on student communications<br />
(emails/discussions/phone)?<br />
Type: Nominal (less than an hour, 1-3 hours, 3-6 hours, within 24 hours, 24+ hours)<br />
Indicator 2.2: Encourages interaction and cooperation among students, encourages active <strong>learning</strong>,<br />
provides prompt feedback, communicates high expectations, and respects diverse talents and<br />
<strong>learning</strong> styles.<br />
Indicator 2.2: Provides timely, constructive feedback to students about assignments and questions.<br />
Question 2.2: How would you rate the quality of your timely, personalized, and constructive<br />
feedback for assignments/questions?<br />
Type: Five-Point Likert Scale<br />
37
Jonathan Barkand<br />
Indicator 2.3: Has taken an online course and applies experiences as an online student to develop<br />
and implement successful strategies for online teaching.<br />
Question 2.3: How many online courses have you taken?<br />
Type: Nominal (0, 1-2, 3-4, 5-6, 6+)<br />
Indicator 2.3: Demonstrates an understanding of the perspective of the online student through<br />
appropriate responsiveness and a supportive attitude toward students.<br />
Question 2.3: How much has taking an online course changed your perspective of teaching online?<br />
Type: Five-Point Likert Scale (must include a Not Applicable choice)<br />
Indicator 2.4: Adapts and adjusts instruction to create multiple paths to <strong>learning</strong> objectives.<br />
Question 2.4: How would you rate your ability to adapt and adjust lessons for students with special<br />
needs?<br />
Type: Five-Point Likert Scale<br />
Indicator 2.5: Creates or selects fair, adequate and appropriate assessment instruments to<br />
measure online <strong>learning</strong> that reflect sufficient content validity (i.e., that adequately cover the content<br />
they are designed to measure), reliability and consistency over time.<br />
Question 2.5: How would you rate your assessments ability to measure the content covered?<br />
Type: Five-Point Likert Scale<br />
Indicator 2.6: Includes authentic assessment (i.e., the opportunity to demonstrate understanding of<br />
acquired knowledge and skills as opposed to testing isolated skills or retained facts) as part of the<br />
evaluation process; assesses student knowledge in a forum beyond multiple guess.<br />
Question 2.6: What estimated percentage of assessments in your course are not computer graded?<br />
Type: Nominal (1-20%, 21-40%, 41-60%, 61-80%, 81-100%)<br />
Indicator 2.7: Reviews student responses to test items to identify issues related to test validity or<br />
instructional effectiveness.<br />
Indicator 2.7: Creates opportunities for self-reflection or assessment of teaching effectiveness<br />
within the online environment (e.g., classroom assessment techniques, teacher evaluations, teacher<br />
peer reviews).<br />
Question 2.7: How many hours per week do you devote to self-reflection and improvement in your<br />
teaching style and content?<br />
Type: Nominal (Less than an hour, 1 – 2 hours, 2 - 4 hours, 4-8 hours, 8+ hours)<br />
Indicator 2.8: Understands that student success (e.g., grade, level of participation, mastery of<br />
content, completion percentage) is an important measure of teaching and course success.<br />
Question 2.8: How much do you feel student success (e.g., grade, level of participation, mastery of<br />
content, completion percentage) measures effective teaching and course success?<br />
Type: Five-Point Likert Scale<br />
Indicator 3.1: Identifies the risks of academic dishonesty for students.<br />
38
Jonathan Barkand<br />
Indicator 3.1: Establishes standards for student behavior that are designed to ensure academic<br />
integrity and appropriate uses of the internet and written communication.<br />
Question 3.1: How important is academic integrity, academic honesty, and acceptable use policies<br />
to you?<br />
Type: Five-Point Likert Scale<br />
Indicator 3.2: Networks with others involved in online education.<br />
Indicator 3.2: Leads collaborative efforts to create common assessments among grade-level and/or<br />
content-area teachers and share assessment results with colleagues to collaboratively plan instruction<br />
that will best meet individual student needs.<br />
Question 3.2: How often do you collaborate with others in the online education field to create,<br />
improve, and/or share instruction strategies?<br />
Type: Nominal (2 or more times daily, daily, weekly, monthly, less than monthly)<br />
5. Research methodology and conclusions<br />
The survey instrument will contain 15 questions following the structure and sequence listed above.<br />
The survey will be provided in electronic form through email to 80 teachers in an online asynchronous<br />
environment. Participants will remain anonymous and personal information will not be collected. The<br />
data will be collected through the survey system and will be entered into SPSS for statistical analysis.<br />
Reliability will be determined using the single-administration method. The internal consistency will be<br />
measured using Cronbach’s alpha. Descriptive statistics including the mean, median, mode, range,<br />
variance, and standard deviation will be calculated. Cross-tabulation will be used to show the<br />
relationship among answers and specifically used for the two questions for Standard 2.3. Bar charts<br />
will be used as the visual representation for many of the distributions.<br />
In summary, the survey instrument design is grounded in specific national online teaching standards<br />
and questions are based on indicators for each standard. Categorizing into NCATE standards will<br />
allow the researcher to show qualifications based on Knowledge, Skills, and Professional<br />
Dispositions. It is important to note that there are not many standards that are associated with<br />
knowledge and professional dispositions. Knowledge is usually proven through other methods of<br />
teacher training, certification, and testing. Professional dispositions usually require a specific survey<br />
for teachers. A standardized teacher disposition instrument would be necessary in order to properly<br />
assess this section.<br />
Teacher assessment and student assessment have similar qualities, but only student assessment is<br />
required by law. If students are performing poorly, a similar measure of the effectiveness of the<br />
teacher should be conducted. But the lack of a consensus on what constitutes good teaching has<br />
made it difficult to create or enforce teacher assessment laws (Rebell, 1991). Student assessment<br />
has become a very important aspect in K-12 education in the United States since the passing of the<br />
No Child Left Behind (NCLB) law in 2002 (U.S DoE, 2009). The assessment was done through tests<br />
developed by each state and determined if the schools were meeting Adequate Yearly Progress<br />
(AYP). The NCLB law also provides funding to help teachers learn to be better teachers, but doesn’t<br />
include any assessment of current teacher quality (U.S. DoE, 2009). The concept of holding students<br />
accountable for low test scores without holding teachers accountable doesn’t appear to be a<br />
sustainable solution. An instrument that can assess what makes an effective teacher, along with the<br />
criteria for effective teaching will be extremely relevant in future education discussions.<br />
Overall, the reliability of the survey instrument for effective online teachers will be examined as an<br />
initial step in the development of the instrument. The self-perception aspect for teachers will allow<br />
organizations to have an anonymous collection of information that assess the online teacher group as<br />
a whole. While it is important in most cases to keep this information anonymous, it is possible to use<br />
this survey as a self-evaluation and collect personal data. With such a rapidly expanding online<br />
education environment the collection of data that can influence standards, practices, certification, and<br />
university programs will be a critical component for the quality of online teachers in the future.<br />
39
References<br />
Jonathan Barkand<br />
Allen, E. I., & Seaman, J. (2010), Class differences: online education in the United States, Sloan Consortium,<br />
Needham, MA, [Online], http://www.sloanconsortium.org/publications/survey/ pdf/class_differences.pdf [10<br />
February 2011]<br />
Kim, K-J, & Bonk, C. J. (2006), The future of online teaching and <strong>learning</strong> in higher education: the survey says...,<br />
Educause Quarterly, [Online], http://faculty.weber.edu/eamsel/ Research%20Groups/Online%20Learning/Bonk%20%282006%29.pdf[12<br />
February 2011]<br />
National Council for Accreditation of Teacher Education (2008), Standards for the Accreditation of Teacher<br />
Preparation Institutions, [Online], http://www.ncate.org/documents/standards/<br />
NCATE%20Standards%202008.pdf [8 January 2011]<br />
National Education Association (n.d), Guide to Teaching Online Courses, [Online],<br />
www.nea.org/assets/docs/onlineteachguide.pdf [8 February 2011]<br />
North American Council for Online Learning (2008), National Standards for Quality Online Teaching, [Online],<br />
www.inacol.org/resources/nationalstandards/NACOL%20Standards%20Quality%20Online%20Teaching.pdf<br />
[8 February 2011]<br />
Picciano, A. G., & Seaman, J. (2009), K-12 online <strong>learning</strong>: A 2008 follow-up of the survey of U.S. school district<br />
administrators, Sloan Consortium, Needham, MA, [Online], http://sloanconsortium.org/sites/default/files/k-<br />
12_online_<strong>learning</strong>_2008.pdf [10 February 2011]<br />
Rebell, M.A.(1991), Teacher Performance Assessment: The Changing State of the Law, [Online],<br />
http://www.pearsonassessments.com/hai/images/NES_Publications/1993_04Rebell_584_1.pdf [15<br />
February, 2011]<br />
Smith, R., Clark, T., and Blomeyer, R. L. (2005), A synthesis of new research on K–12 online <strong>learning</strong>, Naperville,<br />
IL: Learning Point Associates, 2005.<br />
Southern Regional Education Board (2006), Standards for Quality Online courses, Educational Technology<br />
Cooperative, [Online], http://publications.sreb.org/2006/06T05_Standards_quality_online_courses.pdf [8<br />
February 2011]<br />
U.S. Department of Education (2009), Facts and Terms Every Parents Should Know About NCLB, [Online],<br />
http://www2.ed.gov/nclb/overview/intro/parents/parentfacts.pdf [15 February, 2011]<br />
40
When Agents Make Suggestions About Readings<br />
Orlando Belo<br />
Algoritmi R&D Centre, University of Minho, Portugal<br />
obelo@di.uminho.pt<br />
Abstract: Significant efforts have been made during the last few years in the design and implementation of<br />
pedagogical agents for a wide range of application domains. One of the most common target area is the<br />
assistance to students in cases of regular subject studying, promoting means that help them to improve their<br />
performance and expertise in some specific subject areas. Frequently students ask their teachers about the<br />
“best” and more effective bibliographic resources that they could use to study and validate knowledge for some<br />
working topics. In this paper we will discuss the basic characteristics of pedagogical agents, approaching their<br />
typical functional architecture, and services, reinforcing the discussion on a specific class of pedagogical agents<br />
that are responsible to support students during their studying sessions, helping them in the validation of their<br />
knowledge, suggesting bibliographic resources information whenever requested.<br />
Keywords: eLearning platforms, agent based computing, intelligent tutoring systems, software agents, artificial<br />
intelligent tutors, bibliographic resources suggestion<br />
1. Introduction<br />
Agent based applications are very appellative. Software agents have been used to support a lot of<br />
tasks in real world applications (Van der Hoek & Wooldridge 2008). Ranging from telecommunications<br />
to retail, or doing monitoring services on hydroelectric power plants, agent based computing has been<br />
always a very good asset in a lot of problem solving arenas. eLearning is no exception to this<br />
attractive paradigm and to all of its characteristics and potentialities (Agarwal et al. 2004) (Leung & Li<br />
2001). One of the most relevant agent applications on this field was in the design and development of<br />
pedagogical agents, normally designed by artificial tutors or intelligent assistants. Basically, these<br />
entities are conceived to ensure more effective tutoring services, in some very specialized areas of<br />
studying, having clearly pedagogical purposes, giving assistance to students in cases of regular<br />
subject studying or even doing evaluation tasks. Additionally, they are also able to perform<br />
administrative and optimization services inside eLearning platforms, supervising what users are doing<br />
and suggesting better ways to do it or recommending particular resources that can help them in<br />
current tasks.<br />
The use of software agents as sophisticated autonomous means helping students on bibliographic<br />
resource selection seems to be very useful and appellative. The ability that agents have to adapt to<br />
new scenarios and to communicate with other means of <strong>learning</strong> makes possible a very dynamic<br />
eLearning environment, where students needs can be satisfied easily. An agent can also adapt in real<br />
time different plans of readings for current user needs, finding the best solution for a given studying<br />
plan, and personalizing processes and exploitation scenarios (Schiaffino et al. 2008). To do that, they<br />
simply need to act as search engines over their databases looking for a list of references that satisfies<br />
student current, his preferences, and a previous plan of readings prepared by his teachers. This<br />
means that any assistant agent (Okamoto et al. 2009) must establish usage profiles and accordingly<br />
prepare its plan of action an user satisfaction. It is very desirable that the processes of readings<br />
suggestion is versatile and proactive, providing the references that students require and, at the same<br />
time, giving viable alternatives that follow other suggestion indicators (e.g. readings usage ranking,<br />
recommended references, teacher’s preferences, or external sources identification).<br />
In this paper we will focus our attention on a specific class of pedagogical agents that are responsible<br />
to support students during their studying sessions, helping them in the validation of their knowledge in<br />
a particular domain, and (as their priority goal) suggesting bibliographic resources information<br />
whenever requested or inferred as necessary during an evaluation process, accordingly current<br />
studying status of the students. We will discuss the basic characteristics of these pedagogical agents<br />
(section 2), reinforcing the discussion presenting an application domain for bibliographic resource<br />
suggestion in a conventional eLearning scenario (section 3), and present the basic characteristics and<br />
functional architecture of a specific software agent for personalised assistance in bibliographic<br />
resource suggestion (section 4). Finally, in section 5, we will present some conclusions and future<br />
work.<br />
41
2. Pedagogical agents<br />
Orlando Belo<br />
Agent-based computing is not a recent area of research and technological development (Wooldridge<br />
1998) (Jennings 1999). A lot of work and applications were made for a large diversity of domains<br />
(Jennings & Wooldridge 1998). The agent paradigm is by nature quite attractive inspiring researchers<br />
from many distinct application arenas. During the last few years, agents have been introduced in a<br />
large range of applications, ranging from financial markets to medical services, passing by<br />
telecommunications, retail, or transports. Today, we can find agents doing a lot of tasks, 24 hours a<br />
day, 365 day a year, without interruptions.<br />
In a few set of links, a simple Web search reveals us that eLearning is no exception in the adoption of<br />
agent-based technology (Nedev & Nedeva 2008) (Gregg 2007) (Kazar & Bahi 2009). Intelligent or<br />
not, agents have been developing important roles in the eLearning arena, improving effectiveness of<br />
educational games (Conati & Zhao 2004), done procedural training in virtual environments (Jonhson<br />
& Rickel 1997), supporting distance education (Silveira & Vicari 2002), animating educational<br />
sessions (Nunes et al. 2002) (Lester et al. 1997), doing virtual tutorship (Remondino 2007), retrieving<br />
selective information from the web or other information sources (Pankratius et al 2004), or managing<br />
processes in eLearning environments (Lai et al. 2008). This is only a brief panorama of what we can<br />
find about agents in eLearning environments.<br />
Pedagogical agents could be seen today as good alternatives to some parts of the educational<br />
process, being very useful in the formative process of students. Some of the most experienced<br />
eLearning solutions integrate now in some way “expert” software components, especially in areas<br />
where attendance services must be ensured continuously, 24 hours per day. Pedagogical agents are<br />
designed and implemented with the goal to support students in their daily work, providing information<br />
about their classes, supporting studying processes, accessing, or retrieving pertinent information<br />
about some topics. Basically, they intend to improve students <strong>learning</strong> outcomes. Thus, we can see<br />
that this particular kind of agents have the ability to attenuate several usage restrictions and user<br />
services limitations that we can find normally in eLearning platforms (Brusilovsky 2000) (Zhang et al.<br />
2004).<br />
One of the most sophisticated aspects that we are particularly interested to explore is customization<br />
(namely frequently by personalization) of eLearning processes and services (Allison et al. 2005).<br />
Trying to go towards the needs of students, providing the <strong>learning</strong> materials they need and<br />
complement them with other resources in anticipation it’s a clear goal for us. The use of profiling<br />
techniques (Marques & Belo 2010) to establish a standard modus operandi of the student and predict<br />
what students will need and use are quite important for eLearning services personalization (Paneva<br />
and Zhelev 2007). This allows to catch individual studying styles and resources options used,<br />
especially the ones concerning about bibliographic references support – the goal of this work –,<br />
personalizing them and creating more student oriented <strong>learning</strong> objectives and material support. In<br />
the following sections we will see how we used agent technology to design and develop a<br />
bibliographic resource assistant, with the ability to follow studying sessions, receiving reference<br />
requests about some particular studying topic, and suggesting on the fly a set of references that could<br />
help students and contribute to the improvement of knowledge acquisition and, as referred before, of<br />
<strong>learning</strong> outcomes.<br />
3. A bibliographic resource assistant<br />
3.1 Overview<br />
Students are very critical and demanding about bibliographic resources. They never are satisfied with<br />
teachers’ proposals and suggestions concerning with what they must or need to read about some<br />
course’s programme topic. Today, we have strong reasons to believe that a significant part of the<br />
students’ failures is due to their inability to select appropriated bibliographic resources and use it.<br />
Teachers use to indicate in every beginning of a scholar year at least one bibliographic list covering<br />
the topics they approach in classes. Some of them even try to explain for each entry presented in the<br />
lists its coverage and importance. But it seems that such efforts are useless so many times.<br />
In order to attenuate such problem we design a small software component especially oriented to<br />
assist and help students during their working sessions. It wasn’t our intention to conceive a generic<br />
assistant. However all the pointed development lines followed here could be used in such direction.<br />
42
Orlando Belo<br />
Data Warehousing Systems (Golfarelli & Rizzi 2009) were our main scientific and technological target,<br />
which means that we intended to provide an auxiliary mean for studying in the area of decisionsupport<br />
systems, business intelligence, and their related technologies. The target community selected<br />
- university students - is also very specific. But, in our point of view, is sufficient relevant (and critical)<br />
to apply and test such mean of studying support. So, please keep on mind the referred area and<br />
student community during the rest of the paper.<br />
3.2 Readings suggestion life cycle<br />
Our main goal was to design and specified all services and functionalities of specific tool with the<br />
ability to provide studying maps, locating for each course’s programme topic a list of recommended<br />
readings, and promoting, when requested, cross-reference recommendation scenarios for different<br />
working areas. This tool, a specific subject-oriented software agent, was idealized as a typical<br />
assistant, which can be seen as a conventional pedagogical agent specialized in bibliographic<br />
resource suggestion. The way we though this studying assistant, its functional structure and<br />
behaviour, demands some preliminary work, usually done by the lecturing team of a university’s<br />
course, they will be responsible for preparing the bibliographic references according the programme<br />
that they establish for a course.<br />
Easily we see that to provide valid and effective bibliographic suggestions, it is necessary (at least) to<br />
maintain up-to-date a bibliographic catalogue, including a detail reference list, their correspondent<br />
index terms for searching and cross reference, as well a categorization and an importance level<br />
associated with each references (or a set of references) for each specific studying topic referred in the<br />
course’s programme. Having this information we own the minimum to establish a first set of<br />
suggestion. The rest, the most sophisticated part of this project, which consider inclusive student<br />
preferences definition, will be achieved with special techniques of profiling and preferences<br />
establishment.<br />
Figure 1: Readings suggestion process life cycle<br />
The assistant agent’s behaviour follows a very common life cycle for cases related with suggesting<br />
and index based searches. As we can see in Figure 1, according such life cycle, the first task that we<br />
must perform is to prepare a detail (readings list definition) list containing the bibliographic references<br />
(Table 1) recommended by the lecturing team. It is one of the most important pieces of information<br />
that the readings assistant agent has to support its job. This list includes some elementary data about<br />
bibliographic references, complemented with some other attributes that will be used by the agent to<br />
establish suggestion priorities and preferences about readings. The basic structure of the list,<br />
excluding control and monitoring attributes, includes the following fields:<br />
Identification (Id), which identifies uniquely the bibliographic reference in the entire system.<br />
CiteRef, the way reference could be referred.<br />
Reference, the complete description of the reference.<br />
Year, the publication year of the bibliographic reference.<br />
43
Orlando Belo<br />
Type (Tp), the reference type (B-Book, P-Proceedings, J-Journal, etc.).<br />
Area, the knowledge area of the reference.<br />
Importance level (I), the value of importance that the lecturing team established for it, based on<br />
the programme defined; we assumed 1(min) .. 5(max) as the importance scale to associate with<br />
each reference included in the list.<br />
Preference status (P), the preference value that is calculated accordingly the use given by<br />
students; it follows the same scale of the importance level and it’s calculated during the studying<br />
sessions.<br />
Keywords, some priority terms for searching and cross-referencing.<br />
Link, that is an eventual identification of a source where the document could be found.<br />
Figure 2: An excerpt of the semantic network for bibliographic references<br />
Latter, the reference list will be complemented with some additional terms that will establish the<br />
relationships among the references themselves and all the topics presented in the course’s<br />
programme. This will allow the agent in the suggestion process, giving it the basis to present, for a<br />
particular topic (introductory concepts, conceptual modelling, development life cycle, data warehouse<br />
monitoring, optimizations, etc.), the most appropriated references, taking into consideration their<br />
importance and preference levels (if requested).<br />
Table 1: Part of a bibliographic reference list preliminary description<br />
T<br />
Id CiteRef Reference Year p Area I P Keywords Link<br />
1 Kimball Kimball, R., Reeves, 2008 B DW 4 - Data Warehousing, --et<br />
al. L., Ross, M.,<br />
Data Warehouses,<br />
2008 Thornthwait, W.<br />
Dimensional Modelling,<br />
(2008), The Data<br />
System Development<br />
Warehouse Life<br />
cycle Toolkit –<br />
Piratical Techniques<br />
for Building Data<br />
Warehouse and<br />
Business<br />
Intelligence<br />
Systems, John<br />
Wiley & Sons, 2ª<br />
Edição.<br />
Life Cycle, …<br />
2 Golfarell<br />
i &<br />
Rizzii<br />
2009<br />
M. Golfarelli, S.<br />
Rizzi (2009), “Data<br />
Warehouse Design:<br />
Modern Principles<br />
and Methodologies”,<br />
McGraw-Hill.<br />
2009 B DW 5 - Data Warehousing,<br />
Data Warehouses,<br />
System Life Cycle,<br />
Analysis and<br />
Conciliation of Data<br />
Sources, User<br />
Requirements,<br />
44<br />
---
3 Vassilia<br />
dis et al.<br />
2001<br />
P. Vassiliadis, C.<br />
Quix, Y. Vassiliou,<br />
M. Jarke (2001),<br />
“Data Warehouse<br />
Process<br />
Management”, Inf.<br />
Syst. 26, 3 (May<br />
2001), 205-236.<br />
Orlando Belo<br />
<br />
2001 J DW 3 -<br />
Conceptual Modelling<br />
and Design, …<br />
DataWarehousingProc<br />
essModellingEvolution<br />
Quality, …<br />
http://citeseerx<br />
.ist.psu.edu/vie<br />
wdoc/downloa<br />
d?doi=10.1.1.2<br />
8.7227&rep=re<br />
p1&type=pdf<br />
Id CiteRef Reference Year<br />
T<br />
p Area I P Keywords Link<br />
4 Inmon Inmon, W.H., 1996 B DW 3 - Data Warehousing, ---<br />
1996 Building the Data<br />
Design and Build a<br />
Warehouse , John<br />
Data Warehouse, Data<br />
Wiley & Sons, 1996.<br />
Warehouse Monitoring,<br />
Data Models, …<br />
5 Romero O. Romero, A. 2009 C DW 5 - Multidimensional http://citeseerx<br />
& Abelló Abelló: A Survey of<br />
Modelling Survey, .ist.psu.edu/vie<br />
2009 Multidimensional<br />
Comparison, Data wdoc/summar<br />
Modeling<br />
Warehousing,<br />
y?doi=10.1.1.1<br />
Methodologies.<br />
Multidimensional 57.2644&rank<br />
IJDWM 5(2): 1-23<br />
(2009)<br />
Design, OLAP, … =1<br />
(.) (.) (.) (.) (.) (.) (.) (.) (.)<br />
After, we will gather the text sources (if available) and full-index them (index terms generation), in<br />
order to provide data to refined searching and topic context definition. If due to some circumstances<br />
we cannot do full indexing, we substitute it as much as possible with a list of searchable terms<br />
collected in reviews or web pages where the text could be referred or commented. With the reference<br />
list data and the generated indexes we build a concept map (suggestion knowledge acquisition),<br />
representing the knowledge (Figure 2) that will support the suggestion work. With the execution of<br />
these three initials tasks execution we finish the first stage of the readings suggestion life cycle – the<br />
data preparation.<br />
The second stage – studying session evaluation – is concerned exclusively in following what students<br />
request to the agent, and what kid of suggestion that accept and preferred. During this process, the<br />
agent stores all the requests done – studying session profile -, per student and per studying session,<br />
and the readings selection selected. Finishing a studying session, the agent evaluates the preference<br />
level for each reference that was selected – student preferences evaluation -, and rewrites its own<br />
knowledge base, updating the student profile, and registering its preferences. As we can predict,<br />
process after process, the agent’s knowledge base will be richer, and will provide better readings<br />
suggestions and cross-referring paths. This second task group of the agent’s life cycle will be<br />
executed so many times as the number of studying sessions performed under its assistance. While<br />
the first one is executed only when the lecturing team decide to restructured the course’s programme<br />
and the list of bibliographic references.<br />
Figure 3: Agent’s conceptual architecture components<br />
45
3.3 The agent’s architecture<br />
Orlando Belo<br />
To be able to support all the most common services required by a typical process for bibliographic<br />
references suggestion, we designed the agent’s architecture (Figure 3) organized into three functional<br />
layers with the following components:<br />
Layer 1 – Interface<br />
User Interface, which provides sensors and actuators to establish and maintain communication<br />
between the agent and the students.<br />
Information Retrieval, with the special task to retrieve document’s sources from the Web and<br />
make their full indexation.<br />
Layer 2 – Reasoning<br />
Reasoning, which receives the user requests, evaluating them and generating suggestion about<br />
some bibliographic references, taking into consideration the student’s preferences (if already<br />
defined) and the importance level attributed by the lecturing team.<br />
Acquaintance, a component that keeps information about other assistant agents (application area,<br />
expertise and knowledge, references lists and application domains, etc.) working in the same<br />
platform.<br />
Monitoring and Control, which observes and records all user requests and their reference lists<br />
selection for later user profiling and preferences evaluation.<br />
Layer 3 – Knowledge and Data Access<br />
Working Memory, which keeps intermediate results and control parameters.<br />
Knowledge Base, a knowledge repository that keeps all the data objects and their relation ships<br />
about bibliographic references and their relation with the topics of the course’s programme – see<br />
a brief example in Figure 3.<br />
This agent software was conceived to work as a complementary tutoring component that can be<br />
integrated in an eLearning platform, acting accordingly its directives and communicate with other<br />
services. The model proposed in this paper will allow the implementation of an assistant agent that<br />
would reduce lecturing teams efforts and time wasted in the explanation and presentation of<br />
bibliographic resources. At the same time will provide an entity permanently available and that can be<br />
access almost everywhere if we have access to it. It was also considered the integration of several<br />
assistant agents in the same platform, in order to ensure access to different bibliographic resources<br />
and promote agent cooperation in cases of bibliographic cross-referencing.<br />
4. Conclusions and future work<br />
Today, eLearning platforms are introduced in almost teaching institutions. From a partial fulfilling of<br />
some specific oriented services to a global integration in all aspects of teaching, eLearning tools<br />
perform very important roles in <strong>learning</strong> processes, providing tutoring and supporting services<br />
permanently, and with easy access for the entire student (and teaching) community. In this work we<br />
were concerned with only a specific aspect of tutoring: bibliographic suggestion; with the simple<br />
objective to help and support university students improving their <strong>learning</strong> outcomes. The option by an<br />
agent application was related to the intrinsic characteristics of any agent-based applications:<br />
autonomy, adaptation and cooperation. The proposed model followed the characteristics of the most<br />
common agent-based systems, differing essentially in how the agent will acquire and maintain its<br />
knowledge, and extend its application context. The rest of the components are quite regular, as well<br />
as the global behaviour of the agent.<br />
According to the most basic characteristics of the architecture proposed for the implementation of a<br />
personal assistant for bibliographic resources suggestions, we intend to implement the assistant<br />
agent as soon as possible, following the FIPA (The Foundation for Intelligent Physical Agents -<br />
http://www.fipa.org/) specifications, which provides us a set of well know standards for agent-based<br />
services and inter-agent communication. Additionally, to ensure a well-structured and robust platform<br />
for agents we will use the JADE platform (Bellifemine et al. 2008), a generic software for agent-based<br />
applications, which provides the most basic functionalities required by software agents (asynchronous<br />
messages, agent life cycle management, mechanisms for agent subscription, ontology support, etc.).<br />
Jade supports all the major FIPA’s specifications and will allow us to develop rapidly the software for<br />
the bibliographic assistant.<br />
46
References<br />
Orlando Belo<br />
Agarwal, R., Deo, A., Das, S. (2004), “Intelligent agents in ELearning”, SIGSOFT Softw. Eng. Notes 29, 2 (March<br />
2004), 1-1.<br />
Allison, C., Cerri, S.A., Ritrivato, P., Gaeta, A. Gaeta, M. (2005), “Services, semantics and standards: elements of<br />
a <strong>learning</strong> grid infrastructure”, Applied Artificial Intelligence, Vol. 19, pp. 861-79.Bellifemine, F., Caire, G.,<br />
Greenwood, D: (2008). “Developing multi-agent systems with Jade.” New York, John Wiley & Sons, Ltd.<br />
Brusilovsky, P. (2000), “Adaptive hypermedia: from intelligent tutoring systems to web-based education (invited<br />
talk)”, in Gauthier, G., Frasson, C. and VanLehn, K. (Eds), Intelligent Tutoring Systems, Lecture Notes in<br />
Computer Science, Vol. 1839, Springer Verlag, Berlin, pp. 1-7.<br />
Conati, C., Zhao, X. (2004), “Building and Evaluating an Intelligent Pedagogical Agent to Improve the<br />
Effectiveness of an Educational Game”, IUI-CADUI ’04, January 13-16, 2004, Island of Madeira, Portugal.<br />
Golfarelli, M., Rizzi, S. (2009), “Data Warehouse Design: Modern Principles and Methodologies”, McGraw-Hill.<br />
Gregg, D. (2007), “ELearning agents”, The Learning Organization Vol. 14 No. 4, 2007 pp. 300-312.<br />
Van der Hoek, W., Wooldridge, M. (2008), “Multi-Agent Systems”, In F. van Harmelen, V. Lifschitz, and B. Porter,<br />
editors, Handbook of Knowledge Representation, pages 887--928. Elsevier.<br />
Jennings, N. (1999), "Agent-based Computing: Promise and Perils" Proc. 16th Int. Joint Conf. on Artificial<br />
Intelligence (IJCAI), Stockholm, Sweden. (Computers and Thought award invited paper) 1429-1436.<br />
Jennings, N., Wooldridge, M. (1998), "Applications of Intelligent Agents" in Agent Technology: Foundations,<br />
Applications, and Markets (eds. N. R. Jennings and M. Wooldridge) 3-28.<br />
Johnson, W., Rickel, J. (1997), “Steve: an animated pedagogical agent for procedural training in virtual<br />
environments”, SIGART Bull. 8, 1-4 (December 1997), 16-21.<br />
Kazar, O., Bahi, N. (2009), "Agent based approach for ELearning", Page(s): 126-131, MASAUM Journal of<br />
Computing (MJC) (ISSN 2076-0833), Volume: 1 Issue: 2 Month: September 2009.<br />
Lai, H., Wang, M., He, J., Wang, H. (2008), "An Agent-Based Approach to Process Management in ELearning<br />
Environments", IJIIT, pp.18-30.<br />
Lester, J., Converse, S., Stone, B., Kahler, S., Barlow, S. (1997), “Animated pedagogical agents and problemsolving<br />
effectiveness: A large-scale empirical evaluation”, In Proceedings of Eighth World Conference on<br />
Artificial Intelligence in Education, pp 23-30.<br />
Leung, E., Li, Q. (2001), “Agent-Based Approach to eLearning: An Architectural Framework”, In Proceedings of<br />
the First International Conference on The Human Society and the Internet - Internet Related Socio-<br />
Economic Issues, Won Kim, Tok Wang Ling, Yoon-Joon Lee, and Seung-Soo Park (Eds.). Springer-Verlag,<br />
London, UK, 341-353.<br />
Marques, A., Belo, O. (2010), “Discovering Usage Profiles on Web Based eLearning Platforms Using Markov<br />
Chains”, In Proceedings of 9th European Conference on eLearning (ECEL’2010), Instituto Superior de<br />
Engenharia do Porto, Porto, Portugal, 4-5 November.<br />
Nedev, D., Nedeva, V. (2008), “Aspects of multi-agent systems application in eLearning”, Computer<br />
Science’2008, Kavala, Greece, 18-19 September 2008, p. 1022-1028.<br />
Nunes, M., Dihl, L., Fraga, L., Woszezenki, C., Oliveira, L., Francisco, D., Machado, G., Nogueira, C.,<br />
Notargiacomo, M. (2002), “Animated Pedagogical Agent in the Intelligent Virtual Teaching Environment”,<br />
Interactive Educational Multimedia, No. 4, April.<br />
Okamoto, S., Sycara, K., Scerri, P. (2009), Personal Assistants for Human Organizations, in Multi-Agent Systems<br />
- Semantics and Dynamics of Organizational Models, Ed. V. Dignum, IGI-Global, Hershey, Pennsylvania,<br />
U.S.A.<br />
Paneva, D., Zhelev, Y. (2007), “Models, Techniques and Applications of eLearning Personalization”, International<br />
Journal "Information Technologies and Knowledge" Vol.1.<br />
Pankratius, V., Olivier, S., Stucky, W. (2004), “Retrieving content with agents in web service eLearning systems”,<br />
Symposium on Professional Practice in AI, IFIP WG12.5 – in Proceedings of the First IFIP Conference on<br />
Artificial Intelligence Applications and Innovations (AIAI), 91-100.<br />
Remondino, M. (2007), “Agent Based Virtual Tutorship and ELearning Techniques Applied to a Business Game<br />
Built on System Dynamics”, GAMEON 2007 Proceedings,157.<br />
Schiaffino, S., Amandi, A., Gasparini, I., Pimenta, M. (2008), “Personalization in eLearning: the adaptive system<br />
vs. the intelligent agent approaches”, In Proceedings of the VIII Brazilian Symposium on Human Factors in<br />
Computing Systems (IHC '08). Sociedade Brasileira de Computação, Porto Alegre, Brazil, Brazil, 186-195.<br />
Silveira, R., Vicari, R. (2002), “Developing Distributed Intelligent Learning Environment with JADE - Java Agents<br />
for Distance Education Framework”, In Proceedings of the 6th International Conference on Intelligent<br />
Tutoring Systems (ITS '02), S. A. Cerri, G Gouardères, and F. Paraguacu (Eds.). Springer-Verlag, London,<br />
UK, UK, 105-118.<br />
Wooldridge, M. (1998), “Agent-based computing”, In Interoperable Communication Networks. 1(1), pages 71-97.<br />
January 1998.<br />
Zhang, D., Zhao, J.L., Zhou, L. and Nunamaker, J.F. (2004), “Can eLearning replace classroom <strong>learning</strong>?”,<br />
Communications of the ACM, Vol. 47 No. 5, pp. 75-9.<br />
47
Some Reflections on the Evaluation of Virtual Learning<br />
Environments<br />
Nabil Ben Abdallah and Françoise Poyet<br />
ELICO (Équipe de recherche de Lyon en sciences de l’Information et<br />
COmmunication), Lyon, France<br />
nabil.ben-abdallah@iut-dijon.u-bourgogne.fr<br />
francoise.poyet@free.fr<br />
Abstract: This paper aims at pondering over the evaluation question of the VLE. We have carried out an<br />
overview of concepts and methods commonly used in Human-Computer Interaction (HCI) in the evaluation of<br />
information systems and then we have discussed the limits of their application on the VLE. We are proposing to<br />
evaluate the latters according to an activity-oriented approach. It will allow us to expand our field of study to<br />
<strong>learning</strong> situations in which a VLE can be integrated. The <strong>learning</strong> activity is thus observed not only through the<br />
interactions of motivated learners and teachers with VLE but also with the objects and objectives of <strong>learning</strong><br />
implemented via VLE. This observation is based on the theoretical framework proposed by activity theory. The<br />
Engeström model as a form of conceptualization of the structure of activity is essentially explored to understand<br />
the elements interacting in a <strong>learning</strong> situation integrating VLE. We conclude our contribution by the application of<br />
ideas developed by observing the use of a VLE in a <strong>learning</strong> situation of information seeking and involving small<br />
groups of students.<br />
Keywords: VLE evaluation; activity theory; usability; utility; practical acceptability; social acceptability<br />
1. Introduction<br />
The increasingly generalized use of Virtual Learning Environments (VLE) in different <strong>learning</strong><br />
situations places the issue of the evaluation of their impacts on <strong>learning</strong> as well as on the<br />
organizations at the core of designers and managers’ concerns regarding the systems of <strong>learning</strong>.<br />
VLE is not only a technical solution integrating heterogeneous technologies, it is also a space of<br />
information around which a social space that organizes and structures the various interactions<br />
necessary for any <strong>learning</strong> activity can be formed and developed. Several studies and research have<br />
been carried out (Britain 2004; Colace 2006; Keller 2005; Dobson 2001) to help the actors of <strong>learning</strong><br />
to choose the appropriate VLE so as to cope with specific situations of <strong>learning</strong>, to assess the<br />
consequences of the use of these VLEs on learners and on <strong>learning</strong> itself, and to identify interaction<br />
problems with these VLEs.<br />
In this work, the <strong>learning</strong> activity is observed not only through the interactions of motivated learners<br />
and teachers with VLE but also with the objects and objectives of <strong>learning</strong> implemented via VLE. This<br />
observation is based on the theoretical framework proposed by activity theory. The Engeström model<br />
(Engeström 1987) as a form of conceptualization of the structure of activity is essentially explored to<br />
understand the elements interacting in a <strong>learning</strong> situation integrating VLE.<br />
The data resulting from the analysis of the activity system of <strong>learning</strong> based on the model of<br />
Engeström enable us to get, on the one hand, data on the degree of practical and social acceptability<br />
of VLE, and on the other hand, data that can be used to improve the methods of usability and utility<br />
evaluation of VLE. We conclude our contribution by the application of ideas developed above by<br />
observing the use of a VLE in a <strong>learning</strong> situation of information seeking and involving small groups of<br />
students.<br />
2. What is a VLE?<br />
According to Weller (Weller 2007, p.5) a VLE is: "a software system that combines a number of<br />
different tools that are used to systematically deliver content online and to facilitate the <strong>learning</strong><br />
experience around that content". Other authors such as Colace and al. (Colace 2006, p.2) admit that<br />
a VLE or a eLearning platform is organized in three macro-components: "In our opinion the most part<br />
of contemporary eLearning platform can be viewed as organized into three fundamental macro<br />
components: a Learning Management System (LMS), a Learning Content Management System<br />
(LCMS) and a Set of Tools for distributing training contents and for providing interaction". In the light<br />
of these definitions and some others, we are considering that a VLE is a particular computer system<br />
allowing to create <strong>learning</strong> situations with specific <strong>learning</strong> objectives (e.g. <strong>learning</strong> such a type of<br />
knowledge). A VLE is usually conceived with the aim to train the <strong>learning</strong> actors into achieving these<br />
48
Nabil Ben Abdallah and Françoise Poyet<br />
objectives through a favorable activity. VLE offers a set of integrated functionalities to complete the<br />
requirements of various <strong>learning</strong> situations. Four groups of functionalities are usually mentioned:<br />
course management (design, maintenance, and diffusion), tracking <strong>learning</strong>, technical administration<br />
(set-up, security, integration, maintenance) and learner interface (offering communication and<br />
collective work tools, and a customizable space). Depending on the <strong>learning</strong> objectives, a VLE must<br />
also propose expanded parameterization facilities to create more or less elaborate <strong>learning</strong> situations.<br />
eLearning platforms such as Blackboard, Moodle, Claroline, and others are examples of VLE (either<br />
proprietary or free software).<br />
From a technical point of some view, a VLE integrates various types of software tools to support the<br />
various functionalities mentioned in the preceding paragraph. For examples we have: a database<br />
management server, a web server, a mail server, a file server. These servers are developed with<br />
technologies of various origins, mostly meeting with the requirements of various standards. Among<br />
these technologies, those of the Internet have had considerable repercussions on the design of VLEs.<br />
In addition to the technical possibility of having applications interconnected and making use of<br />
resources coming from different sites, collaborative or co-operative activities and as well as human<br />
interaction seem to be favored thanks to tools enabling communication, sharing and the production of<br />
documents.<br />
The use of a VLE encourages written communication and, except with the synchronous<br />
communication tools, makes the learner’s participation less improvised because he has a certain<br />
amount of time for consideration and easy access to resources enabling him to enrich his ideas and<br />
to pursue his thinking. In short, a <strong>learning</strong> situation built around a VLE bears little resemblance to a<br />
conventional class where student is constantly under a "social pressure" emerging from the activity of<br />
the other students and the requirements of the teacher. The metaphors (class, group, office, etc.)<br />
used in the design of VLEs (Falconer 2008) should not allow us to forget that this is above all a<br />
computer system having features which are not those of a <strong>learning</strong> device built around a<br />
conventional class.<br />
In order to understand these characteristics and thus expand our specific acception of VLEs beyond<br />
the principle integrating functional and technological aspects, we quote here Dillenbourg and his<br />
colleagues (Dillenbourg 2002) who think that VLEs can be identified through six characteristics : 1) A<br />
virtual <strong>learning</strong> environment is a designed information space 2) A virtual <strong>learning</strong> environment is a<br />
social space 3) The virtual space is explicitly represented 4) Students are not only active, but also<br />
actors 5) Virtual <strong>learning</strong> environments integrate heterogeneous technologies and multiple<br />
pedagogical approaches 6) Most virtual environments overlap with physical environments.<br />
It is obvious that a given VLE may not verify all these characteristics, the objective here is to have a<br />
comprehensive understanding of roles that a VLE can play in <strong>learning</strong> activities. The information<br />
space being composed, among other things, of the courses and of all the information resources that<br />
enrich and complement them must be sufficiently structured and organized so that it can be suitably<br />
exploited in <strong>learning</strong> activities. Beyond this information space, a social environment which organizes<br />
and structures the various interactions (learner to learner, learner to teacher, etc.) which are essential<br />
to any form of <strong>learning</strong> activity, is built up. The <strong>learning</strong> environment can be based on more or less<br />
sophisticated interfaces going from simple text interfaces to graphical user interfaces in 3D. Whatever<br />
the interface used, it must be sufficiently explicit so that learners can achieve the proposed <strong>learning</strong>.<br />
Although VLEs are often associated with distance <strong>learning</strong> because they are essentially Web-based<br />
applications, they are also widely used as a support for classroom teaching. Integration within a VLE<br />
of several software tools makes the adaptation of various approaches possible. For example,<br />
teachers can request individual or collective work from the students and decide to intervene or not in<br />
the various phases of the completion of this work. The package of software tools (exerciser, wiki, blog,<br />
chat ...) offered by a VLE makes it possible to adopt one or the other approach.<br />
All these characteristics are important to set up a <strong>learning</strong> situation with precise objectives. Beyond<br />
the impact which these VLE can have on the <strong>learning</strong> outcome, it is the learner's statute which<br />
changes: from a passive learner who merely memorizes some patterns or knowledges which are<br />
passed on to him, we will end up with a learner who is not only active but who is also an actor of his<br />
own <strong>learning</strong>. He will use and produce information, and he will continuous interaction with <strong>learning</strong><br />
environment. According to Dillenbourg (Dillenbourg 2002, p.7) "… the notion of a <strong>learning</strong> activity in<br />
virtual <strong>learning</strong> environments refers to something richer than in individual courseware, closer to the<br />
49
Nabil Ben Abdallah and Françoise Poyet<br />
notion of project. The difference between other constructivist environments and what virtual<br />
environments potentially offer can be described as making students not only active, but also actors,<br />
i.e. members and contributors of the social and information space". For example, if we consider the<br />
<strong>learning</strong> user, these theoretical possibilities provided by a VLE should not make us forget that there is<br />
always a gap between the use of VLE as it is defined by the designers according to the representation<br />
that they have of <strong>learning</strong> activities, and the use of VLE as actually practiced by the learner as<br />
regards what he must do to achieve his objectives. The fact that the use of the VLE by the learner is<br />
different from that defined by the designer is not in itself a design problem or a dysfunction in <strong>learning</strong>,<br />
it is inherent to what Perriault (Perriault 2008) called the logic of use which is construction by the<br />
individual of a choice of instrument and a kind of employment he uses to achieve a project.<br />
This description of the various facets which a VLE can have is necessary to even better understand<br />
the difficulties that we have to evaluate it. Indeed, for this evaluation to be reasonably reliable, it must<br />
include both the product VLE (interface and functionalities) and the <strong>learning</strong>s it supports. We believe<br />
the facility with which a VLE can be used and the variety of its functionalities alone does not explain<br />
the success or the failure of its design. Other factors must be taken into account to understand and<br />
perhaps predict the adoption or not of a given VLE by the <strong>learning</strong> actors.<br />
This specific acception of the evaluation takes again the distinction made by Nielsen (Nielsen, 1994)<br />
between practical acceptability and social acceptability of a computer system. Nielsen does not<br />
describe the components of social acceptability in detail, but it clearly emphasizes its significance in<br />
the overall system acceptability. We are indeed dealing with social acceptability which refers to the<br />
way in which the <strong>learning</strong> actors perceive the different stakes related to the use of a VLE in a <strong>learning</strong><br />
situation. These actors react favorably or unfavorably to the use of VLE according to opportunity,<br />
risks, and benefits that may arise from this use. Practical acceptability refers to a kind of "diagnostic of<br />
use" in which the designers as well as the <strong>learning</strong> actors are invited to measure the utility, for<br />
<strong>learning</strong> tasks, of functionalities suggested by a VLE and the way in which they can be used. Here<br />
they are two principal categories of a computer system evaluation: usability and utility (we will deal<br />
with that idea later). Practical acceptability can also take into account the factors (such as the<br />
compatibility with the existing systems, the cost, the reliability, etc.) which support the integration of a<br />
VLE in a given <strong>learning</strong> situation.<br />
3. Evaluation of computer systems and VLEs<br />
The concept of usability, introduced by Eason (Eason, 1984) and developed by other authors (Nielsen<br />
1994; Bastien 2001; Rosson 2001, Shneiderman 2005) generally refers to the quality of the interface.<br />
Bastien suggest that the usability is defined in a way much broader designating the related aspects as<br />
much in the use of the computer systems that those related to the utility of these system. The two<br />
dimensions (usability and utility) are therefore not completely independent: an interface with good<br />
ergonomic qualities (optimal usability) cannot alone satisfy the user’s expectations if essential<br />
functions for carrying out the task (real utility) are not implemented in the application. Standard ISO<br />
9241-11 defines the usability of a computer system as follows: "the extent to which a product can be<br />
used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a<br />
specified context of use".<br />
Two approaches are generally used to evaluate the computer systems: analytical and empirical; these<br />
approaches are not necessarily opposite but complementary. The data collected from empirical<br />
methods are primarily behavioral; they are used to locate the problems encountered by users. As for<br />
data collected from analytical methods, these provide information on the ergonomic quality of the<br />
interface without the user directly interacting with the system.<br />
The evaluation methods can be complementary according to the objective of the evaluation, to the<br />
means used and the moment of the life cycle of the application. Dix (Dix, 1998) defined eight criteria<br />
to help the appraiser in the choice of a suitable evaluation technique. They are: the time of the<br />
evaluation (in relation to the life cycle of the application), the type of data to be collected, human and<br />
material resources which are available, constraints imposed by the evaluation, mode of evaluation<br />
(laboratory or field), the degree of objectivity or subjectivity sought, information available and data<br />
acquisition mode (online/offline). One or several methods may be used to collect the maximum<br />
amount of information on the system being observed in order to bring the evaluation operation to a<br />
successful conclusion. For example, interviews and questionnaires may supplement cognitive<br />
walkthrough (Lewis 1992; Mahatody 2010) by providing information on the characteristics of the users<br />
50
Nabil Ben Abdallah and Françoise Poyet<br />
and the tasks to be performed. In an article devoted to studying methods by inspection (“heuristic<br />
evaluation, cognitive walkthrough, formal usability inspections, and pluralistic usability walkthrough”),<br />
Hollingsed and Novick (Hollingsed 2007) assert that heuristic evaluation and cognitive walkthrough<br />
are methods that are still actively used and they enables to obtain the best possible evaluation in<br />
terms of usability.<br />
A VLE, as a particular computer system, offers a set of functionalities to carry out various activities<br />
directly or indirectly related to <strong>learning</strong> processes. The utility of these suggested functionalities as well<br />
as the way in which they are used by the users may be evaluated by HCI methods based on an<br />
analytical or/and empirical approach (Huart 2004, Hvannberg 2007). Let us examine here if the<br />
cognitive walkthrough method allows us to evaluate the usability and utility of a VLE. This method<br />
requires a specific description of the tasks to be carried out using the computer system, a description<br />
of the actions which the user must carry out in order to perform these tasks and, finally, a general<br />
description of the users and context of use. The appraiser explores the interface in four stages of the<br />
human-computer interaction, to evaluate the facility with which actions will be carried out. The tests<br />
make it possible to determine the parts of the interface which are likely to hinder optimal use of the<br />
application. For a VLE, the description of the tasks to be performed and the sequence of the actions<br />
to be carried out pose a problem for the appraiser. The tasks carried out, for example, by a userlearner<br />
are indeed complex and are closely connected. The appraiser must be able to describe the<br />
reality of a <strong>learning</strong> activity involving different actions which are not naturally carried out in a<br />
predefined sequence; within this <strong>learning</strong> activity, the commitment of the learner is not always<br />
predictable. If the <strong>learning</strong> situation had been broken down into scenarios the appraiser could use the<br />
scenario which seems the most relevant according to the elements to be evaluated. He must be able<br />
to operationalize and implement the chosen scenario on the VLE. This method will be of the greatest<br />
interest if it is focused on "simple tasks" for which the series of actions required to carry them out is<br />
easy to describe. For example, evaluating the ease with which a VLE user-administrator will choose<br />
the actions to be carried out is perfectly possible using the cognitive walkthrough method. These<br />
principles are fairly general and consequently they can be used when evaluating different types of<br />
computer systems, including VLEs. Similar to a cognitive walkthrough, a heuristic evaluation can be<br />
applied to VLEs, as the principles, such as flexibility and coherency, which are at the root of<br />
heuristics, are fairly general. However, for this evaluation to be relevant, it must be adapted to these<br />
specific environments.<br />
We can therefore acknowledge that all the methods presented in the preceding paragraphs are<br />
potentially valid to measure the usability of VLEs and provide, in certain conditions, information on<br />
their practical utility. Nevertheless, as a VLE must make it possible to carry out various tasks that are<br />
directly or indirectly related to the complex process of <strong>learning</strong>, it is necessary to ponder over specific<br />
criteria referring to the teaching uses of VLE.<br />
A VLE, as we have already mentioned above, should be seen as a social space where the learner<br />
can be an active participant in <strong>learning</strong>. It is thus necessary that the evaluation is comprehensive<br />
relating at the same time practical acceptability (utility and usability) and the social acceptability of a<br />
VLE. These two dimensions (practical and social) are not independent, they are articulated and<br />
complement each other. In view of this complexity posed by the evaluation of the VLEs, other<br />
theoretical approaches are worth questioning: this is the case with activity theory.<br />
4. Origins and nature of activity theory<br />
Activity theory comes from the writings of Leontiev, a disciple of Vygotsky in the 1930s. It finds its<br />
roots in the theory of social development, according to which social interactions play a considerable<br />
part in the development of cognition, and it suggests that the social dimension is at the heart of<br />
human activity. Engeström (Engeström 1987) completed the triad (subject, tool, object) initially worked<br />
out by Vygotsky, adding the element “community” and two mediating elements: rules and the division<br />
of labor. The activity is observed thereby like a system with its own structure, its own internal<br />
transformations, and its own development. The diagram below shows a system of human activities<br />
according to Engeström.<br />
The subject-community-object triangle represents a process connecting the object to a community of<br />
work which plays a part, with the subject, in producing or transforming the object. The relationship<br />
between the subject and the object is mediated by the tool; between the community and the subject is<br />
mediated by explicit or tacit rules; and finally, the relationship between the community and the object<br />
51
Nabil Ben Abdallah and Françoise Poyet<br />
itself are mediated by a certain form of division of labor. The whole system provides a set of dynamic<br />
relations which adjust themselves and adapt continuously to maintain a certain productive stability<br />
(transformation of the object into outcome).<br />
Figure 1: Structure of a system of human activities (Engeström1987)<br />
The principles commonly described of the activity theory are: object orientation, mediation, the<br />
hierarchical structure of activity, internalization/externalization, and development. According to the<br />
principle object orientation of the activity theory, all the human activities are directed towards objects.<br />
They make it possible to transform an object considered as "a raw material'' to a significant object<br />
which is potentially shared by various system activities. These interactions with the object are,<br />
according to the principle of mediation, inevitably mediated by material or intellectual tools. These<br />
tools are able to adapt to environments composed of other tools and / or other human actors. The<br />
principle of the hierarchical structure of activity is one of the principal contributions of Leontiev<br />
(Leontiev 1981), it supposes the existence of three hierarchical levels of activities to carry out a given<br />
task: activities in close relation with the motivations and intentions, actions subordinate and related to<br />
specific goals, and operations determined by the real conditions of the activity. For Leontiev the<br />
human activity exists only in the form of an action or a chain of actions. The principle of<br />
internalization/externalization informs us about our mental processes which are the consequences of<br />
our internalized external activities. These processes are connected to their social and cultural<br />
environment. Lastly, the principle of development requires that the human activity is analyzed in the<br />
context of development. It is a question of understanding how an activity develops, in the course of<br />
time, in an historical and cultural context, and how the actions undertaken on the object of the activity<br />
affect such a development. According to Kaptelinin and Bonnie (Kaptelinin 2006, p.72): "principles of<br />
activity theory should be considered as an integrated system because they are associated with<br />
various aspects of the whole activity. That is, systematic application of any of the principles makes it<br />
eventually necessary to engage all the others".<br />
The activity theory is currently used in various studies related directly or indirectly to human activity. In<br />
the field, for instance, the human-computer interaction, activity theory is used as a relevant theoretical<br />
framework for the design and analysis of computer systems by focusing on the objective, constraints<br />
and specificities of the human activity (Engeström 2010; Larry 2009; Kaptelinin 2006; Mwanza 2002;<br />
Kuutti 1996; Nardi 1996).<br />
From the definition of an activity system and principles of activity theory described above, we try to<br />
sketch the outlines of an approach that allows us to conduct a comprehensive evaluation connecting<br />
both practical acceptability (utility and usability) is the social acceptability of a VLE.<br />
5. Towards a comprehensive evaluation of VLE<br />
In the activity theory, the principle of mediation is a key principle to understand the system of activities<br />
in its globality. It informs us about the mediating role of the Tool that essentially mediates subject-<br />
52
Nabil Ben Abdallah and Françoise Poyet<br />
object interactions, as well as community-object interactions. According to the nature of the analyzed<br />
activity, the two other mediators of the activities system (rules and division of work) can influence the<br />
mediation around of the activity tool.<br />
To analyze and understand the role of these mediators in the achievement of an activity, we initially<br />
decompose the global activity system in subsystems represented by triads consisting of three<br />
elements including a mediator. Each subsystem is composed of the following elements: the subject or<br />
community, the object and a mediating element (tool or rules or division of labor). There are therefore<br />
six triads: (subject, tool, object), (subject, rules, object), (subject, division of labor, object),<br />
(community, division of labor, object), (community, tool, object) and (community, rules, object).<br />
The theoretical framework of activity theory (primarily its concepts and its principles) makes it possible<br />
to identify various evaluation questions which an appraiser must ask to determine the various factors<br />
which can influence the global acceptability of the VLE. This is essentially the contextual factors of the<br />
interaction with the VLE whose determination helps to identify and to understand social and practical<br />
acceptability. Here we are suggesting a few series of questions which can be formulated around each<br />
the activity system triads. The data analysis collected from these questions must take into account the<br />
global nature of the activity, and the fact that the application of a given principle inevitably triggers the<br />
application of the other principles. For example, analysis of the impact of the VLE on <strong>learning</strong> must<br />
take into account the hierarchical structure of activity, object-oriented nature of the activity, the<br />
acquisition of knowledge through the process internalization / externalization, and development of<br />
system activities.<br />
Table 1: Examples of questions<br />
(subject, tool, object)<br />
(subject, rules, object)<br />
(subject, division of<br />
labour, object),<br />
(community, tool, object)<br />
(community, rules,<br />
object)<br />
(community, division of<br />
labour, object)<br />
Aspects of individual activity<br />
- Did you find the VLE useful to perform the work?<br />
- Did you meet with obstacles during the use of the VLE?<br />
- Did the rules of the work organization pose problems to you?<br />
- is the use of the VLE compatible with the rules of the work organization?<br />
- Did you agree with the allocation (whether agreed or implicit) of tasks?<br />
- Does the division of labour imposed by the use of the VLE facilitate the<br />
performance of work?<br />
Aspects of collective activity<br />
- Do you wish to use a VLE to complete a collective work?<br />
- Are the functionalities of the VLE related to collective work easy to use?<br />
- Did the rules of the work organization pose problems to you?<br />
- Is the use of the VLE compatible with the rules of the work organization?<br />
- Was there a collective definition of the different roles in the team?<br />
- Was there was an evolution of roles of the team members according to operations<br />
carried out (query writing, information filtering,) and the intermediate results?<br />
- Does the division of labour imposed by the use of the VLE facilitate the<br />
performance of work?<br />
To demonstrate this process, we would like to provide an example. In this example, groups of<br />
students carry out information searches on a given topic. Each group must produce a report gathering<br />
and synthesizing retrieved information. The Representation of the activity according to Engeström<br />
model enables us to understand the work of a student (subject) engaged in the production of a report<br />
(object) in collaboration with the members of its group (community). The relation which links a student<br />
with a group is determined by explicit rules (duration of work, mandatory work participation, collective<br />
presentation of work, concurrent teams, etc) imposed by the <strong>learning</strong> device which has been set up.<br />
The mediation between the subject and the object is carried out, among other things, by tools offered<br />
by a VLE. The division of work is based on an internal organization for each group, the lecturers do<br />
not operate in the allocation of tasks among group members.<br />
Among other things, collected data show us that the students consider the VLE as a communications<br />
tool, as well as a production of intermediate documents tool, and especially as an information sharing<br />
tool among the members of the same group. The tested functionalities appeared useful to the<br />
53
Nabil Ben Abdallah and Françoise Poyet<br />
students, they are almost all of them have agreed on the VLE utility for a collective work: less space<br />
and time constraints ; a better organization of collective work; facilitates the interactions between the<br />
group members and it supports, consequently, the "collaborative work". The sharing of documents<br />
favors the reactivity of the members and stimulates their production. In a nutshell, we have only<br />
favorable opinions as regards of the VLE in this kind of exercise.<br />
The data of this example consolidates our idea of evaluating the VLE compared to the <strong>learning</strong><br />
activities (context of use), while into taking account the evaluation data which can be provided by HCI<br />
methods. Thus, in our example of evaluation, if the students are satisfied with the VLE they use does<br />
not mean that it is powerful and that it meets the various expectations that this kind of activity<br />
generates. If we consider the results of our analysis, we are quite convinced that the functionalities<br />
suggested by the VLE are not completely adapted to the nature of the exercise in question. VLE<br />
would gain social acceptability (way in which the users perceive the various stakes related to the use<br />
of a VLE) and also practical acceptability if the constraints and contradictions between the various<br />
components of the activity system revealed by the analysis are best supported by the VLE.<br />
6. Conclusion<br />
The question of evaluating VLEs is a huge one and is far from being closed. The attempt of this<br />
contribution is a comprehensive approach in order to shed light on this question. Initially, we tried to<br />
understand what differentiates a VLE from other computer systems. It is at the same time a technical<br />
solution integrating heterogeneous technologies, and an information space around which a social<br />
space which organizes and structures the various interactions necessary to any <strong>learning</strong> activity can<br />
be formed and developed. The different facets that can have a VLE make its evaluation difficult. This<br />
evaluation must relate both the VLE product (interface and functionalities) and the <strong>learning</strong> it<br />
supports. We have acknowledged that several evaluation methods of computer systems are<br />
potentially valid to measure the usability of a VLE and provide, in certain conditions, information on<br />
their practical utility. However, as a VLE should allow to perform various tasks related directly or<br />
indirectly with the complex processes of <strong>learning</strong>, it is necessary for the evaluation to be<br />
comprehensive regarding both the practical acceptability (utility and usability) and the social<br />
acceptability of VLE. These two dimensions (practical and social) are not independent; they are<br />
articulated and complement each other. Facing this complexity posed by the evaluation of a VLE, we<br />
have attempted to explore the theoretical framework proposed by the activity theory to reflect on an<br />
approach which enables us to carry a comprehensive evaluation regarding both the practical and<br />
social acceptability of a VLE.<br />
If the idea that the measure of practical acceptability of a VLE must be completed by the measure of<br />
its social acceptability is sufficiently argued in this contribution, the data processing which can be<br />
collected according to our evaluation approach requires to be explained more thoroughly; it is what<br />
we will strive to do in the coming days.<br />
References<br />
Bastien, J. M. C. & Scapin, D. L., (2001). Évaluation des systèmes d'information et Critères Ergonomiques. In C.<br />
Kolski (Ed.), Systèmes d'information et interactions homme-machine. Environnements évolués et évaluation<br />
de l'IHM. Paris, Hermès, vol. 2, 53-79.<br />
Britain, S. & Liber, O. (2004). A Framework for the Pedagogical Evaluation of eLearning Environment, JISCcommissioned<br />
report. http://www.jisc.ac.uk/index.cfm?name=e<strong>learning</strong>_pedagogy<br />
Colace, F., De Santo, M. & Pietrosanto, A. (2006). Evaluation Models for ELearning Platform: an AHP approach.<br />
36th ASEE/IEEE Frontiers in Education Conference, San Diego, CA. http://fieconference.org/fie2006/papers/1168.pdf<br />
Dillenbourg, P., Schneider, D. K., & Synteta, P. (2002). Virtual Learning Environments. In A. Dimitracopoulou<br />
(Ed.), 3rd Hellenic Conference "Information & Communication Technologie in Education", 3-18. Greece:<br />
Kastaniotis Editions.<br />
Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (1998). Human-Computer Interaction. London, Prentice Hall Europe.<br />
Dobson, M., McCracken, J., & Hunter, W. (2001). Evaluating Technology-Supported Teaching Learning: A<br />
Catalyst to Organizational Change. Interactive Learning Environments, vol.9 n° 2. 143-170.<br />
Eason, K. D. (1984). Towards the Experimental Study of Usability. Behaviour and Information Technology, vol. 3,<br />
n° 2, 133-143.<br />
Engeström, Y. (1987). Learning by Expanding : An Activity-Theoretical Approach to Developmental Research.<br />
Helsinki, Finland, Orienta-Kosultit Oy.<br />
Engeström, Y. (2001). Expansive Learning at Work: toward an activity theoretical reconceptualisation. Journal of<br />
Education and Work, vol. 14, n° 1, 133 – 156.<br />
54
Nabil Ben Abdallah and Françoise Poyet<br />
Engeström Y. et al. (2010). Grand challenges for future HCI research. In Proceedings of the 6th Nordic<br />
Conference on Human-Computer Interaction: Extending Boundaries. Reykjavik, Iceland.<br />
Falconer, L. (2008). Evaluating the use of metaphor in online <strong>learning</strong> environments. Interactive Learning<br />
Environments, vol. 16, n° 2, 117-129.<br />
Hollingsed, T. & Novick, D. (2007). Usability Inspection Methods after Fifteen Years of Research and Practice.<br />
Proceedings of the 25th annual ACM international conference on Design of communication. El Paso, Texas,<br />
USA, 249-255.<br />
Hvannberg, Ebba Thora, Effie Lai-Chong Law, & Marta Kristín Lárusdóttir. (2007). "Heuristic evaluation:<br />
Comparing ways of finding and reporting usability problems". Interacting with Computers 19(2):225-240.<br />
Huart, J., Kolski, C., & Sagar, M. (2004). Evaluation of multimedia applications using inspection methods: the<br />
Cognitive Walkthrough case. Interacting with Computers, vol.16, n°2, 183-215.<br />
Kaptelinin, V., & Nardi, B. A. (2006). Acting with technology: Activity Theory And Interaction Design. Cambridge:<br />
MIT Press.<br />
Keller, C. (2005). Virtual <strong>learning</strong> environments: three implementation perspectives. Learning, Media and<br />
Technology, 30(3), 299.<br />
Kuutti, K. (1996). Activity Theory as a Potential Framework for Human-Computer Interaction Research. In Nardi<br />
B. (Ed.): Context and Consciousness: Activity Theory and Human Computer Interaction. MIT Press, 17-44.<br />
Larry, L. Constantine (2009). Human Activity Modeling: Toward A Pragmatic Integration of Activity Theory and<br />
Usage-Centered Design. In Seffah A., Vanderdonckt J., and Desmarais M. C. (Ed.), Human-centered<br />
software engineering: software engineering models, patterns and architectures for HCI. Springer London.<br />
27-51.<br />
Leontiev, A. (1981). Problems of the Development of the Mind. Moscow, Progress.<br />
Lewis, C. and al. (1992). Cognitive walkthroughs: a method for theory-based evaluation of user interfaces.<br />
International Journal of ManMachine Studies, Vol 36, no. 5, 741-773.<br />
Mahatody T., Sagar M., and Christophe Kolski (2010). State of the Art on the Cognitive Walkthrough Method, Its<br />
Variants and Evolutions. International Journal of Human-Computer Interaction 26, no. 8 : 741-785.<br />
http://www.univ-valenciennes.fr/LAMIH-intra/site/specifique/publications/204316.pdf<br />
Mwanza, D. (2002). Towards an activity-oriented design method for HCI research and practice. PhD thesis, The<br />
Open University, United Kingdom.<br />
Nardi, B. A. (1996). Some Reflections on the Application of Activity Theory . In Nardi B. A. (Ed.), Context and<br />
Consciousness: Activity theory and human-computer interaction. MIT Press, 235-246.<br />
Nielsen, J. (1994). Usability Engineering. San Francisco, Morgan Kaufmann.<br />
Perriault J. (2008). La logique de l’usage: essai sur les machines à communiquer. Paris, L’Harmattan.<br />
Rosson, M. B. and Carroll, J. M. (2001). Usability Engineering: Scenario-Based Development of Human<br />
Computer Interaction". Morgan Kaufmann.<br />
Shneiderman, Ben and Plaisant, Catherine (2005). Designing the User Interface: Strategies for Effective Human-<br />
Computer Interaction" (4th ed.). Addison-Wesley<br />
Weller, Martin. (2007). Virtual Learning Environments: Using, Choosing and Developong Your VLE. Routledge.<br />
55
Designing A New Curriculum: Finding The Right Blend<br />
Andrea Benn<br />
Brighton Business School, University of Brighton, UK<br />
a.benn@brighton.ac.uk<br />
Abstract: This paper discusses the development process for a new Undergraduate (UG) Business Programme<br />
to be introduced at the University of Brighton (UoB) for the academic year 2012/13. It is the research and<br />
preparation which forms the basis of this paper. The focus for this paper was about finding the right balance of e<strong>learning</strong><br />
to support learners during their UG experience, with a view to progressing them from passive to active<br />
and independent learners. The principal pedagogical strategy is to be collaborative enquiry-based <strong>learning</strong> and<br />
this research was to identify appropriate blended strategies and technologies which would enhance and<br />
complement this approach. The concept of collaborative <strong>learning</strong> indicates that consideration must be given to<br />
the formation and development of groups for working in both physical and virtual environments. Both will bring<br />
their own challenges but consideration is given to how these will be addressed throughout the lifetime of the<br />
programme. The plethora of research into E and blended <strong>learning</strong> advises that <strong>learning</strong> with technology needs<br />
to be driven by pedagogical principles, including the specific skills that these new learners will require in order to<br />
cope with and enjoy the new approach of enquiry-based <strong>learning</strong>, not normally associated with UG business<br />
students. The development required a review of the current technologies available at UoB, alternative<br />
technologies and how they are being utilised and incorporated by colleagues in other institutions, together with<br />
ways of introducing them to the learners. The research also explored whether or not there should be an<br />
overlapping of educational and recreational technologies and is it right to mix. Final attention was given to the<br />
assessment of the students, how frequently this should occur and how much should be formative or summative,<br />
what format it would take and how much of it would be the responsibility of the tutors. Some of the technologies<br />
and collaborative <strong>learning</strong> were already integrated into the current business programme at UoB and provided<br />
useful evaluation from observations of the students together with their feedback. This provided some useful<br />
understanding of what their expectations were as well as identifying specific requirements prior to starting any<br />
activities.<br />
Keywords: blended; collaborative; problem-based <strong>learning</strong>; curriculum design, action <strong>learning</strong><br />
1. Introduction<br />
The recent Higher Education Funding Review, which will see an increase in tuition fees for the next<br />
academic year, as well as a timely review of the undergraduate (UG) programmes within the Business<br />
School (BBS) at UoB provided the opportunity to explore alternative <strong>learning</strong> and teaching practices.<br />
The focus of this paper is a new UG business course based entirely on an enquiry-based <strong>learning</strong><br />
(EBL) model.<br />
As described in the Abstract, the research and preparation for designing a new curriculum must not<br />
be underestimated, particularly when the idea is to radically redesign the whole curriculum and the<br />
pedagogical strategy underpinning it. This became very apparent at the start of the research into<br />
what enquiry-based <strong>learning</strong> actually meant. The literature refers to problem-based <strong>learning</strong> (PBL),<br />
action <strong>learning</strong> and work-based <strong>learning</strong> rather than enquiry-based <strong>learning</strong> but all offer similar<br />
definitions which would apply equally well to an enquiry-based strategy: all agree that they are models<br />
which merge theory with practice and knowledge with experience.<br />
The EBL model will require the students to work in groups to solve a problem by managing their own<br />
<strong>learning</strong> process and deciding what information they will need as well as the skills they will have to<br />
develop, ascertained from earlier research, (Benn, 2011).<br />
However, Duch et al (2001) advise that PBL is not generally available to many undergraduate (UG)<br />
courses for a variety of reasons among them being class sizes; intellectual maturity; motivation levels;<br />
and epistemological beliefs.<br />
These are entirely valid reasons and were considered continually during the design process, including<br />
the reason to continue with the research in this area. The preparation for redesigning the curriculum<br />
came also from a desire to offer the students an alternative <strong>learning</strong> experience: one which would<br />
challenge them, encourage and instil confidence in them to become active learners; illustrate to them<br />
that more often than not there is no one right answer, and to provide the support; guidance; resources<br />
and feedback opportunities at times when they required it.<br />
56
Andrea Benn<br />
It was also clear that this was going to be as much a challenge for the lecturers as it was hoped it<br />
would be for the students.<br />
Research Questions<br />
The main objectives for this research were:<br />
To identify the specific skills new learners would require in order to cope with and enjoy, enquiry<br />
based <strong>learning</strong> (EBL)<br />
To identify appropriate blended strategies and technologies which would complement EBL within<br />
a business programme<br />
To identify ways to introduce EBL to colleagues, students and the local business community<br />
2. Research Methods<br />
The primary method for this report was desk research reviewing the available literature for problembased;<br />
work-based and action <strong>learning</strong> and in particular the findings of some of the experts in this<br />
field as Barrows & Tamblyn (1980); Savin-Baden (2006), Boud & Feletti (2001) and Duch et al (2001).<br />
More recent journal articles discuss the experiences of implementing PBL but only one example was<br />
found for an UG programme in the UK, Parrott (2009)<br />
Students studying on the existing course at UoB, the FdA Business, were asked for feedback on<br />
completion of a fully online module ascertaining what they liked or disliked most about the activities;<br />
the time spent on each activity and whether they considered they had sufficient IT skills to cope with<br />
the module or how these had developed during the module. They were also asked how they would<br />
feel about using social networking sites ie Facebook for University work and interacting with lecturers.<br />
Results from earlier action research by the author, Moon (2010), which looked at the development of<br />
her practice and skills in order to bring about change in the module delivery to encourage the students<br />
to rise to their new challenge, were also incorporated into this report.<br />
2.1 The Pedagogical Strategies<br />
The concept of EBL was determined as the preferred principal strategy as it requires the programme<br />
to be student-centred, therefore the curriculum design must also have a student-led approach,<br />
allowing students to “take care of their needs... particularly pushing them into reflecting on a certain<br />
number of aspects they are already acquainted with but which they tend to forget when working on<br />
their own”, Brudermann (2010).<br />
The underpinning skills required can be identified as being those suggested by Bloom (1956) as<br />
knowledge; comprehension; application; analysis; synthesis and evaluation as well as collaborative<br />
group working and co-operative <strong>learning</strong>. Developments of these skills is already expected of<br />
graduates upon completion of each stage of existing UG programmes, and are now translated to the<br />
QAA level descriptors for UG programmes. What differs is the expectation that students from the<br />
outset of Level 4, will take control of their <strong>learning</strong> for themselves, ie they adopt an active approach,<br />
Savin-Baden & Wilkie (2006) and put these skills to use immediately. It is recognised that for Level 4<br />
UG this may be a tall ask and the design of the first year needs to be more formative, demonstrating<br />
to the students what is expected of them.<br />
The next stage is to understand how, through the design of the curriculum, the students will<br />
recognise these skills and map their development over the course of their study and whether using<br />
blended strategies and incorporating technology would complement or complicate and how far should<br />
it go.<br />
Earlier research emphasised that the first consideration needs to be for the technology to support and<br />
be driven by the pedagogical principles. From a business perspective, Laudon & Traver (2009, 4-4)<br />
advise that business decisions drive the technology and not the other way around, to ensure that its<br />
technology platform will be aligned with the business and not the other way around. This view is<br />
reiterated by Cox et al (2010) in an educational setting, which is to find the best technology that suits<br />
the subject being offered, the context in which it is offered and which can be utilised for the greatest<br />
advantage for that purpose.<br />
57
Andrea Benn<br />
For this new course, it is proposed that a blended-PBL framework (Savin-Baden, 2006) be considered<br />
as it will combine “an approach for building an online <strong>learning</strong> community” (Yeh, 2010) alongside<br />
traditional group-working enabling students to develop their own working boundaries and task<br />
allocation in both environments. However, blending the curriculum needs to be planned within the<br />
Course Team just as the overall assessment strategy is planned to ensure that the students meet with<br />
a variety of <strong>learning</strong>, teaching and assessment methods and not just one or two. The blend therefore<br />
will come from discussions with colleagues also acknowledging their own areas of expertise and<br />
preferences, as well as the technology available within our own institution at this time.<br />
It is acknowledged too that the students will have different levels of expertise and preferences for how<br />
they wish to participate in the <strong>learning</strong> activities. The current Level 5 students this year who had<br />
participated in a fully online module, fed back via a questionnaire for this report that they liked the new<br />
way of <strong>learning</strong>; the chance to voice their own opinion; being able to compare and discuss and to<br />
share and receive ideas as well as peer support. What they did not like was the volume of posts;<br />
what they considered to be a complicated discussion board; the feeling that they either should not<br />
post too much information as it would not give the others a chance or finding that too much<br />
information had already been posted and for some the dislike of having to get on and do it on their<br />
own with what they perceived to be little support.<br />
With respect to their levels of expertise they advised that the majority of them had some but limited IT<br />
skills, they could navigate their way around the Internet and could use basic office applications but<br />
only half of the group agreed that their IT skills had developed in some way.<br />
Both the L4 and L5 students were asked how they felt about using social networking sites (SNS) ie<br />
Facebook for University work and the results were totally opposite. Not one reply received from the<br />
L4 cohort was in favour of using SNS in this way, yet all of the L5 cohort thought it was a good idea as<br />
most students used it and would check it regularly, a couple were surprised that it was not in use<br />
already.<br />
Previous research and observations from conferences indicates that there is also a conflict of opinion<br />
among academics as to whether the use of SNS for teaching or interacting with students is productive<br />
or whether it is just encroaching “on their space” (Smyth, 2010).<br />
2.2 The Challenges<br />
There are a number identified at this time, not only in relation to the students and how they will<br />
manage or cope with this strategy but also in relation to colleagues who will also be required to<br />
undergo a process of change.<br />
For colleagues it will be about participating in action <strong>learning</strong> which requires us as a set (team) to<br />
collaboratively work on a real issue: the development of the new curriculum; understand a new<br />
strategy: EBL; while reflecting on what we as individuals have previously experienced and can share<br />
with this set, in order to drive this initiative forward (McGill & Brockbank, 2006).<br />
For the Faculty (BBS) it will be to offer support maybe from colleagues already familiar with this<br />
strategy or with time for research and maybe changes to the <strong>learning</strong> environment.<br />
A specific challenge for the Course Team will be the accrual, selection and design of appropriate<br />
problems and materials to enable the students to achieve a more in-depth knowledge of the concepts<br />
and principles of the course, while bringing about a change in students from passive to active<br />
learners.<br />
Otting et al (2010) reviewed the relationship between students’ epistemological beliefs and<br />
conceptions of teaching and <strong>learning</strong> and found that students do recognize the importance of selfdirected<br />
<strong>learning</strong>, but observations of our current students found that they are constantly seeking<br />
reassurance that they are “doing it right” because they expect there to be a prescriptive criteria for<br />
getting good marks; as well as an unwillingness to do more than is required on the assessment brief.<br />
Earlier statements in this report refer to the levels of motivation and intellectual maturity but the<br />
Course Team need to reduce the level of dependency and instead find ways to encourage, instil<br />
confidence in them to try it for themselves to see what happens and thereby persuade the students to<br />
understand and accept the concept that there is no one right answer.<br />
58
Andrea Benn<br />
Duch et al (2001) suggests that it may be the way that we teach that reinforces a view of <strong>learning</strong><br />
which is that the teacher is responsible for delivering content and the students are the passive<br />
receivers of knowledge. Duch continues to argue that the way we test, the expectations we set and<br />
the <strong>learning</strong> materials we use may also contribute to the level of student motivation and intellectual<br />
maturity.<br />
By making the assessment brief so prescriptive are we restricting the students’ natural inquisitiveness<br />
as well as telling them what we expect the answer to be? These views are seen as a very good<br />
reason to radically review and change what we do in order to bring about the change we are looking<br />
for/expecting from UG students.<br />
2.3 The Resources<br />
Continuing with the literature review, Duch et al (2001) offers some practical advice which has greatly<br />
influenced the design suggestions for writing the problems which will be forwarded to the Course<br />
Team for discussion:<br />
The first question to ask in this situation is: what should the students know; value and be able to do by<br />
the end of the course? Followed by what evidence will indicate that they have reached these goals.<br />
The advice continues that the problems need to be written as open-ended, real-world problems which<br />
will require a certain amount of creativity. At this point, links with the local business community can be<br />
created as professionals in the field could be consulted for ideas of a more realistic nature for<br />
application of the concepts of the course. Alternatively, news articles, journals, current events can be<br />
used and adapted for the purpose.<br />
The problem also needs to provide opportunities for the students to be able to demonstrate the<br />
underpinning skills previously identified. Duch et al (2001) identify five characteristics considered to<br />
be important for a good PBL problem but recognises also that they may vary between disciplines.<br />
Those most appropriate to this course are:<br />
It must engage students’ interest and motivate them to probe for deeper understanding of the<br />
concepts introduced;<br />
Ideally requires the students to make decisions or judgements based on facts, information, logic<br />
and/or rationalization;<br />
It should be complex enough to necessitate cooperation from all members of the group to<br />
effectively work toward a solution;<br />
The initial questions should be open-ended based on previously learned knowledge and/or be<br />
controversial to draw all students into a discussion;<br />
The content objectives need to be incorporated into the problems, connecting previous knowledge<br />
to new concepts.<br />
The intention at BBS is for the whole course to be delivered as EBL therefore it could provide more<br />
time for the students to adapt to this process as they progress. It is of course the freshers’ year that<br />
will require more support and guidance, therefore Duch et al (2001) suggest that the problems be<br />
delivered in stages, allowing the students to cooperate, to make assumptions based on the<br />
information they have and decide what more they need to request. On the problem brief the <strong>learning</strong><br />
objectives stated should include those beyond the content objectives and should be more complex<br />
and involve process skills objectives too. Eventually, as they become more familiar with this process<br />
and their knowledge and experience grows, this could also help the students to identify the direction<br />
they need to work towards.<br />
Linking this again to QAA level descriptors, the authors need to ensure that there is sufficient<br />
distinction between the stages particularly as students progress from level 4 to level 5 and level 6.<br />
Reviewing how technology can assist with this process, Savin-Badin and Wilkie (2006) explore the<br />
concept of a fully online PBL strategy stating that it is an approach to <strong>learning</strong> that is both varied and<br />
flexible. A fully online strategy is not the main objective here; but to find a way that technology can<br />
support the principal strategy of EBL, their findings provide a good basis for the decisions for this<br />
purpose.<br />
59
Andrea Benn<br />
Technology is already integrated into the existing course to varying degrees but is entirely subject to<br />
the module tutor’s experience, innovation and aspiration to try new approaches. The technology<br />
needs to complement the principle for the new course to be student-centred. The students will work<br />
in teams and will have to work collaboratively to achieve their shared goal. Communication will<br />
therefore form a major part of this collaboration but may not necessarily take place face to face.<br />
Students too, will be required to leave the University premises for periods of time, and if not timetabled<br />
to be together may find that their commitments to the activity in terms of time and hours spent<br />
may not coincide with everybody else. Using the technology as an asynchronous tool may assist the<br />
students here but does this tool need to be supplied by the University? It begs the question do we<br />
want to or need to see the <strong>learning</strong> process exactly as it unfolds or are we happy to see the end result<br />
in whatever format that is determined via the course assessment strategy? The issue of whether to<br />
join the students in the environment of their social network sites need not arise, particularly if we are<br />
timetabled to meet with the students on a face to face basis during the scheduled tutorial sessions<br />
which would be required as part of the EBL strategy. It also puts the responsibility for this part of the<br />
<strong>learning</strong> process onto the students, and in particular forming the virtual and actual group dynamics,<br />
resolving conflicts and division of labour, just as they would in a social environment and without the<br />
interference from the tutors.<br />
It is here that the issue of assessment and in particular formative assessment may arise. The<br />
students will be required to reflect as part of this <strong>learning</strong> strategy and one way this can be achieved<br />
through IT is via the ‘journal’ facility currently available in the UoB’s virtual <strong>learning</strong> environment<br />
(VLE). It can offer the students the opportunity to evaluate their own <strong>learning</strong> and because the tool<br />
can be restricted to collaboration between one individual student and the tutor, it can offer 1-1<br />
feedback which would be entirely relevant to each student and timely at the point that it is required.<br />
Either online or with pre-arranged 1-1 tutorials the student can discuss specific points of concern. At<br />
a recent UoB symposium relating to feedback and assessments, students advised that they needed<br />
instant feedback and that waiting for 2 weeks or longer is of no use – the moment has gone.<br />
The VLE that UoB has currently invested in is primarily used as more of a repository of information<br />
provided by the tutors. Savin-Baden and Wilkie (2006) warn that some online environments can be<br />
over managed by the tutors but there is scope on the VLE to permit the students to upload information<br />
via the group forums/discussions.<br />
Tuition to demonstrate exactly how this <strong>learning</strong> tool can be used should be offered during the early<br />
stages of the course, and maybe now it is time to look to the Web 2.0 technologies that students will<br />
experience in the field to see how businesses share documents and information and work across time<br />
and geographical zones and prepare for the limited occasions when they will meet face to face. The<br />
inference on this latter point is that the time spent face to face is to be regarded as precious and<br />
should therefore be planned for to ensure that the time is maximised. This is another expectation that<br />
the tutors need to advise students of from the outset.<br />
Some information will still be disseminated by tutors or professional guest speakers via lectures or<br />
electronic methods but the drive for information needs to come from the students, discovered and<br />
shared between them but also allowing for peer feedback, amendments and additions.<br />
2.4 Transition<br />
Before looking at the transition period for the students it is as well to consider the transition period for<br />
the tutors. Savin-Badin & Wilkie (2006) warn that for many there will have to be a recognition of a<br />
loss of power and control because it will be about letting go of decisions about what students should<br />
learn and instead trusting the students to acquire knowledge for themselves and accepting that<br />
students will learn even if they have not been supplied with a lecture or a handout from their tutor.<br />
Given the observations and statements mentioned earlier in this report, the transition period needs to<br />
be carefully planned and supported allowing colleagues time to discuss and share ideas and to reflect<br />
and evaluate.<br />
Discussions with colleagues over the years have found that for many of us, the best teaching<br />
experiences are those when we are presenting information for the first time, ie at the cutting edge of<br />
our knowledge. Perhaps this is because we are yet to become comfortable with the new information<br />
and so we take more time in preparing how we disseminate it. So, this may also be true with this new<br />
60
Andrea Benn<br />
pedagogical strategy, in order to fully appreciate how we want the students to react so we have to<br />
experience the same process for ourselves, to understand the concerns, fears and high points that<br />
the students will go through.<br />
The literature review finds that careful planning and consideration of how the problem is introduced to<br />
the students will enable them to identify the <strong>learning</strong> issues and therefore which theory concepts need<br />
to be researched. As a Team the planning needs to include the structure of the problem, how many<br />
stages it should be broken down into, over how long a period of time it will take to complete, the<br />
resources that need to be made available as and when the students request them and what the end<br />
product is going to look like before moving onto discuss the format of the tutorial sessions with the<br />
students to facilitate discussion and progression of the task. For this a suggested model as an initial<br />
guide has been provided by a UoB colleague based on an example illustrated by Barrows & Tamblyn,<br />
1980 and is shown below:<br />
Table 1: Facilitating Initial Discussions:<br />
1<br />
INFORMATION<br />
What facts do we have?<br />
What do we already know?<br />
What is our experience of this situation?<br />
Move to: what do we still need to find out?<br />
3<br />
QUESTIONS<br />
What do we need to learn/understand before we<br />
can deal with/improve the situation?<br />
2<br />
IDEAS/HYPOTHESES<br />
Define the situation/the “problem” strengths,<br />
negatives<br />
Move to: how to manage the situation in light of<br />
the present understanding?<br />
4<br />
ACTION PLAN<br />
How/where will we learn these things?<br />
What resources will we need/use?<br />
This model can be adapted to support the task within our discipline and help to create an environment<br />
where discussion and brain storming can be promoted and deeper exploration of the topic developed.<br />
Turning to the student experience, Parrott (2009) warns that the first year at UG level represents a<br />
major challenge for students, acknowledging that students can benefit from greater levels of support<br />
during the early stages of the course. It is essential to offer the students a carefully planned<br />
transition period. The Course Team will need to consider what our expectations of the students will<br />
be for this new strategy which will have to be explained, demonstrated, reflected on and reiterated<br />
through an active transition period. The expectations will also include ‘course rules’ explicitly informing<br />
students of their tutor’s availability, whether online or face-to-face and the preparation required from<br />
them for all tutorials.<br />
The ideas already under discussion for the new course include students from the higher levels<br />
offering peer support to the newer students through mentor schemes already in operation with UoB<br />
but also during the course of their academic year by presenting research that they have undertaken.<br />
Demonstration of the IT tools that will be used through the course should be provided at this stage ie<br />
any discussion or group forums and in particular the journal facility where the concept of reflection<br />
should be introduced.<br />
3. Conclusion<br />
The reason for exploring alternative <strong>learning</strong> and teaching strategies is to bring about a change in the<br />
students in their approach to their <strong>learning</strong> which will last beyond their academic careers. The ultimate<br />
aim is to change the students’ behaviour from being passive to active learners.<br />
The literature review argues that the reason for the way the students currently behave is because of<br />
the way they are taught and the expectations we as tutors are portraying. It would therefore suggest<br />
that the change initially needs to come from us and our approach to our teaching and the resources<br />
we wish to utilise in doing so.<br />
This change however, will not be without some substantial challenges that the teaching team will<br />
encounter not only from the students’ perspective but their own too. It will require the teaching team<br />
to let go of the responsibility for providing the information while relying on the co-operation of the<br />
students to find out for themselves, albeit with guidance and support from the teachers.<br />
61
Andrea Benn<br />
To bring about the co-operation of the students, they are going to have to want to engage with the<br />
programme, be interested in the topic and see some value to them in it. It will also be about<br />
managing the students’ expectations in relation to their new <strong>learning</strong> environment as most will<br />
probably have experienced a more teacher-led approach so time and patience must be invested in<br />
explaining what a student-led framework will look like. Demonstrating the skills to cope with this will be<br />
a priority and a carefully planned transition period will be essential for a group of level four UG<br />
students.<br />
Co-operation and collaboration across the Course Team will be essential to ensure that no <strong>learning</strong><br />
outcomes are left uncovered or are doubled-up particularly as this is now an holistic approach to the<br />
programme. This experience will help the Course Team to reflect on the course progression and to<br />
give the opportunity to anticipate the needs and possible fears of the students.<br />
Finding the right blend is not just about finding the right mix of IT to include but choosing the right IT to<br />
complement the <strong>learning</strong> strategies and support the students in their progression through the course,<br />
particularly through collaboration with each other and with the Course Team. The Course Team will<br />
need to continue to be innovative with their application of any technology to enhance their teaching or<br />
dissemination of information, but care needs to be taken not to over manage the <strong>learning</strong> environment<br />
when implementing IT. Some aspects of the course may be offered fully online other aspects may<br />
provide an opportunity for personal reflection and tutorial feedback. The collaborative elements for<br />
this pedagogical strategy perhaps need to be managed by the students themselves with an invitation<br />
to the tutors to join them if they desire it. This approach may give the control to the students to<br />
develop a group working relationship between them, leaving the time allocated to tutorial or seminars<br />
to focus completely on progressing the task, presenting to or receiving feeding from the tutor.<br />
It will without doubt be a challenge but one that should prove to be exciting and rewarding for all<br />
stakeholders. The timing for the change is also right.<br />
References<br />
Benn, A., (2011) Changes Facing Higher Education – Opportunity or Challenge? Exploring Alternative Learning &<br />
Teaching Practices for Undergraduates ICICTE 2011 Annual Conference, Rhodes, 7-9 July<br />
Boud, D., Feletti, G., (2001) The Challenge of Problem-Based Learning,2 nd Edition, London: Kogan Page<br />
Brudermann, C., (2010) From Action Research to the Implementation of ICT Pedagogical Tools: Taking Into<br />
Account Students’ Needs to Propose Adjusted Online Tutorial Practice, European Association for Computer<br />
Assisted Language Learning, 22(2): 172-190<br />
Cox, M. J., San Diego, J. P., Quinn B., Hindmarsh, J., Dunne S., Newton T., Woolford, M., Reynolds, P., Green,<br />
T-A., Shahriari-Rad, A., Hyland, L., Banerjee A., (2010), University Teachers’ Pedagogical Practices in<br />
Using Technology-Enhanced Learning: strategies to integrate haptic technologies into undergraduate<br />
teaching, p1-6, BERA 2010 Annual Conference, University of Warwick, 1-4 September<br />
Duch, B.J., Groh, S.E., Allen, D.E., (2001) The Power of Problem-Based Learning – A Practical “How To” For<br />
Teaching Undergraduate Courses In Any Discipline, USA: Stylus Publishing LLC<br />
Laudon, K. C. and Traver, C. G. (2009) E-Commerce Business Technology Society 5 th Edition, London: Pearson<br />
McGill, I., Brockbank, A., (2006) The Action Learning Handbook – powerful techniques for education,<br />
professional development & training, Oxon: Routledge Falmer<br />
Moon (nee Benn), A., (2010) Exploration and Experimentation: applying new technologies to <strong>learning</strong> – 2 nd stage<br />
action research, BERA 2010 Annual Conference, University of Warwick, 1-4 September<br />
Otting, H., Zwaal, W., Tempelaar, D. and Gijselaers, W. (2010) The structural relationship between students’<br />
epistemological beliefs and conceptions of teaching and <strong>learning</strong>, Studies in Higher Education, 35(7), 741-<br />
760, First published on 7 October 2010<br />
Parrott, G., (2009) Redesigning the first year business curriculum at the University of Bedfordshire” International<br />
Journal of Management Education 8(2): 13-21<br />
Sadlo, G., Piper, D. W., Agnew, P. (1994), Problem-Based Learning in the Development of an Occupational<br />
Therapy Curriculum, Part 1: The Process of PBL British Journal of Occupational Therapy (0308-0226),<br />
57(2), 49-54<br />
Savin-Baden, M., Wilkie, K., (2006) Problem-based Learning Online, Maidenhead: OU Press<br />
Smyth, K. (2010), Our Space or Theirs? ICT Skills Briefing, Issue 14<br />
Yeh, Yu-Che., (2010) Integrating Collaborative PBL with Blended Learning to Explore Preservice Teachers’<br />
Development of Online Learning Communities, Teaching and Teacher Education, 26: 1630-1640<br />
62
Critical Success Factors for the Adoption of eLearning in<br />
the Kingdom of Saudi Arabia Educational Institutions<br />
Latefa Bin Fryan and Lampros Stergioulas<br />
Department of Information Systems and Computing, Brunel University,<br />
London, UK<br />
Latefa.Fryan@brunel.ac.uk;<br />
Lampros.Stergioulas@brunel.ac.uk<br />
Abstract: In recent years, Electronic Learning (eLearning) has gained interest and has been widely adopted and<br />
acknowledged; it is integrated in many institutions' educational systems. Owing to the major transformation of<br />
traditional educational system in the Kingdom of Saudi Arabia (KSA), most of higher educational institutions are<br />
currently switching to eLearning system and the rest are expected to switch in the near future. Nowadays there is<br />
a universal need to investigate such Critical Success Factors (CSFs) that have significant impact on the adoption<br />
of eLearning. The aim of this paper is to identify and prioritise, in order of their importance, the Critical Success<br />
Factors (CSFs) for the adoption of eLearning in educational institutions of the Kingdom of Saudi Arabia (KSA).<br />
Therefore, the chosen area of research for this paper holds importance in terms of considering the CSFs before<br />
and during the adoption of eLearning process especially at this time when KSA is ready to prepare for this<br />
educational reform. Although the significance of eLearning cannot be denied and its impact on the improvement<br />
of the traditional educational system is well-recognised, there are still gaps reported in the literature for the<br />
adoption of eLearning. In order to fully realise the potential of eLearning and understand the challenges in its<br />
adoption, it is imperative to identify the CSFs that affect the adoption of eLearning in KSA educational institutions.<br />
In order to do so, this paper identifies key success factors that affect the adoption of eLearning through extensive<br />
analyses of case studies and the existing literature. For the case study analysis, five KSA educational institutions<br />
are investigated through interviews and questionnaires; for the purpose of literature review, data is collected from<br />
different resources such as journals, books, conferences papers, newspaper articles, magazines, etc. The<br />
research gap is identified so as to highlight the importance of this research. Additionally, the features and benefits<br />
of eLearning are discussed and the CSFs for eLearning adoption are identified and presented in a table with<br />
respective categorisations. The interpretive, qualitative data analysis methods are used, and findings are presented<br />
in tables and figures. Findings show that of the total 52 CSFs identified, 25% were highlighted only through<br />
interviews and survey questionnaires and were not found in the existing literature.<br />
Keywords: eLearning, adoption of eLearning, distance <strong>learning</strong>, critical success factors, higher educational<br />
institutions, education in Kingdom of Saudi Arabia, educational reform<br />
1. Introduction<br />
Technology has played a crucial role for the enhancement of <strong>learning</strong>. Nowadays, online <strong>learning</strong><br />
programs have significant value in the educational process due to the growing trend of globalization,<br />
technical innovation, increasing competition, and rising student expectations (Rovai and Downey,<br />
2010). Recent developments in Information System (IS) and Information Communication<br />
Technologies (ICT) have led to the facilitation of <strong>learning</strong> and teaching process. eLearning, also<br />
known as online <strong>learning</strong>, refers to the use of ICTs to enhance and/or support <strong>learning</strong> in the<br />
educational process (Al-Jaghoub et al, 2009; NZCER, 2004).<br />
The use of technology in the educational process is recognised globally. Availability of eLearning<br />
provides equal opportunities for learners who may be deprived from such opportunities of <strong>learning</strong> (Al-<br />
Jaghoub et al, 2009); and use of digital technology has positive impact on the <strong>learning</strong> process<br />
(Underwood, 2009). The global demand for seats in higher education will double to 200 million by<br />
2020 which will increase the number of universities providing distance <strong>learning</strong> (Rovai and Downey,<br />
2010); this illustrates a positive trend in the uptake of eLearning. Sun et al. (2008) reported a growth<br />
rate of 35.6% for the eLearning market. Due to increasing competition among various education<br />
providers (specifically higher education), eLearning is being adopted in order to increase their number<br />
of students both nationally and globally (Rovai and Downey, 2010).<br />
There are several benefits of eLearning including location and time independence; students prefer<br />
eLearning as compared to the traditional face-to-face <strong>learning</strong> methods (Kelly et al, 2007). It facilitates<br />
the provision of education to large communities at a very low cost by reducing (or eliminating) the<br />
number of classrooms, staff, or hard copies of teaching materials needed (Papp, 2000). eLearning<br />
supports both individualised and collaborative <strong>learning</strong> (Suddaby and Milne, 2008; Bell, 2007; Volery<br />
and lord,2000), and compensates any deficiencies in the traditional <strong>learning</strong> system (Hsbollah and<br />
Idris, 2009).<br />
63
Latefa Bin Fryan and Lampros Stergioulas<br />
When comparing eLearning with the traditional classroom <strong>learning</strong>, one can find pros and cons for<br />
either case. Zhang et al. (2004) assert that: eLearning offers a learner-centred approach, while the<br />
conventional classroom <strong>learning</strong> is less centred on the tutor and puts more emphasis on the<br />
instructor. Zhang et al. (2004) also demonstrate that the eLearning offers more advantages than the<br />
traditional classroom <strong>learning</strong>, which implies that the new technology is changing the way of <strong>learning</strong><br />
and constitutes a real alternative to conventional classroom <strong>learning</strong>. However, Russell (2001)<br />
indicates that there is no significant difference between eLearning and traditional <strong>learning</strong> and offers<br />
considerable evidence that technology does not depreciate teaching.<br />
Many countries are adopting eLearning for the purpose of improving educational standards,<br />
processes, and to overcome time, space or resource limitations. Abouchedid and Eid (2004)<br />
conducted a survey about the challenges for eLearning adoption in the Arab world and commented on<br />
the need for change in the attitudes of instructors towards the adoption of eLearning, however he also<br />
argues that it is imperative for the Arab world to adopt eLearning in order to compete with the<br />
educational systems of the developed countries of the world. Pursuing this further, this research<br />
focuses on Kingdom of Saudi Arabia (KSA) for investigating and analysing the adoption of eLearning<br />
in their educational system.<br />
2. Background: Concept and adoption of eLearning in KSA educational<br />
institutions<br />
Kingdom of Saudi Arabia (KSA) has population expanding at 3.7% to 4.0% per year (Field, 1994;<br />
Long, 1997), with 16.9 million in early 1900 and expected to reach 25 million by 2011 (Saudi Arabian<br />
Information Resource, 2007). This rapid growth in population is expected to exert a huge pressure on<br />
the country’s educational systems creating increasing demands for technological solutions (Albalawi,<br />
2007). Since majority of KSA’s population is young requiring more jobs and good education (Al-Faisal<br />
et al., 2010). The educational system in KSA is struggling to cope with increasing demands of growing<br />
population and diversifying economy with student to teacher ratio of 15:1, which is one of the lowest in<br />
the world (ameinfo.com, 2009). Since 1998, the number of universities in KSA has increase from 7<br />
public universities to 17 public universities and 24 private colleges, containing almost one million<br />
students (Al-Faisal et al., 2010). In order to enable KSA educational system to cope with the 21 st<br />
Century developments, it is essential to improve the system by using ICT in <strong>learning</strong> and<br />
management and by increasing cooperation with local and international partners (Al-Faisal et al.,<br />
2010). KSA Government is undertaking initiatives for educational system reform to improve quality<br />
and efficiency of <strong>learning</strong> at all levels by allocating SR137 billion ($36.5 billion) for education in<br />
country’s 2010 budget. (Al-Faisal et al., 2010); examples of these reforms are curriculum<br />
development, ICT and eLearning strategy, and partnering with foreign schools. Universities and<br />
colleges in KSA vary in their levels of technology advancements and developments; for instance,<br />
some universities are less developed in terms of using technology or ICT (i.e. limited to a few<br />
numbers of computers which can only be used by faculty members), while others are highly equipped<br />
with technology such as King Abdullah University of Science and Technology (KAUST) which<br />
operated in 2009, with cost of $10B at estimated area of 36 million meters square (KAUST, 2010).<br />
Owing to the rapid population growth, lack in the number and quality of teachers and instructors, and<br />
to adhere with KSA Government’s initiative for the utilization of ICT, the National Centre of eLearning<br />
and Distance Learning (NCEL) was established in 2006, with an aim to facilitate the next generation<br />
eLearning across higher education sectors in KSA and to organise the change and prepare eLearning<br />
material (NCEL, 2010). The goals of NCEL (NCEL, 2010) are: extending the use of eLearning<br />
applications in higher educational institutions; participating in increasing the capacity for institutions by<br />
using eLearning system; improving awareness of the benefits and importance of eLearning; holding<br />
seminars, workshops and conferences; building educational software applications; and supporting<br />
researches in the field of eLearning and distance <strong>learning</strong>. NCEL provides training to various<br />
university staff through workshops. NCEL encourages its trainees to use eLearning system as some<br />
of them are resistant to use it.<br />
Resulting from the major transformation of traditional education in KSA, most educational institutions<br />
are switching to eLearning system; some universities have already started implementation of<br />
eLearning system while others are considering this now.<br />
64
3. Motivation of research - The gap<br />
Latefa Bin Fryan and Lampros Stergioulas<br />
Although the significance of eLearning cannot be denied and its impact on the improvement of the<br />
traditional educational system is well-recognised, there are still gaps in the adoption of eLearning.<br />
Reasons for this include: lack of understanding about the success factors of eLearning, and<br />
neglecting the factors influencing lecturers to adopt the internet as a teaching tool (Hsbollah and Idris,<br />
2009). Existing literature reports that considerable amount of work has been done on the benefits or<br />
features of eLearning and about eLearning implementation, and several studies have been conducted<br />
for the adoption of eLearning listing many CSFs; however, the lists are not comprehensive; for some<br />
CSFs student’s perspectives alone are taken into consideration while the others merely consider the<br />
education provider’s perspectives. Due to the lack of research into this area, further research is<br />
needed for investigating and identifying such CSFs that would lead to a successful adoption of<br />
eLearning. In order to fully realise the potential of eLearning and understand the challenges in the<br />
adoption of eLearning, it is imperative to identify the Critical Success Factors (CSFs) that affect the<br />
adoption of eLearning in KSA educational institutions.<br />
The purpose of the planned research is to address this gap and present a comprehensive list of such<br />
CSFs that influence the adoption of eLearning in KSA educational institutions.<br />
4. eLearning features and benefits<br />
Electronic <strong>learning</strong> (eLearning) can be defined as a way of <strong>learning</strong> that is facilitated and supported<br />
through the use of information and communications technology (NZCER, 2004; Al-Jaghoub et al,<br />
2009). eLearning is a web-based educational system that uses IT and computer networks; it widely<br />
utilises modern technology, internet, electronic media, or web-based applications to deliver the<br />
ultimate <strong>learning</strong> experience (Selim, 2007; Hsbollah and Idris, 2009). It is one of the most significant<br />
recent developments in the information system industry (Wang, 2003) and is considered as an<br />
appropriate means of providing education for universities which lack enough staff, study material, or<br />
classes. With the adoption of eLearning there is no need to minimise the number of student in each<br />
course. eLearning is a major transformation of traditional education provision to more modern,<br />
effective and efficient alternative educational methods (Freire, 1994 Selim, 2007). It has been<br />
reported in the literature that eLearning outranges the traditional <strong>learning</strong> methods in terms of courses<br />
and student performance (Beyth-Marom et al., 2003). Moreover, students prefer to use eLearning<br />
because it makes their <strong>learning</strong> more effective, efficient, and flexible; i.e. they can study anytime,<br />
anywhere, and in their own way (Papp, 2000). eLearning supports both individualised and<br />
collaborative <strong>learning</strong> (Suddaby and Milne, 2008; Bell, 2007; Volery and lord,2000), and compensates<br />
any deficiencies in the traditional <strong>learning</strong> system (Hsbollah and Idris, 2009).<br />
The effectiveness-level increases as we shift from distance <strong>learning</strong> to eLearning (as shown in Figure<br />
1). eLearning has also been used interchangeably or with partly overlapping meanings as computer<br />
mediated <strong>learning</strong>, advanced distributed <strong>learning</strong> (ADL), online <strong>learning</strong> (OL), networked <strong>learning</strong>,<br />
virtual <strong>learning</strong>, web-based <strong>learning</strong> (WBL), internet-based training (IBT), web-based instruction<br />
(WBI), open/flexible <strong>learning</strong> (OFL), and digital <strong>learning</strong> (Khan, 2001; NZCER, 2004; Selim, 2007).<br />
While eLearning can be used as part of distance <strong>learning</strong> programs, it does not necessarily have to<br />
involve distance <strong>learning</strong> (NZCER, 2004).<br />
Several benefits of eLearning have been reported in the literature, the key benefits are locationindependent<br />
and time-independent <strong>learning</strong> (Fayter, 1998; Homan and Macpherson, 2005; Bell,<br />
2007). Using IT in education not only facilitates the access to education and training but also<br />
improves teaching and <strong>learning</strong> quality (Newton, 2003). Through eLearning, the delivery of<br />
educational programs to more students can be realised at a much lower cost (Peled, 2000; Volery<br />
and Lord, 2000). Moreover, using IT in education has added value to the existing teaching<br />
methodology (Broad et al., 2003). eLearning makes it possible to continue studies in parallel with job<br />
or work due to it is flexible nature in term of time and location independence. The quality for eLearning<br />
needs to be maintained and the importance of quality cannot be compromised (Inglis, 2008).<br />
Existing literature highlights gaps in the adoption of eLearning and reports that despite considerable<br />
amount of work done in this area, there is still lack of understanding about the success factors of<br />
eLearning (Hsbollah and Idris, 2009). Most studies present the Critical Success Factors (CSFs)<br />
listings for the adoption of eLearning; however, the lists are not comprehensive. In some studies<br />
student’s perspectives alone are taken into consideration (Selim, 2007; Sun et al., 2008; Underwood,<br />
65
Latefa Bin Fryan and Lampros Stergioulas<br />
2009) while the others merely consider the education provider’s perspectives (Volery and Lord, 2000;<br />
Bell, 2007; Hsbollah and Idris, 2009).<br />
Distance<br />
Learning<br />
eLearning<br />
LEARNING<br />
More effective<br />
LEARNING<br />
eLearning<br />
Figure 1: Effectiveness of <strong>learning</strong><br />
Due to the lack of research into this area, further research is needed for investigating and identifying<br />
such CSFs that would lead to a successful adoption of eLearning. In order to fully realise the potential<br />
of eLearning and understand the challenges in the adoption of eLearning, it is imperative to identify<br />
the Critical Success Factors (CSFs) that affect the adoption of eLearning in KSA educational<br />
institutions.<br />
5. Critical success factors for eLearning adoption<br />
Literature suggests that there has been enormous research conducted on the adoption of eLearning<br />
and several CSFs have been identified. There are two perspectives to the adoption of eLearning, one<br />
is the students’ perspective and the other is the education providers’ (or teaching staff) perspective.<br />
Since, it is important to consider both these perspectives while adopting and implementing eLearning,<br />
therefore this research considers both perspectives.<br />
Bell (2007) highlighted cost effectiveness and time-independent as the CSFs for the adoption of<br />
eLearning, and Homan and Macpherson (2005) reported flexibility in term of the time of <strong>learning</strong>, the<br />
way of <strong>learning</strong>, where you learn as the CSFs for eLearning adoption. Johnson et al. (2009) argued<br />
that technology can play a key role in the effectiveness of eLearning. Volery and Lord (2000) defined<br />
four categories of the CSFs for the uptake of eLearning by education providers:<br />
Expanding access<br />
Alleviating capacity constraints<br />
Capitalising on emerging market opportunities<br />
Serving as a catalyst for institutional transformation<br />
In another study, Hsbollah and Idris (2009) conducted a survey from 244 lecturers of University Utara<br />
Malaysia and identified the following five CSFs for eLearning adoption:<br />
Relative advantage<br />
Compatibility<br />
Complexity<br />
Trialability<br />
Observability<br />
However, the CSFs for all of the above mentioned studies do not cover the student’s perspective in<br />
terms of what makes eLearning adoption a success from student’s point of view. Selim (2007)<br />
conducted a survey of 538 university students and highlighted 8 categories of eLearning CSFs; each<br />
66
Latefa Bin Fryan and Lampros Stergioulas<br />
of the categories contained several critical eLearning acceptance and success measures which are<br />
as follows:<br />
Instructor’s attitude towards and control of the technology<br />
Instructor’s teaching style<br />
Student motivation and technical competency<br />
Student interactive collaboration<br />
eLearning course content and structure<br />
Ease of on-campus internet access<br />
Effectiveness of information technology infrastructure<br />
University support of eLearning activities<br />
However, these CSFs are taken only from the student’s perspective and are lacking the education<br />
provider’s perspective which is equally essential to be considered in order to realise the potential for<br />
eLearning adoption. Underwood (2009) discussed the following CSFs for education with digital<br />
technology (i.e. eLearning):<br />
Increased student effectiveness or performance<br />
Increased student efficiency<br />
Increased student engagement or satisfaction<br />
More positive student attitudes twards <strong>learning</strong> (student’s motivation)<br />
However, these four factors focused only on the student’s performance irrespective to the general<br />
CSFs (i.e. it lacks several CSFs that in general are significant to the adoption of eLearning such as<br />
time and location independence).<br />
Another study (Papp, 2000) presented CSFs for distance <strong>learning</strong> using electronic means (i.e. via<br />
eLearning), these factors include:<br />
Security of Intellectual Property<br />
Suitability for distance <strong>learning</strong> environment<br />
Familiarity with the courses<br />
Course contents<br />
Mechanisms for troubleshooting<br />
Sun et al. (2008) conducted a survey and presented seven CSFs for the satisfaction of an eLearning<br />
user (or student):<br />
Student computer anxiety<br />
Instructor attitude toward eLearning<br />
eLearning course flexibility<br />
eLearning course quality<br />
Perceived usefulness<br />
Perceived ease of use<br />
Diversity in assessments<br />
However, these lists are not comprehensive and lack several important aspects such as effectiveness<br />
(or impact) of eLearning in terms of performance. Rovai and Downey (2010) discussed the major<br />
seven factors causing the failure of many distance education programs:<br />
Planning<br />
Marketing and recruitment<br />
Financial management<br />
Quality assurance<br />
Student retention<br />
67
Faculty development<br />
Online course design and pedagogy<br />
Latefa Bin Fryan and Lampros Stergioulas<br />
However, these factors are not sufficient as they neglect other important <strong>learning</strong> aspects such as<br />
time and location independence focusing mostly on the management-type of CSFs; on the other hand<br />
Park and Wentling (2007) presents another set of CSFs that focuses on the CSFs from the individual<br />
or personal perspective, suggesting that the popularity of eLearning is due to the following factors:<br />
Space flexibility<br />
Cost savings<br />
Timely access to educational resources<br />
Workplace flexibility<br />
Increased students interest<br />
Quick response to the student<br />
6. Findings and analysis<br />
In order to: “Identify and prioritise the Critical Success Factors (CSFs) for the adoption of eLearning in<br />
the educational institutions of the Kingdom of Saudi Arabia (KSA)”. The data was collected through<br />
primary and secondary data sources and then analysed using the interpretative qualitative analysis.<br />
Findings from the secondary data collection are already presented in the Literature Review.<br />
The primary data were collected through conducting semi-structured interviews and a questionnairebased<br />
survey in order to extract the CSFs for the adoption of eLearning in KSA. For this purpose,<br />
FIVE different KSA educational institutions including universities and eLearning training centres were<br />
considered. The participants were chosen according to their related jobs in the area of eLearning in<br />
KSA to ensure high quality and consistent findings. The questions for both the interview and the<br />
survey were designed keeping into consideration the literature surveyed about the adoption of<br />
eLearning. Findings from the interviews and the survey were prioritised according to their significance<br />
in terms of their average or cumulative percentage importance from five educational institutions (the<br />
repetition of each CSFs).<br />
The data collected from both sources was interpreted and qualitatively analysed. The CSFs found<br />
from the primary and secondary data collection were listed in individual tables and then mapped to<br />
each other in another table in order to show the relationships amongst various CSFs; finally the<br />
findings were analysed using Miles and Huberman (1994) scale, and the CSFs were ranked in order<br />
of significance. The resulting CSFs are presented in table 1.<br />
Findings from the interviews and survey questionnaires confirm the anecdotal evidence reflected in<br />
the literature. Most of the CSFs found through primary data collection are common to the ones that<br />
are identified through secondary data collection (literature review), hence confirming the significance<br />
of CSFs identified. However, there are some CSFs that were only identified through primary data<br />
collection and were not found in the literature (presented in Table 2). These CSFs that are identified<br />
only through interviews and survey questionnaires are a value-add provided by this paper as they will<br />
form basis for future research in the area of eLearning adoption.<br />
Findings from primary data and secondary data are shown in Table 1, which presents a list of CSFs<br />
for eLearning adoption, elicited from the existing literature as well as interview and questionnaire<br />
conducted in KSA. The CSFs are listed in four categorisations: Individual; Social; Economic; and<br />
Governmental or Organisational. The individual aspect encompasses both learner and instructor.<br />
These four categorisations help in understanding the different aspects of eLearning adoption<br />
From secondary data 39 CSFs were identified; from primary data 46 CSFs were identified. Most of<br />
the CSFs identified are common in both sources; 25% identified through primary data only; and 12%<br />
identified through secondary data only (presented in Table 3). Finally, a total of 52 CSFs split into four<br />
categories presented in the table 1.<br />
68
Latefa Bin Fryan and Lampros Stergioulas<br />
Table 1: Findings from primary data and secondary data<br />
No. Categorisations CSFs for eLearning Adoption in KSA Educational Institutions<br />
1.<br />
2.<br />
3. Individual<br />
Accuracy<br />
Attendance not compulsory<br />
Attractiveness<br />
4. Compatibility (the degree to which the technology is easy to understand and<br />
use)<br />
5.<br />
Ease of use (user-friendly interfaces)<br />
6.<br />
Effectiveness<br />
7.<br />
Efficiency<br />
8.<br />
eLearning course content and structure<br />
9.<br />
Encourages personalised experience (independent <strong>learning</strong> or self-study)<br />
10.<br />
Enhances learner's achievements<br />
11.<br />
Enriches learner's experiences<br />
12.<br />
Flexibility in term of time, location, and ways of <strong>learning</strong><br />
13.<br />
Increased learner’s engagement or satisfaction<br />
14.<br />
Instructor’s attitude toward eLearning<br />
15. Instructor’s attitude towards the adoption of technology (acceptance for the<br />
technology)<br />
16.<br />
Instructor’s teaching style<br />
17.<br />
Instructors are able to ensure more practice for learners in different ways<br />
18.<br />
Instructors can share good practice (teaching methodology)<br />
19.<br />
Interactive way of <strong>learning</strong><br />
20.<br />
Learner’s motivation for adopting new technology<br />
21.<br />
Learner’s technical competency<br />
22.<br />
Online technical support for providers and learners<br />
23.<br />
Positive trend of accepting eLearning by learner<br />
24.<br />
Reduces effort<br />
25.<br />
Rich and various resources<br />
26.<br />
Time saving<br />
27. Trialability (the degree to which lecturers can test the technology before<br />
adopting)<br />
28.<br />
using internet in the classroom<br />
29. Social<br />
Adds value to the existing teaching methodology<br />
30.<br />
Ease inter and intra communication<br />
31.<br />
Equal <strong>learning</strong> opportunities for all learners<br />
32.<br />
Facilitates both individualised and collaborative <strong>learning</strong><br />
33.<br />
High quality of teaching and <strong>learning</strong><br />
34.<br />
Improved education process<br />
35.<br />
Improves access to education and training (accessibility)<br />
36.<br />
Improves teaching and <strong>learning</strong> performance<br />
37.<br />
Increases inter and intra communication and discussion<br />
38.<br />
knowledge enhancement<br />
39.<br />
Modern (cope with the development)<br />
40.<br />
Observability (the degree to which the results of technology are visible)<br />
41.<br />
Security of Intellectual Property<br />
42.<br />
Serves as a catalyst for institutional transformation<br />
69
Latefa Bin Fryan and Lampros Stergioulas<br />
43.<br />
Technology innovation<br />
44.<br />
Wide acceptance of distant <strong>learning</strong><br />
No. Categorisations CSFs for eLearning Adoption in KSA Educational Institutions<br />
45<br />
46.<br />
47.<br />
48.<br />
49.<br />
50.<br />
51.<br />
52.<br />
Economic<br />
Governmental<br />
organizational<br />
Alleviating capacity constraints in terms of classrooms and staff<br />
Delivering educational programs to more learners at a lower cost<br />
Scalability in terms of learners’ volume<br />
Capitalising on emerging market opportunities<br />
Establish partnerships with national and international universities<br />
Facilitate and support Distance Learning<br />
Internationally recognised<br />
University's support for eLearning activities<br />
The contribution of this research is a comprehensive list of CSFs for eLearning adoption, identified<br />
through primary and secondary data collections methods, which is significant to this area of research.<br />
Of the total 52 CSFs identified, 25% were highlighted only through interviews and survey<br />
questionnaires and were not found in the existing literature. These new CSFs are:<br />
Table 2: CSFs identified in primary but missing in secondary<br />
No. Categorisations CSFs missing from the primary data<br />
1. Enhanced learner's achievements<br />
2. Enriches learner's experiences<br />
3.<br />
4.<br />
Individual<br />
Interactive way of <strong>learning</strong><br />
Instructor’s teaching style<br />
5. Positive trend of accepting eLearning by learner<br />
6.<br />
Reduces effort<br />
7. Ease inter and intra communication<br />
8.<br />
9.<br />
Social<br />
Improved education process<br />
Knowledge enhancement<br />
10.<br />
Wide acceptance of distant <strong>learning</strong><br />
11. Economic Scalability in terms of learners’ volume<br />
12. Governmental/<br />
Establish partnerships with national and international universities<br />
13. Organisational International recognition<br />
Some CSFs that were identified through secondary data but were missed in the primary data are<br />
presented in Table 3.<br />
Table 3: CSFs identified in secondary but missing in primary<br />
No. Categorisations CSFs missing from the primary data<br />
1.<br />
2.<br />
Individual<br />
Instructors can share good practice (teaching methodology)<br />
Trialability (the degree to which lecturers can test the technology before<br />
adopting)<br />
3.<br />
4.<br />
Social<br />
Observability (the degree to which the results of technology are visible)<br />
Security of Intellectual Property<br />
5. Governmental/<br />
Capitalizing on emerging market opportunities<br />
6. Organisational<br />
University's support for eLearning activities<br />
7. Reflection on primary data findings and analysis<br />
According to the interviews and the survey conducted in KSA, a prominent inclination observed from<br />
all the academic staff was towards the positive acceptance of eLearning with the main focus on:<br />
improved quality of teaching and <strong>learning</strong>; increased number of students; and need to cope with 21st<br />
Century development. Undoubtedly, using technology to enhance the educational process will help<br />
KSA to achieve this major transformation of the traditional education. Furthermore, adoption of<br />
eLearning within the educational processes adds value to the existing methods of traditional <strong>learning</strong><br />
and facilitates preparing student to the complex economic society. KSA is in the initial stages of<br />
70
Latefa Bin Fryan and Lampros Stergioulas<br />
adopting eLearning in educational institutions compared to other developing countries; however, the<br />
transformation process is quite rapid in order to achieve the goal of technology-enhanced <strong>learning</strong>.<br />
KSA educational institutions seem to believe strongly that eLearning will help to solve education<br />
issues resulting from rapid population growth and the technology-intensive demands of the future;<br />
moreover, eLearning will facilitate and support distance <strong>learning</strong> which is very common in KSA<br />
universities.<br />
8. Future research and development<br />
For future research, the number of participant and educational institutions can be increased for<br />
extensible research which will help to investigate more CSFs from different perspectives.<br />
Some universities in KSA are already undergoing the process of implementing the eLearning system<br />
and they are using student feedbacks as an essential resource to determine the ongoing quality and<br />
to ensure improvement and refinement of the eLearning system. Considering this positive trend of<br />
eLearning adoption in KSA educational institutions, findings from this research can prove to be a<br />
significant tool for future enhancement and implementation of the eLearning systems in KSA.<br />
9. Conclusion<br />
eLearning adoption in KSA educational institutions has been considered as a huge and major<br />
transformation from traditional educational system. Although the significance of eLearning cannot be<br />
denied and its impact on the improvement of the traditional educational system is well-recognised,<br />
there are still gaps in the adoption of eLearning due to the lack of understanding about the CSFs for<br />
eLearning adoption. In order to fully realise the potential of eLearning and understand the challenges<br />
in the adoption of eLearning, it is imperative to identify the CSF that affect adoption of eLearning in<br />
KSA educational institutions.<br />
For this purpose, the existing literature was thoroughly reviewed for identifying the CSFs for<br />
eLearning adoption and important issues were explored. It was obvious that limited research has<br />
been conducted in this area and a clear gap was identified through the literature reviewed, which led<br />
to the need for investigating the CSFs for eLearning adoption. There are two perspectives to the<br />
adoption of eLearning: students’ perspective and education providers’ (or teaching staff) perspective.<br />
It was indicated through the existing literature that only some studies consider both perspectives and<br />
are limited in terms of the comprehensiveness of the CSFs identified. Since, considering both<br />
perspectives is imperative while adopting and implementing eLearning, therefore this research<br />
considers both of them.<br />
The primary data was collected through conducting semi-structured interviews and survey<br />
questionnaires in order to extract the CSFs for the adoption of eLearning in KSA. For this purpose,<br />
FIVE different KSA educational institutions including universities and eLearning training centres were<br />
considered. The participants were chosen according to their related jobs in the area of eLearning in<br />
KSA to ensure high quality consistent findings. . The interpretive, qualitative data analysis methods are<br />
used, and findings are presented in tables and figures. Findings show that of the total 52 CSFs<br />
identified, 25% were highlighted only through interviews and survey questionnaires and were not<br />
found in the existing literature.<br />
References<br />
Abouchedid, K. & Eid, G. (2004),'eLearning challenges in the Arab world; revelations from a case study profile',<br />
Quality Assurance in Education, 21(1) pp. 15-27.<br />
Alcock, M. (2007),' Rapid eLearning software: what should we be looking for?', development and <strong>learning</strong> in<br />
organizations , 21(6) pp. 15-17.<br />
Al-Faisal, L. Al-Hakami, A. & Al-Sabti, K. (2010). Education: Saudi Arabia’s Commitment to its Future. U.S.-Saudi<br />
Business Opportunities Forum, Chicago, Illinois. Available online: http://www.ussabc.org/i4a/pages/Index.cfm?pageID=3799<br />
[last access 2010-11-03]<br />
Al-Jaghoub, S. et al. (2009), 'eLearning adoption in higher education in Jordon: vision, reality and change'<br />
European and Mediterranean Conference on Information Systems, Izmir<br />
AME Info Website, 2009 http://www.ameinfo.com/219649.html [last access 2010-11-02]<br />
Banyard, P. and Underwood, J. (2008). Understanding the <strong>learning</strong> space. eLearning Papers (9): 1-12. ISSN<br />
1887-1542<br />
Bell, J. (2007),'Feature articles ELearning: your flexible development friend?', development and <strong>learning</strong> in<br />
organizations, 21(6) pp. 7-9.<br />
71
Latefa Bin Fryan and Lampros Stergioulas<br />
Beyth-Marom, R., Chajut, E., Roccas, S., & Sagiv, L. (2003), ‘Internet-assisted versus traditional distance<br />
<strong>learning</strong> environments: factors affecting students’ preferences', Computers & Education, 41(1) pp.65–76.<br />
Broad, M.J., Matthews, M. & Shephard, K. (2003), 'Audit and control of the use of the internet for <strong>learning</strong> and<br />
teaching: issues for stakeholders in higher education', Managerial Auditing Journal, 18 (3) pp. 244-53.<br />
Candy, P.C., Crebert, G. & O'Leary, J. (1994),’Developing Lifelong Learners through Undergraduate<br />
Education’,National Board of Employment, Education and Training, Australian Government Publishing<br />
Service, Canberra..<br />
Fayter, D. (1998), 'Issues in training lecturers to exploit the internet as a teaching resource', Education + Training,<br />
40 (8) pp. 334-339.<br />
Field, M. (1994). Inside the Arab world. Cambridge, MA: Harvard University Press.<br />
Freire, P. (1994). Pedagogy of the oppressed. London, UK: Continuum..<br />
Homan, G. & Macpherson, A. (2005), 'eLearning in the corporate university', Journal of European Industrial<br />
Training, 29 (1) pp. 75-90.<br />
Hsbollah, H. & Idris, K. (2009) ‘eLearning adoption: the role of relative advantages’, trialability and academic<br />
specialisation, 26(1) pp. 54-70.<br />
Inglis, A. (2008) 'Approaches to the validation of quality frameworks for eLearning', quality Assurance in<br />
Education, 16 (4) pp. 347-362<br />
Johnson, R., Gueutal, H. & Falbe, C. (2009), 'Technology, Trainees, Metacognitive activity and eLearning<br />
effectiveness',Journal of Managerial Psychology, 24(6) pp. 545-566.<br />
Kelly, H., Ponton, M., & Rovai, A. (2007), 'A comparison of student evaluations of teaching between online and<br />
face-to-face courses, Journal of Internet and Higher Education, 10(2) pp. 89-101.<br />
Khan, B. H. (2001). A framework for web-based <strong>learning</strong>. Engelwood CliVs, NJ: Educational Technology<br />
Publications.<br />
Miles, M.B. & Huberman, A.M. (1994). Qualitative Data Analysis: A sourcebook of new methods. Sage<br />
Publications.<br />
Newton, R. (2003), 'Staff attitudes to the development and delivery of eLearning', New Library World, 104 (1193)<br />
pp. 412-25.<br />
NZCER. (2004). ’Critical Success Factors and Effective Pedagogy for eLearning in Tertiary Education’. New<br />
Zealand Council for Educational Research (NZCER), Background paper for ITP New Zealand.<br />
Papp, R. (2000),’Critical success factors for distance <strong>learning</strong>’ Paper presented at the Americas Conference on<br />
Information Systems, Long Beach, CA, USA.<br />
Park, J. & Wentling, T. (2007),' Factors associated with transfer of training in workplace eLearning', Journal of<br />
Workplace Learning, 19(5) pp. 311-329<br />
Peled, A. (2000), 'Bringing the internet and multimedia revolution to the classroom' , Campus-Wide Information<br />
Systems, 17 (1) pp. 16-22.<br />
Rovai, A. & Downey, J. (2010), ' Why some distance education programs fail while others succeed in a global<br />
environment', Internet and Higher Education, 13 (3) pp. 141–147.<br />
Selim, H. (2007). ‘Critical success factors for eLearning acceptance: Confirmatory factor models’ , computer &<br />
Education, 49(2) pp. 396–413.<br />
Suddaby, G. & Milne, J. (2008),' Coordinated, collaborative and coherent; Developing and implementing<br />
eLearning guidelines within a national tertiary education system', Campus-Wide Information Systems, 25(2)<br />
pp.114-122.<br />
Sun, P., Tsai, R., Finger, G., Chen, Y. & Yeh, D. (2008), ' What drives a successful eLearning? An empirical<br />
investigation of the critical factors influencing learner satisfaction', Computers & Education, 50 (4) pp. 1183–<br />
1202.<br />
Thomas L. Russell, The No Significant Difference Phenomenon, 5th ed. (Montgomery, AL: International Distance<br />
Education Certification Center, 2001).<br />
Underwood, J. (2009) 'The Impact of the Digital Technology; A review of the evidence of the impact of digital<br />
technologies on formal education' Becta, Publication ID: BEC1-15790<br />
Volery, T., Lord, D. (2000),' Critical success factors in online education', The International Journal of Educational<br />
Management, 14(5) pp.216-223<br />
Wang, Y. (2003). Assessment of learner satisfaction with asynchronous electronic <strong>learning</strong> systems. Information<br />
& Management, 41(1) pp. 75-86.<br />
Wang, Q., Zhu, Z., Chen, L. & Yan, H. (2009),' Guest Editorial eLearning in China', Campus-Wide Information<br />
System, 26(2) pp. 77-81.<br />
Zhang, D., Zhao, J., Zhou, L., & Nunamaker, J. (2004),’ Can ELearning Replace Classroom Learning?’,<br />
Communication of The ACM,47(5) pp. 75-79<br />
72
Challenges in Developing e-Submission Policy and<br />
Practice<br />
Alice Bird<br />
Liverpool John Moores University, Liverpool, UK<br />
a.bird@ljmu.ac.uk<br />
Abstract: In the current economic climate, higher educational institutions are seeking strategies to enhance the<br />
student <strong>learning</strong> experience and, at the same time, reduce administrative costs. Electronic submission of<br />
coursework, e-submission, is frequently perceived as one approach that can help achieve both of these<br />
objectives. However, this view fails to acknowledge the diverse needs of the key stakeholders (institutional<br />
strategic management, academic staff and the student body) and the increased influence of the latter on<br />
institutional policy development. This shift in powerbase imposes additional challenges for those responsible for<br />
developing and implementing high stakes institutional change initiatives, such as e-submission policy and<br />
practice. This paper begins by reviewing the background to developing and implementing electronic submission<br />
policy and practice, in the context of institutional cultures and orientations to academic development. It<br />
acknowledges increased influence of the student voice and enhanced difficulty in developing policy and practice<br />
when mediating between three equally powerful stakeholders; strategic management, academia and the student<br />
body. It offers an insight into the dynamics of these triumvirate influences through a three-year e-submission case<br />
study at one higher educational institution. The paper charts the progress of the case through three phases<br />
(feasibility study, pilot study and early stage implementation), highlighting the influences of the key stakeholders<br />
on outcomes and attempts to gain consensus on policy and practice. The paper concludes by reflecting on the<br />
experiences and findings from the case study and making recommendations on how other institutions might<br />
approach developing similar e-submission policy and practice.<br />
Keywords: e-submission policy, institutional change, institutional compliance, academic freedom, student voice<br />
1. Introduction<br />
In the current economic climate, higher educational institutions are seeking strategies to enhance the<br />
student <strong>learning</strong> experience and, at the same time, reduce administrative costs. Electronic submission<br />
of coursework, e-submission, is frequently perceived as one approach that can help achieve both of<br />
these objectives. The affordances of web-based technologies and content repositories lend<br />
themselves well to the administrative aspects of submission receipting, electronic feedback, online<br />
moderation and overall rapid turnaround. However, this technological view of the process fails to<br />
acknowledge the diverse needs of the key stakeholders; institutional management, academic staff<br />
and the student body. It also ignores the current shift in influence on institutional policy development,<br />
brought about by increased attention to the student voice.<br />
Electronic submission of coursework represents a high stakes cultural change in the way student<br />
work is submitted, marked and feedback provided. The three key stakeholders have diverse, but not<br />
necessarily, mutually exclusive objectives. Higher educational management are mindful that<br />
administrative costs need to be curtailed and view the handling of paper-based items of coursework<br />
as a non-essential activity. Their preference is for institutional compliance on e-submission to reduce<br />
costs and offer a consistent student experience. The student body, empowered by National Student<br />
Survey findings, seeks more flexible approaches to assessment and more timely and informative<br />
feedback on assessment. Members of academic staff continue to advocate academic freedom in<br />
choosing how to teach and assess students, with concerns about radical change in practice and<br />
disempowerment. For successful development of institutional policy and practice, these three groups<br />
must come together and agree an appropriate way forward.<br />
This paper begins by considering the background to changes in institutional powerbases before<br />
charting a three-year case study aimed at implementing e-submission at one higher educational<br />
institution. It concludes by reflecting on the experience and making recommendations for other<br />
institutions considering similar development of e-submission policy and practice.<br />
2. Background<br />
The context of the current paper has resonance with many UK higher educational institutions as they<br />
respond to and prepare for changes in funding and anticipated increase in accountability demanded<br />
by students. Institutions are seeking to reduce costs, particularly those not associated directly with<br />
research or teaching. In many institutions, administrative and support activities have been or are<br />
73
Alice Bird<br />
being reviewed under the banner of improving the student experience. This focus on cutting costs<br />
coincides with increased power of the student voice through the medium of the National Student<br />
Survey. In particular, the outcomes of the survey continue to suggest sector-wide issues in meeting<br />
student expectations on assessment and feedback on assessment. In recent years, the National<br />
Union of Students (NUS) has proactively sought to engage institutions in addressing these issues<br />
through a series of campaigns. For many institutions, this has resulted in a three-way split in the<br />
institutional powerbase; institutional strategic management, academia and the student body, often<br />
represented by the local Student Union. At this point, it should be acknowledged that overall power<br />
within this triumvirate will differ from one institution to another depending on the predominant<br />
institutional culture.<br />
The topic of culture in higher education is one that has been widely researched and discussed.<br />
Trowler (2008) presents an exploration into approaches to understanding higher education cultures,<br />
including references to macro, micro and meso level approaches. At the macro (organisational level),<br />
Trowler cites Berquist’s four cultures (1992); collegial, managerial, developmental and negotiating.<br />
Whilst at the micro (individual) level, culture is defined in terms of the institution and its<br />
distinctiveness, e.g. Tierney (1988). An alternative meso level approach acknowledges a multiple<br />
cultural configuration which encompasses more open, dynamic and interactive concepts e.g.<br />
Alvesson (2002, pp.186-187). In an earlier study, Land (2001) also makes reference to organisational<br />
cultures and maps a set of orientations to academic development against conceptions of change.<br />
These orientations, opportunistic/entrepreneurial, reflective practitioner, interpretive-hermeneutic,<br />
romantic (outreach), professional competence activist-modellers, political strategist (pragmatist),<br />
consultant researcher, disciplinary, managerial/HRM, are adopted by practitioners promoting change<br />
in academic practice according to the context that they find themselves in. Groups and individuals<br />
involved in promoting change in academic practice are generally aware of the predominant and subcultures<br />
that exist within a particular institution. However, the shifts in power, with the student voice in<br />
the ascendancy, mean that they may need to review and revisit their approaches to implementing<br />
change.<br />
As with any work-based or social relationship, it is easier to mediate between two parties than three.<br />
With three stakeholder groups influencing policy and practice, the direction of change can be swayed<br />
with one party often feeling disempowered by the perception of the other parties operating in league.<br />
This paper offers an insight into the dynamics of triumvirate influences through a three-year case<br />
study to develop and implement electronic submission policy and practice at one higher educational<br />
institution. The predominant organisational culture of the institution is generally acknowledged as<br />
managerial, with strategic leadership responsible for driving through change, but with sub-culture<br />
influences often aligned to specific subject disciplines. However, student contribution to development<br />
of policy has become more influential through proactive engagement by the Student Union and a<br />
strategic ‘you say, we do’ approach to improving the student experience.<br />
3. Case study<br />
The recommendation to introduce electronic submission of coursework (e-submission) first evolved<br />
from an institutional review of student administrative processes. The recommendations from the<br />
review were translated into a student experience review implementation project aimed at restructuring<br />
student support services and activities across the institution. Part of the review included an<br />
observation of activities in three centralised centres, processing student submissions of paper-based<br />
coursework items. This revealed a negative impact on student experience; in particular, issues<br />
associated with processing submissions at peak submission deadlines. The outcome was a<br />
recommendation to develop institutional policy and practice for the electronic submission of<br />
coursework. This recommendation was approved and strategically driven from the top through an<br />
institutional development project. The progress of the project through three phases, feasibility study,<br />
pilot study and early-stage implementation, constitutes the core of this case study, with references to<br />
triumvirate influences on the direction and outcomes.<br />
3.1 Feasibility study<br />
The feasibility study was initiated in September 2008 and conducted according to institutional<br />
principles based on the Prince2 methodology. A Project Board was convened, with a School Director<br />
appointed as the Chair and membership representatives drawn from each Faculty, relevant Central<br />
Service Teams and the Student Union. Faculty representatives, who were nominated by School<br />
74
Alice Bird<br />
Directors, for their general interest in teaching, <strong>learning</strong> and assessment, were not necessarily<br />
advocates for e-submission. An initial challenge was defining the project brief and initiation document.<br />
There was no explicit articulation from senior management of the rationale for e-submission policy<br />
and practice, other than fulfilling the recommendation within the student experience review. However,<br />
the general consensus of the project board was that the scope of the feasibility study should not be<br />
limited to exploring administrative benefits but should also cover pedagogical benefits. In particular, it<br />
was considered essential to explore potential improvements in the timeliness and relevance of<br />
feedback to students on assessment, in the light of the sector-wide dissatisfaction expressed in the<br />
2007 and 2008 National Student Surveys.<br />
The project objectives included, as a starting point, outlining the activities associated with the overall<br />
coursework management process. Defining the process was vital to gaining a better understanding of<br />
the potential role for technological support, as a preliminary stage in specifying technical and<br />
functional requirements. The project also sought to assess the needs and perceptions of key<br />
stakeholders, in particular academic staff and students. A final objective was to estimate the resource<br />
implications in moving from paper-based to electronic submission. A project reporting deadline was<br />
set for March 2009, in order to make recommendations for implementation in the academic year<br />
2009/10. Figure 1 provides the final version of the outline process map, agreed after frequent<br />
iterations.<br />
Figure 1: Outline process map<br />
75
Alice Bird<br />
Developing the process map presented a further challenge to the Project Board, in terms of gaining<br />
consensus on the various activities. The process involves much more than submission, feedback and<br />
recording of grades. For example, submission of coursework is used as an indicator of a student’s<br />
continued participation on a course, an essential item of information in tracking local and international<br />
students. The process map also took account of the fact that implementing e-submission would not<br />
necessarily guarantee e-marking, with some staff electing to continue marking printed copies of<br />
submitted coursework items. Refining the start/end points and various routes through the process<br />
generated some debate, with representatives from some subject areas having more clearly defined<br />
conceptualisations than others. However, defining an outline of the process proved invaluable for<br />
highlighting the overall complexity and also in demonstrating that recommendations for submitting<br />
coursework electronically could not be made in isolation from other activities that preceded or<br />
succeeded it. It was also used as the basis for specifying the technical and functional requirements for<br />
the technology<br />
Faculty representatives consulted academic staff on types of assessment items classified as<br />
‘coursework’ and implications for submission file types. This revealed considerable diversity in items,<br />
other than the standard text-based essay, report or portfolio. (See Table 1 below).<br />
Table 1: Types of coursework item<br />
Essays Studio display<br />
Class test (seen and unseen) Video and audio recordings<br />
Documentary exercises Field notebooks<br />
Research reports Posters<br />
Response papers Data authenticity folders<br />
(confidential material)<br />
Presentations (group and individual) Dioramas<br />
Performances Log books/diaries<br />
Portfolios Drawings<br />
Seminar journals Computer programmes<br />
Drawings Mathematical items<br />
Artistic rendering Graphs<br />
Physical models Computer-generated models<br />
The range of possible file types was equally diverse, although common text-based types were likely to<br />
predominate. (See Table 2 below)<br />
Table 2: Example file types<br />
MS Word Web pages/HTML<br />
MS Excel CAD files<br />
MS PowerPoint Computer programmes<br />
Pdf ChemDraw<br />
Video/audio GIS<br />
Image files Other specialist software files<br />
The possible file types indicated the need to identify a technological solution capable of supporting<br />
non-standard submissions. The assignment tool within the institutional virtual <strong>learning</strong> environment<br />
(VLE) was one solution that met this requirement. Users of an existing plagiarism detection service<br />
were keen to continue using it but the limited range of file types that it supported meant that it could<br />
not be the sole solution for electronic submission.<br />
Student and academic staff consultations were also undertaken to assess perceived benefits and<br />
issues in implementing e-submission. Members of academic staff were also asked to comment on<br />
prior experience of e-submission. The Student Union co-ordinated the student consultation by inviting<br />
course representatives to carry out face-to-face interviews with students that they represented.<br />
Faculty consultation was undertaken through Project Board Faculty representatives sending out a set<br />
of standard questions by Faculty email lists. The total number of respondents was not recorded but a<br />
summary of the views of respondents from each of the six Faculties was collated into a general<br />
statement of perceptions and experiences.<br />
76
Alice Bird<br />
The student consultation collated feedback from 30 course representatives who interviewed a total of<br />
160 students. The majority of students interviewed (83%) suggested they would welcome electronic<br />
submission of coursework. Some perceived advantages included ‘less travel for students who<br />
commute’ and, for international students, the ability to ‘return home and still submit’. However, there<br />
was some concern about the potential robustness and reliability of any system for supporting<br />
submission. In particular, students expressed concern about the ability of a system to handle high<br />
volumes of student submissions, at the same time. Other concerns related to potential failure to meet<br />
deadlines due to problems with their own internet connections or computer systems. When asked to<br />
comment on existing submission and feedback experiences, 76% thought deadlines were scheduled<br />
effectively, although some of the remainder felt that deadlines tended to be very close together.<br />
‘Quicker’ and ‘more detailed’ feedback featured highly as a key area for improvement, with 32%<br />
receiving feedback over a month after submission and almost a quarter (23.4%) never receiving any<br />
feedback on assessment.<br />
The Faculty consultation highlighted a wide variation of prior experience across the institution from<br />
individuals with in-depth experience of e-submission and e-marking to other members of staff having<br />
limited experience of technology, in general. The perceived benefits and issues reflected this<br />
diversity. In most cases, the benefits were considered only realisable with associated assumptions; in<br />
particular, a move to electronic marking and feedback. Tables 3 and 4 below summarise the<br />
perceived benefits and issues with associated assumptions.<br />
Table 3: Faculty perceived benefits<br />
Perceived Benefit Assumption<br />
Faster, more direct access by markers to submitted coursework<br />
‘Greener’ coursework submission approach Overall reduced levels of printing<br />
Reduced manual handling of paper-based coursework Reduced levels of printing<br />
More detailed and timely feedback e-Marking and e-feedback<br />
Easier deterrence/detection of collusion/plagiarism Integration with plagiarism detection<br />
service<br />
Table 4: Faculty perceived issues<br />
Perceived Issues Assumption<br />
Failed submission due to students’ own PCs/connections Off-campus submission<br />
Robustness and scalability of submission/feedback system<br />
Lack of staff/student ICT skills and confidence<br />
Lack of local staff support for changing practice<br />
Staff screen usage health and safety issues e-Marking and e-feedback<br />
Increased departmental printing cost Staff opting to mark paper-based copies<br />
Delays in receipt of coursework/provision of feedback Staff opting to use Print Room service<br />
Ability of technology to fulfil associated ‘marking’ requirements:<br />
anonymous, team, double and external<br />
The feasibility study concluded with the publication of a report and set of recommendations,<br />
incorporating a summary of associated resources and costs. The main recommendation was to<br />
implement a pilot study within Faculties in the academic year 2009/10 to assess the extent to which<br />
perceived benefits and issues were likely to be actual benefits and issues when scaling up to<br />
institution wide implementation. A second recommendation proposed further technological<br />
development and performance testing of submission via the VLE to ensure scalability to institutional<br />
requirements. The report also recommended expansion of central and local support to minimise<br />
institutional risk during the pilot phase. Finally, the report acknowledged the need to consider<br />
alternative approaches to assessment and not to simply replicate traditional modes of assessment<br />
within a technology enhanced <strong>learning</strong> environment. There was a strong belief that the affordances of<br />
technology could support alternative, more authentic methods of assessment that should be explored<br />
more widely.<br />
The report was presented to the institutional strategic management group in May 2009. The<br />
recommendations to implement the pilot study and to undertake the technological work were<br />
approved. Additional resources for central and local Faculty-based support were not approved at this<br />
stage as a key objective was to reduce overall administrative and support costs.<br />
77
Alice Bird<br />
Reflecting on the feasibility study, the division of power still seemed to be split mainly between<br />
strategic management and academic staff. Project Board meetings frequently returned to discussions<br />
on issues associated with academic staff engagement in e-submission. The project Chair played a<br />
vital role in keeping the project on track in achieving its objectives. The role of the student union was<br />
mainly consultative rather than a force for change. However, there were early indicators of the<br />
increasing influence of the local Student Union through lobbying in response to a National Union of<br />
Student (NUS) Higher Education Campaign. Working in collaboration with the institutional Learning<br />
Development Unit, Mark My Words, Not My Name, NUS (2008), was translated into a<br />
recommendation for institutional implementation of anonymous marking of coursework.<br />
3.2 Pilot study<br />
The second phase of the e-submission project was initiated in September 2009, following on from the<br />
recommendations of the feasibility study. The stepping down of the Chair from the Project Board<br />
resulted in a Faculty Dean being appointed to fulfil this role.<br />
Preparation for the pilot study included the development of online support resources for staff and<br />
student participants prior to the start of the academic year. During August and September 2009,<br />
participating members of academic staff were recruited and identified specific module cohorts for the<br />
pilot. The pilot was implemented in 52 modules from a range of different programmes. A key aim was<br />
to ensure the pilot covered a diversity of subject disciplines, different cohort sizes and programme<br />
levels, different types of submissions (file types, sizes and single/multiple files) and range of student<br />
and staff levels of ICT competencies. In practice, although all Faculties participated in the pilot,<br />
engagement varied considerably with some Faculties involving whole subject areas, whilst others<br />
allowed self-nomination by individual members of staff. The latter tended to be techno-enthusiasts<br />
and this had to be taken into account when evaluating the pilot. Group and one-to-one hands-on<br />
training sessions were provided for participating staff in using the VLE assignment tool. In some<br />
cases, participating staff implemented e-submission for a single coursework assignment with one<br />
cohort, whilst others piloted e-submission with several cohorts in Semester 1 and Semester 2.<br />
Although the pilot required staff to use a specified submission tool, they were at liberty to determine<br />
how they marked coursework and provided feedback to students. Online help resources were<br />
provided for a range of methods for e-marking but instructions were also provided on how to batch<br />
submit coursework items for printing via the university print room.<br />
Evaluation of the pilot study focused on three key areas; submission, marking and feedback. The<br />
methods used included online questionnaires issued to participating staff and students. Students were<br />
invited to participate in a post-submission questionnaire, whilst members of staff were invited to<br />
complete pre- and post-submission questionnaires in order to assess any shifts in their perceptions<br />
after engaging with e-submission. Triangulation of questionnaire findings was achieved through<br />
analysis of staff and student support calls throughout the pilot and more general feedback collated by<br />
the Faculty and Student Union representatives on the Project Board.<br />
In parallel with the pilot study, technological development and performance testing was implemented.<br />
During the pilot, some additional requirements for augmentation of the VLE assignment tool emerged.<br />
In addition, the outcomes from a Higher Education Academy funded study at the University of<br />
Sheffield Hallam, Hepplestone et al (2010), focused on the use of a customisation of the tool that<br />
helped promote student engagement with feedback on assessment. A demonstration of the system to<br />
the Project Board and other members of academic staff resulted in a strong recommendation for<br />
acquisition of the customised version for any future phase of the project.<br />
The participant response rate to online questionnaires was 49% of academic staff (25) and 17% of<br />
students (167). The outcomes of the evaluation suggested that only a few members of academic staff<br />
in the pilot had prior experience of e-submission. Most staff and student respondents rated their levels<br />
of ICT confidence as confident or very confident. However, approximately 5% of staff had limited or<br />
no confidence and expressed some apprehension in engaging in the pilot. Some staff and students<br />
had experienced technical issues during the pilot but both staff and students agreed that there were<br />
benefits for students, in particular, with the submission stage of the process. The main benefits<br />
included savings on travel time and costs and enhanced ability to submit at anytime and from any<br />
location. Some staff also acknowledged benefits through faster receipt of submissions, although some<br />
of those who chose to print out submitted coursework disagreed with this view. Members of staff were<br />
much more divided on the benefits when taking into account other stages in the process. Some were<br />
78
Alice Bird<br />
unhappy about feeling obliged to mark electronically, whilst others had issues in dealing with specific<br />
coursework types. Some members of staff also experienced difficulties in downloading very large files<br />
submitted by students or in dealing with multiple file submissions. The findings, in terms of whether<br />
students thought that submitting coursework electronically had enabled better and/or timelier<br />
feedback, were inconclusive. A slight majority (39%) thought it had but 30% did not and the remaining<br />
31% were unsure.<br />
When asked to rate their overall experience, two thirds of students considered the experience to be<br />
worthwhile or very worthwhile. A further 22% rated the experience as satisfactory whilst 11% believed<br />
the experience to be not worthwhile or (for two students) totally unsatisfactory. Despite the concerns<br />
raised by staff, 48% rated their overall experience as worthwhile or very worthwhile. A further 24%<br />
found the experience to be satisfactory but 28% considered their experience to be not worthwhile or<br />
totally unsatisfactory. In addition, staff continued to reiterate concerns about health and safety, issues<br />
with printing and overall capacity to cope with any scaling up of e-submission activity.<br />
The findings from the evaluation demonstrated a need to accommodate the diversity in needs<br />
between staff and students. Many on both sides acknowledged benefits but the balance was swung in<br />
favour of students, in terms of convenience and savings to them. As a consequence, the Project<br />
Board developed a proposed ‘third way’ forward that would require, as a minimum expectation, esubmission<br />
for some but not all items of coursework. Hence, the main recommendation was for a<br />
phased implementation in 2010/11 based on the types of coursework item identified as most suitable,<br />
in the pilot study, for submitting electronically. This included items meeting the following criteria:<br />
A single file<br />
In Word or pdf format<br />
Of no more than 2000 words<br />
Submitted before the end of main assessment period<br />
It was believed that assessments meeting these criteria would be more appropriate for academic staff<br />
to test out e-marking, to develop other steps in the submission/feedback process and to look at<br />
alternative approaches to delivering feedback electronically, e.g. voice feedback. This<br />
recommendation was approved by the strategic management group along with the agreement to<br />
implement the customisation of the native VLE assignment tool.<br />
During the pilot study, the influence of the student voice on strategic engagement in institutional<br />
decision-making gained greater significance. A second NUS Higher Education Campaign, NUS<br />
(2009), focused on improvements to assessment and feedback and identified ten principles on which<br />
good feedback should be based. In addition, Government sponsorship for defining ‘a reasonable<br />
student experience’, through the National Student Forum, further raised the need for improved<br />
feedback on assessment along with some fundamental changes to teaching and <strong>learning</strong> organisation<br />
and practice in universities, NSF (2009). (The continued strategic contribution by students, at a<br />
national level, is evidenced in the report commissioned by HEFCE through the NUS on students’<br />
perspectives of technology in <strong>learning</strong>, teaching and assessment, as part of the Online Task Force<br />
consultation, HEFCE (2010)).<br />
In the current case, the Student Union developed its own version of ten ways to make feedback better<br />
based on consultation with students. This was presented to the <strong>Academic</strong> Board, in May 2010, in the<br />
form of a printed document with real examples of positive and negative student experiences. The first<br />
three objectives were approved for implementation. These included a requirement for feedback<br />
deadlines to be published along with assessment, all feedback to be made available three weeks after<br />
the deadline and all feedback to relate to assessment criteria. Of these three objectives, the three<br />
week turnaround for feedback proved most contentious, with academic staff feeling that they had not<br />
been consulted in the decision-making process. As a sense of disempowerment emerged, academic<br />
staff became more aware of greater student influence and apparent collusion between strategic<br />
management and students in decision-making processes. This was further enhanced by other<br />
institutional changes to the academic framework and administrative processes which culminated in a<br />
feeling of initiative overload from the member of academic staff perspective.<br />
79
3.3 Early stage implementation<br />
Alice Bird<br />
During the academic year 2010/11, a number of factors combined to transform the proposed early<br />
stage implementation into an extended pilot rather than wide-spread implementation. <strong>Academic</strong> staff<br />
initiative overload referred to earlier was one factor and there was also a need to align more generally<br />
with other institutional policies, including the proposal for anonymous marking. Participation was wider<br />
than in the previous pilot study, with increased general awareness of a developing e-submission<br />
policy. This led to wider debate amongst the academic community, with the main union representing<br />
academic staff questioning aspects of policy and practice, already raised as perceived issues by staff.<br />
Union concerns about health and safety resulted in a revision of the university’s safety code of<br />
practice on visual display equipment to accommodate laptop and tablet devices. This did not lead to<br />
the identification of any specific e-submission and e-marking safety issue, although staff remained<br />
concerned about their ability to comply with the code and also meet the three week turnaround for<br />
feedback. Another union concern was to ensure the right for staff to print electronically submitted<br />
coursework, although this did conflict with directives in some Schools that electronic submission<br />
should not result in increased departmental costs. A further union concern related to the ability of the<br />
institutional print room to cope with large volume printing a peak deadline times but this was<br />
countered as a historical view of the capabilities of the service.<br />
The early phase implementation is currently being evaluated but the institution has decided to<br />
continue with the ‘third way’ approach for electronic submission. This has been translated into policy<br />
for 2011/12 but in a way that still allows academic staff to choose how to mark and provide feedback.<br />
The decision to maintain the approach, without further amendment, acknowledges the general sense<br />
of initiative overload by academic staff. E-submission does represent a major cultural shift in staff<br />
assessment practices and our experience suggests that it cannot be achieved without agreeing some<br />
form of consensus in approach. The project has also highlighted to strategic management the<br />
complexity of the overall coursework submission process and the need to limit change to what is<br />
practicable at any point in time.<br />
4. Reflection and recommendations<br />
In reflecting on the case study experience, the institution has made progress towards implementing<br />
electronic submission of coursework and to achieving its objectives. The proposal to implement a<br />
‘third way’ was critical in moving the process forward. However, we are conscious that implementing<br />
e-submission is only one aspect of reviewing and improving assessment practices within the<br />
institution. Evidence from existing and emerging research supports the need to redesign assessment<br />
practices across the whole institution, REAP (2011), and within programmes of study, TESTA (2011).<br />
The intention is to continue working with institutional management, academic staff and students to<br />
develop practices that benefit all stakeholders.<br />
In recommending how other institutions might approach development of e-submission policy and<br />
practice, a key starting point is analysis of the predominant culture and sub-cultures. Acknowledging<br />
the increased influence in student power is essential and early inclusion of student leadership does<br />
promote better understanding between key stakeholders. The needs of key stakeholders need to<br />
align but giving stakeholders the opportunity to vent their concerns is also critical in promoting better<br />
understanding. This needs to be conducted in the spirit of listening and moving forward with<br />
agreement to minimise concerns. In our case, frequent consultation throughout the various stages of<br />
the project did not translate into members of academic staff being aware of the introduction of new<br />
policy. Hence, further debate can be expected once any new policy is first implemented. Providing<br />
evidence that perceived issues have already been discussed and solutions considered is essential at<br />
this stage.<br />
Progress can only be achieved through managing the expectations and understandings of all key<br />
stakeholders. Our experience also highlights the need to take an institutional perspective of all<br />
initiatives impacting on academic practice, administrative activities and the student experience, at any<br />
point in time. Finally, any implementation of e-submission policy and practice needs to be considered<br />
in the context of changing assessment practices, in general, and the opportunities that technology<br />
provides for alternative approaches to assessment and feedback on assessment.<br />
80
References<br />
Alice Bird<br />
Alvesson, M (2002) Understanding Organizational Culture. London: Sage.<br />
Berquist, W.H. (1992) The Four Cultures of the Academy. San Francisco: Jossey Bass.<br />
HEFCE (2010) Student Perspectives on Technology – demand, perceptions and training needs. Available online<br />
at http://www.hefce.ac.uk/pubs/rdreports/2010/rd18_10/<br />
Hepplestone, S., Parkin, H., Irwin, B., Holden, G and Thorpe, L (2010) Technology, Feedback, Action! The<br />
impact of <strong>learning</strong> technology upon students' engagement with their feedback. Available online at<br />
http://www.heacademy.ac.uk/assets/EvidenceNet/TFA_report_final.pdf<br />
Land, R. (2001) Agency, context and change in academic development. International Journal for <strong>Academic</strong><br />
Development, 6 (1), 4-20.<br />
NSF (2009) National Student Forum Annual Report 09. Available online at<br />
http://www.bis.gov.uk/assets/biscore/higher-education/docs/n/09-p83-national-student-forum-annual-report-<br />
09<br />
NUS (2008) Higher Education Campaign: Mark my words, not my name. Further information available online at<br />
http://www.nus.org.uk/en/Campaigns/Higher-Education/Mark-my-words-not-my-name/<br />
NUS (2009) Higher Education Campaign: Assessment feedback. Further information available online at<br />
http://www.nus.org.uk/en/Campaigns/Higher-Education/Assessment-feedback-/<br />
REAP (2011) Re-Engineering Assessment Practices in Higher Education. Further information available online at<br />
http://www.reap.ac.uk/Home.aspx<br />
TESTA (2011) Transforming the Experience of Students Through Assessment. Further information available<br />
online at http://www.testa.ac.uk/.<br />
Tierney, W. G. (1988) Organisational culture in higher education. Journal of Higher Education, 59, (1) 2-21.<br />
Trowler, P. (2008) Cultures and Change in Higher Education, 1 -15. New York: Palgrave MacMillan.<br />
81
Enhancement of e-Testing Possibilities With the Elements<br />
of Interactivity Reflecting the Students’ Attitude to<br />
Electronic Testing<br />
Martin Cápay, Martin Magdin and Miroslava Mesárošová<br />
Department of Informatics, Faculty of Natural Sciences, Constantine the<br />
Philosopher University in Nitra, Slovakia<br />
mcapay@ukf.sk<br />
mmagdin@ukf.sk<br />
mmesarosova@ukf.sk<br />
Abstract: Testing in electronic environment is usually considered to be very effective, however, that does not<br />
mean it is objective at the same time, mainly from the students’ point of view. Within three years we carried out a<br />
research based on questionnaire investigation which aim was to explore students’ attitude toward electronic<br />
testing. We found out that students’ perception of this type of testing depends on whether classification is the<br />
main objective of the testing or not. The research also showed that the attitude of students is to a large extent<br />
influenced by the level of their previous experience with e-testing. In spite of the fact that testing is in all<br />
mentioned groups perceived as modern, effective and interesting, it is not completely accepted by all the<br />
students, nor is it considered to be objective. Teachers, on the other hand, perceive e-testing assuredly more<br />
positively. The differences in the opinions were obviously caused also by insufficient interactivity in standard LMS<br />
systems as well as limited support in evaluating the essay tasks. These were the main reasons for us to<br />
concentrate on modification and adaptation of FlashQuestions module implementable into LMS Moodle. The<br />
module allows interactive testing utilizing animations. At the moment, we are carrying out an empirical research<br />
focused on the effectiveness of testing that employs the interactive media objects. So far, we found out that this<br />
method of testing required less time spent on test elaboration by the students in comparison with traditional<br />
“paper” testing. The other interesting feature pertains to the increase of average result (number of points gained<br />
by the students).<br />
Keywords: e-testing, FlashQuestions, interactivity, animations, feedback<br />
1. Introduction<br />
Evaluation and assessment is undoubtedly a very important constituent of teaching and <strong>learning</strong><br />
process since the primary education levels. One of the most frequently used didactic means for<br />
assessing the rate of cognitive skills gained by the students is a didactic test. There are several ways<br />
how to perform it in class, one of them being electronic testing – e-testing. Its advantages lie in its<br />
modernity connected with the usage of ICT in education, simplicity of administration, usability in larger<br />
group of students, as well as reducing of subjectivity in testing and assessing the students’ attempts.<br />
Although it is possible to create own authoring applications, after several years of experience in this<br />
field we found out that the most effective way is to use an existing tool – in our case Learning<br />
Management System (LMS) - that has large community of users and at the same time was open to<br />
modification according to the users’ needs. As the most suitable environment the LMS Moodle was<br />
chosen, not only because of cost (free open software) but mainly because it is a system with growing<br />
user community that continuously helps to enhance its possibilities. By now, the university has<br />
already been using it for several years as the common support of education at most of its<br />
departments – being the first university in Slovakia using LMS in such an extent.<br />
In spite of the fact that LMS Moodle enables the users to create new or modify existing modules, we<br />
decided to implement the model of e-testing in the standard Moodle activity – Quiz. This module<br />
allows the teacher to get the view of students’ knowledge acquired during the term. The tests are<br />
often used as elements of particular lessons were they not only let the teacher to check the students’<br />
knowledge but at the same time are also a feedback tool for the students – they can check their own<br />
progress and understanding of the topic. The outcomes of the test attempts may be included in the<br />
final evaluation. Another possibility is to use the tests as a form of the exam necessary for successful<br />
completion of the subject.<br />
Moreover, the LMS provides a quick statistical report and item analysis of tests, with the possibility of<br />
export to different file types. There are two types of item analysis, quantitative and qualitative, and<br />
they are used for exploring the characteristics of particular items of the test. (Kučera et. al 2008).<br />
82
Martin Cápay et al.<br />
The system supports several question types, the multiple choice questions are the most frequently<br />
used. These questions, in which the choice of one correct answer is usually necessary, are mostly<br />
aimed at testing of facts (dates, values, categorisations) – in fact, they are based on memorization.<br />
The learner can only guess the answers – that is why we qualify these questions as low-level ones.<br />
On the other hand, the questions, in which the student has to form the answer or find it using the<br />
application of learnt procedures, are more valuable. These questions can be considered high-level<br />
questions. The questions that require only recitation of facts take less time to ask and answer which<br />
might be the reason for teachers to sometimes avoid asking high-level questions (Kipper & Rüütmann<br />
2010). Unfortunately, mainly because of the above mentioned reasons, multiple choice questions with<br />
one correct answer usually dominate in most of our courses.<br />
However, we do not want to ask only questions which require rote memory for a correct response, the<br />
questions should also facilitate critical thinking and problem solving. At the same time, we are<br />
interested in application of such a way of testing which reduces the degree of subjectivity in testing. In<br />
the paper, we are dealing with the possibilities of various applications of LMS Moodle in testing, the<br />
usage of multimedia and interactive elements in quiz questions to take advantage of wide range of<br />
existing LMS Moodle tools.<br />
2. The first experiments with on-line testing<br />
eLearning can construct an independent and individualized <strong>learning</strong> environment and break through<br />
the restrictions of conventional <strong>learning</strong> (Wen and Lin 2007). The online testing procedure using<br />
LMSs is probably the most widely used and the quickest way to test students without prejudice<br />
(Skalka and Drlik 2009). Our first attempt for serious usage of online testing in practice was within the<br />
course of programming. We developed a testing model containing a bank of questions and tasks<br />
adapted for implementation in LMS environment. There are questions that require active application<br />
of knowledge and programming skills for the correct answer – in this case the correctness of<br />
responses could be evaluated automatically (Cápay 2010). We used standard Moodle question<br />
types– multiple choice, short answer and true/false questions. To evaluate this type of testing and its<br />
benefits we carried out a questionnaire investigation during the pilot testing period.<br />
2.1 Methodology of research<br />
The main questionnaire research was carried out in course of three years among the Department of<br />
Informatics students at the end of each term. There were different test procedures used during the<br />
terms in three groups of students (Table 1):<br />
1st group – students attempted only traditional testing – writing programs down on a paper, oral<br />
examination, evaluation without computers,<br />
2nd group – students first attempted traditional examination and in the next term they attempted<br />
on-line testing,<br />
3rd group – students attempted only on-line testing.<br />
The research participants were the university students of the first grade (each year different group of<br />
students) and they passed the testing in subjects Programming 1 and Programming 2.<br />
Table 1: Structure of research participants<br />
Number of respondents – Bachelors students of Gender<br />
Group<br />
Applied Informatics<br />
Teaching Informatics<br />
in combination with another subject sum female male<br />
1st 55 14 69 10 84<br />
2nd 55 39 94 10 84<br />
3rd 14 24 38 3 35<br />
sum 69 63 132 13 119<br />
Another feedback we gained to online testing of programming was realised as a national research<br />
among a selected group of secondary and university teachers of Informatics. (Cápay 2009)<br />
Our questionnaires were composed of several questions regarding students’ preferences in<br />
programming testing. We mostly used multiple choice questions with five or seven choices in the form<br />
83
Martin Cápay et al.<br />
of Likert scales. We compared electronic testing to traditional (“paper”) testing using a scale with<br />
seven point scale between the options stated as antonyms, e.g. we used the scale objective vs.<br />
subjective. The question was: Do you think that traditional testing is<br />
objective 1 2 3 4 5 6 7 subjective<br />
effective 1 2 3 4 5 6 7 ineffective etc.<br />
We used nine scales (Figure 1, Figure 2). If the respondent marked number 4, it meant that he/she<br />
regards the particular statement neutrally. On the other hand, marking any other option from given<br />
scale, denoted respondent’s inclination to certain term (on the left or on the right side of the scale).<br />
The results may be compared to the balance pans - one of the terms tips the scales according to the<br />
value it was given by the respondent while the pans are balanced if the respondent does not prefer<br />
any of the terms (antonym responses to the statement).<br />
2.2 Research outcomes<br />
One of the findings of our survey was that the students did not clearly prefer computer testing and<br />
evaluation to teacher’s assessment.<br />
We asked them about the perception of two terms, electronic testing (Figure 1) and “traditional”<br />
(paper) testing (Figure 2).<br />
Figure 1: Comparison of responses in particular phases. Students’ and teachers’ perception of the<br />
term “electronic” testing<br />
The research showed that the attitude of students is to large extent influenced by the level of their<br />
previous experience with e-testing. In spite of the fact that testing is in all mentioned groups perceived<br />
as modern, effective and interesting, it is not completely accepted by all the students, nor is it<br />
considered to be objective. Students who were tested only traditionally (answering the questions on a<br />
paper) claimed they would like to have the opportunity to be tested electronically. Those who<br />
experienced both traditional and electronic testing partly preferred the traditional way of testing.<br />
Finally, the students tested only using electronic tests expressed mostly positive attitude toward this<br />
testing method (Figure 1 and Figure 2).<br />
Based on the responses to other questions in the questionnaire in which 5-choice Lickert scales were<br />
used, we found out that students’ perception of this type of testing depends on whether classification<br />
is the objective of the testing or not. The students preferred combined form of testing where the<br />
teacher still plays the essential role. In some cases they also required to check the automatically<br />
evaluated tests – in these cases the teacher went through the test again with the student and checked<br />
the answers.<br />
84
Martin Cápay et al.<br />
Figure 2: Comparison of responses in particular phases, students’ and teachers’ perception of the<br />
term “traditional” testing (note: The data of the 3 rd group are not available)<br />
On the other hand, we expected a little more traditional view in teachers’ responses but the answers<br />
exposed their modern thinking. Teachers perceive e-testing assuredly more positively. However, the<br />
survey showed that in spite of positive view of e-testing, this method still remains very doubtful, mainly<br />
in the tests used for grading.<br />
The biggest complaints from the students, and partially from the teachers as well, were directed to the<br />
display of remaining time, which according to their opinion rose their nervousness during the testing.<br />
We also found out several differences between the students’ and the teachers’ point of view on<br />
advantages and disadvantages of electronic testing (Cápay and Tomanová 2010). Some of the<br />
students’ responses on disadvantages of e-testing could be summed up as the lack of multimedia<br />
features – the possibility to demonstrate the problem. From the teachers’ point of view the absence of<br />
the question parameterisation was among the most often mentioned disadvantages. Therefore we<br />
endeavoured to improve these two drawbacks in the tests implemented in LMS Moodle in the<br />
following years.<br />
3. Increase of testing possibilities in LMS Moodle<br />
One of the possibilities to increase the testing effectiveness is creation of models (Balogh at al.)<br />
focusing on the methods of adaptive providing of study materials (Kapusta et al, 2009) and<br />
conditioned tests. LMS Moodle is an open system, which means it should not be a problem to suggest<br />
a concept of involving multimedia elements and/or the parameterised questions and implement it into<br />
the system.<br />
3.1 Parameterisation of the questions<br />
LMS Moodle offers quiz questions with the Calculated answers with a collection of input parameters<br />
generated for each question. These parameters are inserted automatically into the question text and<br />
are unique for each student. Currently it is possible to enter a prescription (function) in the process of<br />
inputting the parameters to one output (problem solution). However, it has some limitations. That is<br />
why we also need to consider some other possibilities, e.g. the response in the form of clicking to a<br />
particular part of a picture map, the question in form of flash application, etc., to solve this problem.<br />
Gangur (2011) shows an example of generating a unique test containing cloze questions in the<br />
selected LMS. He proposes cloze questions to be generated in the XML structure (Figure 3) and later<br />
it can be transformed to a more suitable form.<br />
85
Martin Cápay et al.<br />
Figure 3 Process of questions generation in XML format and its transformation to other output<br />
formats (Gangur 2011)<br />
The creation procedure written in any programming language or in any appropriate environment is the<br />
basis of the generating process. For example, Matlab system is used to calculate the results of the<br />
given problems created from randomly generated input parameters. This system is appropriate for<br />
questions that are aimed to e.g. passive programming (the skill to read and analyse unknown source<br />
codes). There would be an existing algorithm (sorting, searching). A question to this algorithm would<br />
be generated according to the particular input values randomly by the basic process. At the same<br />
time, also the appropriate answer would be generated. In the following chapters we focus on the<br />
implementation of multimedia elements that would serve not only to display the task but they would be<br />
the interface to insert the input (answer) of the user as well.<br />
3.2 Multimedia and interactivity<br />
Interactivity is a wide term that can be specified (in case of ICT usage) as an ability of the device<br />
and/or application to react on the user’s stimuli that are evaluated and processed according to the<br />
options, e.g. the program code. With the development of the ICT together with the increasing<br />
measure of interactivity these terms started to be used also in the field of education – in this case the<br />
stimuli are given by the students (Neo and Neo 2004; Sonwalker 2001).<br />
Implementation and usage of interactive media elements within the teaching-<strong>learning</strong> process in the<br />
form of eLearning is these days regarded as one of the essential methods of supporting the<br />
interactivity of the student with the study material. Therefore, there is a need to enable the usage of<br />
interactive questions in the quizzes as well as the evaluation of elaborated interactive questions<br />
directly in the Book module in LMS Moodle e-courses.<br />
4. Implementation of interactivity into the electronic testing in LMS Moodle<br />
Regarding that the internal structure of LMS Moodle is quite unintelligible. It was necessary to design<br />
such a technological solution that would be acceptable not only from the programming point of view<br />
but also from the point of view of simple organization and saving the data in the system database.<br />
First, it was necessary to suggest the way of inserting the interactive animation directly into the<br />
content of the study materials. The students had to be able to manipulate with the animations as well<br />
as to use the interactivity to change the input values. Currently, we are able to integrate the Flash<br />
animation and/or applets into the e-courses thanks to a simple application designed to generate the<br />
HTML code necessary to include the animation or applet within the study material (Magdin 2010).<br />
The next step was developing an electronic testing with the features of interactivity. The most suitable<br />
solution (at this time the only one functional in Moodle version 1.9x) is considered to be the<br />
implementation and partial modification of FlashQuestion module created by Dmitry Pupinin.<br />
FlashQuestion module allows interconnection of the data we get during the usage of interactive tasks<br />
with the system database. Thanks to this interconnection, we are able to statistically process and<br />
evaluate the variables representing correct and incorrect solutions of the tasks. Implementation of the<br />
module in education is very impressive. We can say it is a new approach in the field of testing. The<br />
module offers the possibility of rated assessment of a student as well as the storage of the answers in<br />
the Moodle database. There is a possibility to stop working on the task and continue later. After the<br />
86
Martin Cápay et al.<br />
test is submitted, the student’s answers are displayed together with the correct ones which serve as a<br />
feedback for the student.<br />
Another advantage of this type of quiz questions is the possibility to create interactive tests that<br />
include various types of tasks. Test containing such type of questions is not limited to selecting one<br />
correct answer or filling in the correct word(s) because it is possible to create combinations of tasks in<br />
the form of simulations that are solved by the students. We can insert such flash application among<br />
other quiz questions and combine them according to our needs.<br />
Great advantage of this type of testing is the fact that it eliminates former imperfection of the system –<br />
to estimate if the student “only” made a slip – now it points to the process of tackling the task (whether<br />
it is correct or not). From this point of view the interactive tasks give the students the same<br />
opportunity of communication and correction as if he/she were in direct contact with the teacher.<br />
4.1 Didactic effectiveness of interactive animations<br />
The experimental verification of didactic effectiveness as well as the pedagogical and psychological<br />
research indicate that the effectiveness of perception and retention is directly commensurable to the<br />
number of sensors that are activated during the <strong>learning</strong> process.<br />
According to Lindstrom (1994) the interactivity brings in the ability to remember the information as<br />
follows - when people:<br />
See (20%),<br />
See and hear (40%),<br />
See, hear and can intervene into the action (75%).<br />
Interactive animations, according to the above mentioned definition, do not only function as means of<br />
sense-perception but also as means of cognitive perception as they point to and at the same time<br />
break into the objects’ fundamentals. Furthermore, this continuous increase of the visuality enables<br />
the students to lively look at and understand the basic principles. This is very important especially in<br />
the abstract thinking since the student not only develops his/her concepts but constructs the<br />
fundamentals of logical thinking as well.<br />
Identification of the degree of didactic effectiveness of innovated study materials (i.e. its increase or<br />
decrease) is a relatively problematic step that depends on particular implementation of designed<br />
educational tool into the educational process. The reason is that the question of its function or<br />
effectiveness is always answered in the same way: its function is to support the development of<br />
cognitive and intellectual abilities of the student, i.e. to teach something new. This answer is based<br />
mainly on the proven claim - the more diverse methods of <strong>learning</strong>, the better retention of information<br />
(Mayer 1997, Hvorecký et al. 2010). It is possible to verify the didactic effectiveness of interactive<br />
animation usage by means of didactic tests in which the expected outcome of educational process is<br />
detected by written exam. The student answers a specific question or solves a particular task and<br />
there is usually only one correct answer. There is a possibility of more correct answers – in this case<br />
the student has to be informed in advance. The following task is an example of electronic interactive<br />
didactic test from the Computer Architecture subject (Figure 4).<br />
4.2 Verification of didactic effectiveness of interactive e-testing<br />
In order to correctly verify the didactic effectiveness of a created interactive question, it was<br />
necessary to carry out a research for which we chose the test method – the students were given the<br />
tests in various forms (electronic and printed). Consequently we compared the results of particular<br />
groups of students. There were two research groups:<br />
Experimental group - the group of students that used the study materials with implemented<br />
interactive animations and interactive problem tasks during their study,<br />
Control group - the students who studied from the materials in the form of PDF files without any<br />
interactivity implemented.<br />
87
Martin Cápay et al.<br />
Figure 4: An example of interactive task – Karnaugh map<br />
Prior to conducting the experiment, the following hypotheses were stated:<br />
H1: There is no statistically significant difference in the elaboration and evaluation of given task<br />
between experimental and control group.<br />
H2: The usage of interactive tests will enforce better results of students than the usage of printed<br />
tests.<br />
The tests used in the research were the following:<br />
Electronic interactive test – students elaborated the task of Karnaugh map in the form of<br />
interactive test, after each elaboration of the electronic task we reduced the time limit of the group<br />
by 3 minutes,<br />
Printed test – students elaborated the task on a paper. They were given a table and a picture of<br />
Karnaugh map and had to fill in the appropriate values. In order to specify the time effectiveness<br />
the maximum time for elaboration was stated, in this case 15 minutes, in both groups.<br />
88
Martin Cápay et al.<br />
The test in the experimental as well as in the control group was realised six times. In all cases the<br />
task was to work with Karnaugh map, the difference was in the basic function that was stated to work<br />
with. During particular testing, all the students in research groups got the same task - the difference<br />
was in the form of the task elaboration (electronic test vs. printed test.<br />
To verify the stated hypotheses, we decided to exchange the form of testing after the third test, i.e.<br />
the control group elaborated the test electronically and experimental group used printed form. This<br />
way we ensured the same conditions at elaborating the problem task and each of the groups had<br />
three attempts to elaborate the test in particular form.<br />
Table 2: The data of the experimental and the control group before the exchange<br />
Experimental group<br />
(used interactive materials during their study)<br />
Average<br />
evaluation<br />
Control group<br />
(used non-interactive materials during their study)<br />
Average<br />
evaluation<br />
(mark A-FX)<br />
Test<br />
Time<br />
Test<br />
Time<br />
no. Form of testing limit (mark A-FX) no. Form of testing limit<br />
1 electronic 15 B 1 printed 15 E<br />
2 electronic 12 B 2 printed 15 D<br />
3 electronic 9 A 3 printed 15 D<br />
4 printed 15 E 4 electronic 15 A<br />
5 printed 15 E 5 electronic 12 B<br />
6 printed 15 E 6 electronic 9 A<br />
In the first and the second test, the experimental group gained B mark as the average evaluation, in<br />
the last electronically elaborated test the average mark was A, in spite of the fact that the time limit<br />
was decreased after each test. On the other hand, the control group had the same time limit in all<br />
three tests (15 minutes) and their average evaluation was E in the first test and two times D in the<br />
following tests.<br />
After we exchanged the form of testing in experimental and control group, we found out that the<br />
results of the students were similar to those before the exchange from the point of view of the form of<br />
testing, i.e. the experimental group that now elaborated the printed test gained mark E at average (the<br />
same result was in the control group in the first three tests), in spite of the fact that previously they<br />
elaborated the test electronically and their evaluation was significantly better. Similarly, in the control<br />
group that now elaborated the test in electronic form the average evaluation was two times A mark<br />
and once B mark which is comparable to the results of the experimental group in the first three tests,<br />
in spite of the fact that the time limit was decreased after each test.<br />
Based on the three realisations of testing in two different forms (electronic and printed) we have to<br />
conclude that there is a statistically significant difference in the evaluation of the given task between<br />
the experimental and the control group (Table 2). Therefore, hypothesis H1 has to be dismissed. On<br />
the other hand, we observed significantly better results when using electronic form of testing in both<br />
the experimental and the control group. That means the hypothesis H1 was confirmed. Furthermore,<br />
we can say that the success in the task elaboration and the following evaluation result is not<br />
dependent on the form of previous study of the learners.<br />
5. Conclusion<br />
Several years ago, we found out the perception of electronic testing is more positive than the<br />
perception of traditional “paper” form. At that time, our courses, as well as the tests, were<br />
characterised by static form to quite a wide extent. In the study materials, there were no animations<br />
nor interactivity, the testing was carried out only within the basic possibilities of the used system. This<br />
fact was obvious not only to the teachers but to the students as well. It was expressed in the final<br />
questionnaires they filled in at the end of each term. After some time, the interactive animations were<br />
implemented in several courses. The illustrative value of the study materials was increased and the<br />
testing of particular topics was more clear and transparent. Consequently, we were able to verify the<br />
didactic effectiveness of designed and created tests.<br />
We found out that the implementation of interactive animations and interactive problem tasks really<br />
facilitates the development of intellectual and cognitive abilities. At the same time, based on the<br />
results of the experiment, we can see that the usage of interactive problem tasks is effective from the<br />
point of view of time necessary for the elaboration of the task, as well as the successful task solving.<br />
89
Martin Cápay et al.<br />
On the other hand, there is still the question of problems the students face while elaborating the same<br />
task on a paper (in printed form) even if they are able to solve the task easily in electronic form. After<br />
the discussion with psychologists, we suppose that the reason lies in higher acceptability of electronic<br />
form of testing which, unfortunately, eliminates students’ critical thinking as well as the ability of<br />
formally correct writing. The elaboration of given task is in this case faster but only mechanical.<br />
The observation also showed that electronic tasks, in which there is the possibility to “undo” or “clear<br />
the desktop” (e.g. delete the configurations in Karnaugh map), are more likely to be better solved by<br />
the students. In case of a bad solution in printed test, it is not possible to go back or change the<br />
direction – this significantly shortens the time as well as decreases the willingness to revise the<br />
solution process.<br />
Interactive animations and interactive problem tasks are undoubtedly a very interesting possibility of<br />
the educational process enhancement. However, there are several problems occurring in their usage<br />
that need to be investigated in greater detail and eliminated in the future. We can assume that<br />
utilization of this testing method may help to mobilize the students but, on the other hand, it can also<br />
cause partial elimination of critical thinking as well as the ability to follow the formalism in writing.<br />
Acknowledgements<br />
This paper is published thanks to the financial support of the ESF project ITMS 26110230012 “Virtual<br />
faculty – Distance Learning at FSVaZ UKF in Nitra”, and the national project KEGA 080-019SPU-<br />
4/2010 “Development of creative potential in eLearning”.<br />
References<br />
Balogh, Z., Klimeš, C. and Turčáni, M. (2011) “Instruction Adaptation Modelling Using Fuzzy Petri NETs.“<br />
Distance Learning, Simulation and Communication 2011. Brno. pp. 22-29.<br />
Brooks, I. and Weatherston, J. (1997) The Business Environment: Challenges and Changes, Prentice<br />
Cápay, M (2010). “Testing the programming knowledge and skills in electronic environment.“ Problems of<br />
Education in the 21st Century, Vol. 23.<br />
Cápay, M. (2009). Development and use of intelligent computer systems to support the teaching of informatics<br />
subjects. Disertation thesis.<br />
Cápay, M. and Tomanová, J. (2010). “Enhancing the Quality of Administration, Teaching and Testing of<br />
Computer Science Using Learning Management System.“ WSEAS Transactions on Information Science &<br />
Applications. Vol. 7, No. 9, p. 1126-1136.<br />
Gangur, M. (2011). “Automatic generation of cloze questions.“ CSEDU 2011: Proceedings of the Third Internation<br />
Conference on Computer Supported Education. ISSN-ISBN: 978-989-8425-49-2 .p. 264-269<br />
Hvorecký, J. Drlík, M. and Munk, M. (2010) “The effect of visual query languages on the improvement of<br />
information retrieval skills.“ Procedia - Social and Behavioral Sciences. Vol. 2, No. 2, pp. 717-723.<br />
Kapusta, J., Munk, M. and Turčáni, M. (2009) “Experimental comparison of adaptive links annotation technique<br />
with adaptive direct guidance technique. “ International Conference on Web Information Systems and<br />
Technologies, pp. 250-256.<br />
Kipper H. and Rüütmann, T. (2010). “Strategies and techniques of questioning effectuating thinking and deep<br />
understanding in teaching engineering at Estonian centre for engineering pedagogy. “ Problems of<br />
Education in the 21st Century, pp.36 – 44.<br />
Kučera, P., Kvasnička, R. and Vydrová, H. (2008). “Evaluation of test questions using the item analysis for the<br />
credit test of the subject of mathematical methods in economics in the Moodle LMS.“, Efficiency and<br />
Responsibility in Education. Praha p. 145 - 153.<br />
Lindstrom, R. (1994) The Business Week Guide to Multimedia Presentations: Create Dynamic Presentations<br />
That Inspire. New York: McGraw-Hill.<br />
Magdin, M. (2010).“Integrating interactive flash animations ana java applet in LMS Moodle“. Journal of<br />
Technology and Information Education. Vol 2, No 1, http://www.jtie.upol.cz/10_1.htm.<br />
Mayer, R. (2004) “Multimedia <strong>learning</strong>. Are we asking the right questions? “ Educational Psychologist [online] [cit.<br />
2010-12-07], http://www.ameprc.mq.edu.au/docs/conferences/2003/EileenChau.pdf.<br />
Skalka, J. and Drlík, M. (2009). “Avoiding Plagiarism in Computer Science ELearning Courses.“ Information &<br />
Communicaton Technology in Natural Science Education. Vol. 16.<br />
Sonwalker N. (2001) A New Methodology for Evaluation: The Pedagogical Rating of Online Courses. Campus<br />
Technology from Syllabus Media Group.<br />
Tse-Kian Neo, Mai Neo. (2004). “Integrating multimedia into the Malaysian classroom: Engaging students in<br />
interactive <strong>learning</strong>“, The Turkish Online Journal of Educational Technology – TOJET Vol 3 No 3 [cit. 2011-<br />
08-08], http://www.tojet.net/articles/334.pdf<br />
Wen, T.S. and Lin,H.Ch. (2007). “The Study of ELearning for Geographic Information Curriculum in Higher<br />
Education. “ 6th WSEAS International Conference on Applied Computer Science (ACOS '07).<br />
90
e-Assessment Using Digital Pens – a Pilot Study to<br />
Improve Feedback and Assessment Processes<br />
Tim Cappelli<br />
University of Manchester, UK<br />
Timothy.cappelli-2@manchester.ac.uk<br />
Abstract: Manchester Medical School is the largest medical school in the UK, with over 2000 students on the<br />
MBChB programme. During the final three years of the programme all students undergo regular assessments of<br />
their clinical skills through a series of Objective Structured Clinical Examinations (OSCEs). The OSCEs require<br />
students to carry out a series of simulated exercises in front of an examiner. The examiner completes a scoresheet<br />
for each student, giving a mark between 1 and 7 for each of four criteria together with a ‘global mark’, again<br />
between 1 and 7. The examiner also leaves a small piece of written feedback on the bottom of each form.<br />
Following the exam, all the score-sheets from each of the four teaching hospitals attached to the University are<br />
scanned using an optical reader. This involves a large amount of effort and provides many opportunities for<br />
errors. Due to the work involved and logistical problems, student feedback from the OSCEs is currently limited to<br />
a single mark. Despite the examiners providing the piece of text on the score-sheet, this is only made available to<br />
students scoring less than 4 on their global mark. The students and the School are increasingly motivated to<br />
allow all students access to the written feedback. Hence, in an effort to increase efficiency of the OSCE process<br />
and enable the delivery of student feedback, the Medical School has piloted the use of digital pens as a method<br />
of capturing and processing scoring and feedback. This case study presents the process and evaluation of the<br />
pilot. The study examines the choice of technology, the aims of the pilot and an evaluation of the technology to<br />
assess whether objectives have been achieved. An impact analysis of the use of the pens over a five year period<br />
also shows the return on investment.<br />
Keywords: e-assessment, digital pens, feedback<br />
1. Background<br />
Manchester Medical School is the largest medical school in the UK, with over 2000 students across a<br />
five year MBChB programme. The final three years of the programme are spent based at one of five<br />
teaching hospitals across the North-West of England. With so many students dispersed across so<br />
many sites, the logistics of curriculum delivery and examination is immensely challenging. This is<br />
particularly so when it comes to the assessment of students’ clinical skills. In common with most<br />
medical schools, a student’s clinical skills are assessed using a series of Objective Structured Clinical<br />
Assessments (OSCE) (Harden and Gleeson 1979). OSCEs have been used for over three decades<br />
as a way of providing a reliable and objective method for testing students’ clinical abilities. OSCEs<br />
consist of series of short tests – or stations – that are designed to test a student’s clinical performance<br />
along with competences in history taking and clinical examination. Each of the stations has a different<br />
examiner and students rotate through the stations, to complete all the stations on their circuit. In this<br />
way, all candidates take the same stations, with the same set of examiners, providing the basis of an<br />
objective, structured assessment. The students at the University of Manchester undertake OSCEs<br />
twice a year from year three onwards, with each of the four base hospitals delivering 16 stations per<br />
circuit, 4 times a day over a week to assess all the students in a year group. Each examiner has one<br />
form for each of the students; so with 16 stations and 16 students per circuit, four circuits per day per<br />
hospital and four hospitals, it’s clear that the number of forms soon mounts up. At present all the<br />
forms are marked, collected and checked manually and the forms returned to the Medical Exams<br />
Office (MEO) at the University for scanning and collating. The opportunities for making this process<br />
more efficient and secure through the introduction of electronic marking are clear; several studies<br />
show that e-assessment improves information management and ‘avoids the meltdown of paper<br />
systems’ (Buzzetto-More and Alade 2006, Ridgeway, McCusker and Pead 2007). However, efficiency<br />
savings where not the driving-force behind the Medical School’s commission of a pilot scheme in June<br />
2010; instead it was the lack of feedback given to students after the exam. Each form completed by<br />
the examiner consists of a series of criteria on which the student is given a score between one and<br />
seven, with four representing the minimum standard required. The examiner then gives a ‘global<br />
mark’ for the student’s performance, again between one and seven, and is encouraged to write one or<br />
two lines of feedback in a text box at the bottom of the form. However, due to difficulties in making the<br />
manual forms available to students, only those students who have scored less than four on any given<br />
station are provided with this feedback, and then only sometime after the exam. This is clearly not in<br />
keeping with best practice for summative feedback, with most authors agreeing that feedback should<br />
be constructive, timely and freely available (Nicol and Macfalane-Dick 2006, Gibbs 2010). Increasing<br />
91
Tim Cappelli<br />
demand from students to be able to view this feedback, and in good time, prompted the Medical<br />
School to task a small group within the school to examine, pilot and evaluate the use of electronic<br />
marking by OSCE examiners. The use of information technology to provide immediate feedback from<br />
assessments is well documented (Ridgeway et al 2010), the challenge was to determine the most<br />
appropriate technology.<br />
2. The challenges<br />
The main objective of the pilot was to improve the delivery of feedback to students, both in terms of<br />
the number of students and the time taken for them to receive their feedback. This information was<br />
already being captured, and therefore required no change to the practices of the examiner, but rather<br />
the process by which it was captured, collated and delivered needed to be made more efficient. In<br />
order to determine how this process might be improved through the use of technology, the team<br />
started by detailing the current process. This is shown as a process diagram (Figure 1).<br />
Figure 1: Current OSCE process<br />
92
Tim Cappelli<br />
In order to determine this process, the team carried out a series of interviews with each of the relevant<br />
stakeholders and in doing so also identified issues inherent in the process that stakeholders wanted<br />
to improve or eliminate. From this a set of key objectives were articulated, namely:<br />
Provision of faster results and feedback to students.<br />
Reduction of the opportunities for errors in data handling and erroneous results.<br />
Provision of timely textual feedback to all students.<br />
Reduction of time-pressured manual handling of the marking forms.<br />
Less labour-intensive and faster preparation of forms for the different OSCE exams with less<br />
manual form preparation.<br />
More efficient and faster preparation and transmission of data for analysis by the different roles<br />
involved in administration, management and quality assurance.<br />
3. The solution<br />
According to Love and Cooper (2004), assessment systems should consider the interface,<br />
accessibility, security, reliability and the information to be captured. In considering a potential<br />
technology solution, it was therefore important to consider these issues alongside the system’s ability<br />
to achieve the pilot’s objectives. In particular, the following features where considered crucial:<br />
Data security: transmission of data must be secure and storage of exam scenarios, forms and<br />
results is also secure.<br />
Performance: OSCE exams run on a tight time schedule. The system must not introduce risk of<br />
time delays due to slow response times.<br />
Connectivity: the system must offer solutions for the communications infrastructure that is in<br />
place in the hospitals. Wireless internet availability or provision for wired networks cannot be<br />
guaranteed.<br />
Data integrity: it must be ensured that no data is lost in transmission and that there can be no<br />
mix-up between fields in a form or between forms. It would be an advantage if mandatory fields<br />
could be checked in real-time so that only complete forms are submitted to the system.<br />
Scalability: the system must be able to support the volume of forms needed in the annual cycle<br />
of OSCE exams.<br />
Human Computer Interaction: the examiners’ interaction with the system must be intuitive and<br />
not alter the marking process for the examiner. For example, the examiner must be able to amend<br />
his marks on a form prior to submission.<br />
Training: the device chosen must also be intuitive enough to avoid the need for lengthy or<br />
expensive training for examiners.<br />
Robustness: the chosen technology must be robust and reliable, with a failsafe backup system,<br />
as any delay in the OSCE exam is untenable.<br />
When the team looked at potential technology, there emerged two principal approaches to supporting<br />
the OSCE process. These were:<br />
Hand held devices together with an electronic forms software application. The types of devices<br />
considered were:<br />
Mobile phone<br />
iPad<br />
Tablet PC<br />
Netbooks<br />
Digital Pens together with form management software.<br />
Digitised forms and digital pens<br />
Digital pen and sensor with non-digitised forms<br />
Preliminary research into the technologies and their suitability to support the above process indicated<br />
that this list could be narrowed down still further. The consensus was that the screens available on<br />
mobile phones are too small for marking sheets, allowing examiners to see only a fraction of the sheet<br />
93
Tim Cappelli<br />
at a time and increasing the possibility for incorrect data entry. Feedback from stakeholders indicated<br />
that interaction with the small screens would be difficult for examiners.<br />
Further, a review of the digital pens and sensor solution suggests that, whilst extremely useful for<br />
capturing notes, this approach would be problematic for forms such as the marking sheets. It would<br />
be very difficult to ensure that the student and OSCE data were correctly matched, to verify that the<br />
marks were as intended or to easily compile the marks into a global spreadsheet for analysis.<br />
Removal of these options left the comparison of hand-held devices such as Tablet PC, Netbook or<br />
iPad systems with the digital pen and digitised forms systems. An analysis of both these approaches<br />
against the objectives of the pilot revealed that all these technologies could potentially achieve the<br />
objectives. Hence, the choice of technology rested on which offered the cheapest and most practical<br />
solution. Each approach was assessed against the set of criteria above and the results are shown in<br />
Table 1 below.<br />
Table 1: A comparison of touch based devices with digital pens<br />
Criteria Tablet PC based solution<br />
Handheld device with electronic<br />
forms software on the client and/or<br />
server. Examiner interacts with the<br />
Data security<br />
Performance<br />
Connectivity<br />
Data integrity<br />
Scalability<br />
Human<br />
Computer<br />
Interaction<br />
Training<br />
Robustness<br />
device with stylus and/or finger<br />
Data can be cached securely on the<br />
local device and transmitted or<br />
downloaded in encrypted format<br />
Data can be captured locally on the<br />
device without the need for<br />
connection to the network. Devices<br />
generally reliable but are at risk from<br />
battery or other failure<br />
Data can be stored on the local<br />
device and transmitted or<br />
downloaded when possible.<br />
Data can be stored on the local<br />
device and could be checked on<br />
each device, or downloading to a<br />
local PC prior to transmitting to<br />
Central Office for collation<br />
Scaling to all base hospitals would<br />
require the purchase of 16x4 (64)<br />
devices with sufficient reserves as<br />
back up<br />
Touch devices are dependent on the<br />
software that presents the form. The<br />
software could be designed to be<br />
intuitive and allow for corrections,<br />
etc.<br />
Some training would be required for<br />
any examiners unfamiliar with touch-<br />
based technologies.<br />
The devices tend to be reliable<br />
though a major device failure that<br />
results in all data loss cannot be<br />
ruled out.<br />
Set up Costs Cost per device: £600+<br />
Software: £20k +<br />
Licence cost (per device): £180<br />
Maintenance<br />
Costs<br />
Training costs: £3500<br />
Ongoing support: £2500<br />
Forms cost: £0<br />
Digital Pen based solution<br />
Digital pens and digitised forms. Forms<br />
management software on the server. Examiner<br />
interacts with the system with the pen and paper<br />
in the same way as currently<br />
Data can be cached securely on the local device<br />
and transmitted or downloaded in encrypted<br />
format<br />
Data can be captured locally on the device<br />
without the need for connection to the network.<br />
Devices generally reliable but are at risk from<br />
battery or other failure, though the use of hardcopy<br />
forms ensures the test can continue and<br />
data captured later.<br />
Data can be stored on the local device and<br />
transmitted or downloaded when possible.<br />
Data can be stored on the local device and could<br />
be checked by downloading to a local PC prior to<br />
transmitting to Central Office for collation<br />
Scaling to all base hospitals would require the<br />
purchase of 16x4 (64) devices with sufficient<br />
reserves as back up<br />
The pens look and feel like an ordinary pen and<br />
are therefore already familiar to examiners. The<br />
software that comes with the pen covers the<br />
required features.<br />
No training should be required, but rather a<br />
highlighting of the pen’s features in the current<br />
briefing<br />
The devices tend to be reliable though a major<br />
device failure that results in all data loss cannot<br />
be ruled out. Fortunately, the continued use of<br />
hard-copy forms and ink with this approach<br />
provides a back-up without further investment.<br />
Cost per device: £120<br />
Software: £500<br />
Licence cost (per device): £315<br />
Training costs: £0<br />
Ongoing support: included<br />
Forms cost: £250<br />
On the basis of the data gathered, the digital pen based solution appeared to offer a number of<br />
advantages, namely:<br />
94
Tim Cappelli<br />
It was closest to the examiners’ current way of working so does not require acceptance of new<br />
technology and training is negligible.<br />
If the technology fails for some reason, there is always a backup in the form of the paper forms<br />
that have been completed.<br />
The technology is lightweight, simply stored and maintained.<br />
The solution is scalable with no additional cost for software on multiple servers.<br />
The cost of the solution overall is the lowest.<br />
Based on the team’s recommendation, the Medical School went ahead with a pilot of the digital pens<br />
and forms in November 2010.<br />
4. The pilot<br />
It was decided to run the pilot with the selected technology at one of the base-hospitals in order to<br />
assess the viability and practicalities of the pens, how well the stakeholders responded and how well<br />
the devices fulfilled the original objectives.<br />
The team invited proposals from a number of digital pen providers. From a series of email exchanges<br />
and follow up interviews, Ubysis Technologies (http://www.ubisys.co.uk/) were chosen based on their<br />
experience in the Health Service and their accompanying software, Formidable. Formidable is a<br />
server-based application that lets users design, create, print and manage digital forms. It also<br />
interacts with the pen-software, Pen Pusher, to enable the upload and automatic checking of forms<br />
prior to exporting the results to Excel or another data analysis tool. This end-to-end support appeared<br />
to offer the best solution in supporting the OSCE process and a pilot was agreed.<br />
The pilot consisted of an initial dummy run of the pens using a number of dummy OSCE forms<br />
provided by the School and created in Formidable by Ubysis. Members of the development and<br />
OSCE administration teams played the roles of students and examiners. The purpose of the dummy<br />
run was to identify any issues or pressure-points in the process that might occur in the real exam, but<br />
also to reassure OSCE administrators that the technology was reliable and fulfilled their requirements.<br />
Following the dummy run, the School decided to go ahead with a pilot on a real OSCE. Year four<br />
OSCEs were chosen at one of the hospitals and it was decided to run the pilot across 8 stations, with<br />
the other 8 using traditional pens and forms to act as a direct comparison. The pilot would stretch<br />
across 4 cycles per day for 4 days; in total, 1024 forms would be marked using digital pens. The<br />
Medical School purchased 10 digital pens (8 for the pilot and 2 back-up) along with the necessary<br />
licences and a ‘shelf’ of printing patterns that was required to print the required number of unique<br />
forms.<br />
The OSCEs took place in December 2010, and during November, Ubysis worked with the<br />
development team to create and print the forms, train staff on the use of Formidable and ensure all<br />
the technology was ready prior to the OSCEs. They also provided a member of staff to be present on<br />
the first day of the pilot in case of any software or hardware issues occurring.<br />
For the pilot, Formidable was installed on a development server, and two network-connected laptops<br />
were placed in an IT room where the exams took place. Both laptops had docking devices linked via a<br />
USB port, in order to upload the data from the pens to the server. Although it is possible to transmit<br />
the completed forms directly from the pen to the server using a linked mobile phone, it was felt this<br />
would result in additional cost, more confusion for the examiners and a higher risk of data loss.<br />
Instead, it was decided the forms would be downloaded via the laptops at each break, after which<br />
they could be checked for completeness both automatically by the software, and also by the<br />
administrators as a fail-safe.<br />
Forms downloaded automatically once the pens were placed in the docking device and an image of<br />
the mark sheet, alongside a table of the marks gathered from the form, then appeared on the screen.<br />
The software automatically checked for any missing marks and for the presence of text in the<br />
feedback field if the overall score was less than 4. A visual inspection by the administrators ensured<br />
that wherever there was more than one mark for any criteria, that the intended score had been<br />
accepted. Following these checks, each form was saved on the server. At the end of each day, all<br />
95
Tim Cappelli<br />
forms were processed and the data (pdf copies of all the forms and a csv file of all the scores) made<br />
available to the Medical Exams Office (MEO) over a secure link.<br />
Prior to every OSCE cycle, all examiners attend a compulsory briefing session on how to complete<br />
the forms. During the pilot, a single slide was added to this briefing to inform examiners about the<br />
technology and that the pens would be collected during the break. They were not required to change<br />
their practice in any way.<br />
5. Evaluation<br />
The pilot was observed and key points recorded throughout by the development team. In addition,<br />
participating examiners were given questionnaires to complete and examination and support staff<br />
were interviewed two weeks after the event.<br />
5.1 Feedback from stakeholders<br />
Examiners: 54 examiners took part in the pilot and 42 completed the evaluation questionnaire (78%).<br />
91% of examiners felt that using the pens was the same as normal OSCE exams and 7% felt they<br />
were better. 50% of examiners felt positive and 36% neutral about providing written feedback for all<br />
students. Two examiners (5%) reported feeling displeased about this citing extra burden for<br />
examiners and questioning whether the purpose of an OSCE exam was to provide teaching for the<br />
students. There was also a desire for more information about the technology i.e., what it is for, how<br />
accurate it is. In terms of using the pen, examiners commented that, apart from being a bit bulky and<br />
therefore perhaps altering the readability of their handwriting, the pen was just like using any other<br />
pen.<br />
Administrators: Examination support staff reported that the examiners were positive but surprised<br />
when they were given the briefing in the morning. They reported that there seemed to be initial<br />
concern from the examiners but that the examiners appeared reassured when it was just a ‘pen and<br />
paper’. Staff felt that they needed two more members of staff on exam days than usual for docking the<br />
pens and checking the mark sheets as they docked. However, this would not be the case if the whole<br />
circuit had been completed using pens, since pen docking and form checking occurred at the break,<br />
when staff traditionally collect and check the hardcopy forms. Staff did express enthusiasm to trial the<br />
mobile connection as this would decrease their workload further (no need to collect and dock the<br />
pens) and would allow checking to be done whilst the OSCE was ongoing. Exam staff reported that<br />
instead of sending marks to central exams office on the Tuesday following a week of OSCEs, they<br />
were able to send marks at the end of each day of exams. This would speed up the data analysis<br />
and would enable data analysts to have extra time to perform statistics prior to the exams board.<br />
5.2 Achievement of objectives<br />
Pulling together the observed data and feedback from stakeholders, the outcomes of the pilot were<br />
assessed against the original objectives. These are shown in Table 2.<br />
5.3 Return on Investment<br />
Given the potentially high costs of implementing the digital pens across all the hospital bases, it was<br />
decided to complete a Return on Investment impact analysis, using the potential efficiency savings in<br />
time and resources identified in the pilot. This was designed to sit alongside the evaluation of the<br />
objectives in order to demonstrate the period of time required before any financial savings on the<br />
initial investment were realised. Although financial savings were not seen as a driving force for the<br />
pilot, any evidence of efficiency savings could be used to justify the large initial outlay required for<br />
technologies to be adopted.<br />
The impact analysis was carried out using the JISC InfoNet Impact Calculator<br />
(http://www.jiscinfonet.ac.uk/impact-calculator) which allows articulation of the business processes,<br />
the nature of the benefits to be achieved, the capture of performance data for each of the benefits and<br />
recording of the costs associated with realising the change. The calculator was completed, focusing<br />
on the savings in staff time and resources afforded by the adoption of the technology across the<br />
hospitals. Given the large number of forms involved, the few minutes saved in checking each form<br />
and the few pence saved in not photocopying each form soon accrued to a substantial amount.<br />
However, the largest savings came from no longer having to transport the forms manually from each<br />
96
Tim Cappelli<br />
hospital back to the MEO, and from MEO staff no longer having to manually input rejected forms. In<br />
total, the annual savings from introducing digital pens across all the hospitals was calculated to be<br />
over £12,500 per year. Given the initial set up costs and ongoing support, maintenance and licensing<br />
fees, the pens would therefore become funding neutral after four years, after which they would<br />
continue to save the Medical School, year on year.<br />
Table 2: An evaluation of the digital pens pilot against the original objectives<br />
Objectives<br />
Evaluation Impact<br />
Faster results and<br />
feedback to students.<br />
Provision of timely<br />
textual feedback to all<br />
students, not just<br />
those who performed<br />
poorly.<br />
Reduction of timepressured<br />
manual<br />
handling of the<br />
marking sheets<br />
(checking correct<br />
completion, dealing<br />
with exceptions,<br />
manual data entry,<br />
transfer of forms).<br />
Less labour-intensive<br />
process and faster<br />
preparation of forms<br />
for the different OSCE<br />
exams with less<br />
manual form<br />
preparation.<br />
The pilot demonstrated that it is possible<br />
to provide the mark sheets, scores and<br />
text feedback immediately to the MEO on<br />
completion of an exam.<br />
Once checked by MEO and exams<br />
board, the forms with the scores and<br />
feedback (as pdfs) could be made<br />
available to students in electronic form.<br />
The pilot was designed to provide a pdf<br />
image of the text feedback for students<br />
Having the possibility to provide an image<br />
of the mark sheet or of the feedback text<br />
alone provides the facility to give<br />
feedback to all the students, whatever<br />
their overall score.<br />
Initially, the digital pens were taken from<br />
examiners during ‘breaks’ in the cycle to<br />
upload and check forms on the laptop.<br />
However, it became apparent that<br />
examiners often returned to a student’s<br />
form later in the cycle to amend the entry,<br />
meaning the form had to be re-checked.<br />
Hence, it was decided to leave the<br />
upload and checking to the end of each<br />
cycle.<br />
It was not necessary to photocopy the<br />
mark sheets. Once checked and saved,<br />
each exam batch of results was<br />
converted to a CSV file and sent to MEO.<br />
The necessity to scan forms, deal with<br />
exceptions and manually input incorrect<br />
forms was removed, saving significant<br />
time.<br />
The forms for this pilot were created<br />
using Formidable, The forms preparation<br />
was done by the Ubisys technical support<br />
team for this pilot. They also prepopulated<br />
the forms with student data<br />
provided to them a short time before the<br />
exam.<br />
The same number of forms for the OSCE<br />
stations using the pen was created as<br />
they would be using the manual process<br />
– i.e. one form per student per station.<br />
97<br />
Students would receive results faster<br />
than at present and could receive the<br />
feedback without having to come to<br />
exam administrators or their tutors to<br />
see their mark sheets.<br />
Provision of feedback with the digital<br />
pens will require some additional<br />
training for OSCE examiners. They<br />
must keep the text within the box<br />
provided and it may require them to<br />
write more legibly. This can be included<br />
in the current training and briefing of<br />
examiners.<br />
Providing feedback legibly appears to<br />
be a small change to current practice<br />
and the impact of providing timely<br />
feedback to students could be<br />
extremely positive.<br />
Once the administration staff were<br />
familiar with the process, the time taken<br />
to upload and check the forms on the<br />
laptops was significantly less than the<br />
time taken to collect, collate and check<br />
the forms manually.<br />
The business rules written into the form<br />
software made checking the mark<br />
sheets easier, and in the future<br />
additional rules could be included to<br />
further aid this.<br />
There is a significant saving in the time<br />
and cost of photocopying the mark<br />
sheets.<br />
The time taken for collection and<br />
collation of data is significantly reduced,<br />
saving central office staff considerable<br />
effort.<br />
The Formidable software allows<br />
preparation and pre-population of<br />
forms. Some further analysis is required<br />
to establish the training and skills<br />
needed for MEO to do this in future.<br />
There is a significant saving in the use<br />
of paper (only one copy is created and<br />
no photocopying done) and in the<br />
moving and storage of paper forms.
Objectives<br />
More efficient and<br />
faster preparation and<br />
transmission of data<br />
for analysis by the<br />
different roles involved<br />
in administration,<br />
management and<br />
quality assurance.<br />
6. Conclusion and next steps<br />
Tim Cappelli<br />
Evaluation Impact<br />
The exams data was sent from the base<br />
hospital to MEO on the day of each<br />
OSCE exam. It was sent electronically<br />
over a secure link rather than as bundles<br />
of paper forms, reducing the time, cost<br />
and security of data transmission back to<br />
the centre.<br />
Because the data is already in<br />
electronic format, the data can be easily<br />
collated and assimilated into existing<br />
systems for analysis and distribution.<br />
Having business rules in the software<br />
that instantly flag incorrect forms or<br />
missing data before the examiners<br />
leave the site reduces the number of<br />
errors and facilitates the QA of the<br />
process.<br />
The pilot of the digital pens was evaluated and reported on to the Senior Management Team of the<br />
School together with a set of recommendations for large-scale roll out across all the base hospital.<br />
The recommendations were:<br />
Clarification of the data requirements of the Medical Exams Office (MEO).<br />
Demonstration of the process of using the pens to MEO staff.<br />
Assessment of the value of providing timely feedback to students: Another pilot, similar to the<br />
first, should be run. This time the results from the pens, including the textual feedback, should be<br />
passed directly to the students once the data has been collated, checked and approved for<br />
distribution.<br />
Test the viability and reliability of the mobile phone upload.<br />
Exploration of vendor support: Although Ubysis provided adequate support and advice prior to,<br />
and during, the pilot, it would be prudent to seek assurances that this level of support will continue<br />
in any future<br />
Training of Base administration staff in downloading and checking the pen’s content.<br />
Training of OSCE examiners: Some examiners expressed concerns over lack of training to<br />
provide appropriate feedback.<br />
The first two points were particularly important as there had been a great deal of resistance to the<br />
introduction of the technology from the MEO. This was partly due to a lack of communication and<br />
resulting misunderstanding of the technology and partly due to concerns about the technology making<br />
certain roles redundant. The result was that despite the pilot, MEO staff were still opposed to the<br />
technology and promoted the view the pilot had been unsuccessful. This ‘dis-information’ rapidly<br />
became the prevailing view and work was required to assure the concerns of the MEO staff. This<br />
demonstrates the importance of embracing all stakeholders at the start of any technology change.<br />
At present, the SMT are reviewing the results of the pilot, together with other initiatives in student<br />
feedback, to determine the most appropriate way forward. A final decision will be based on available<br />
resources and the views of staff and students.<br />
Acknowledgements<br />
Thanks to Dr’s Hilary Dexter and Lucie Byrne-Davies from the Manchester Medical School for their<br />
work on the student evaluation that informs much of this study.<br />
References<br />
Buzzetto-More, A. and Alade, J.A. (2006) “Best Practices in e-Assessment”, Journal of Information Technology<br />
Education, Vol 5, pp 251-269.<br />
Gibbs, G. (2010) Dimensions of Quality, The Higher Education Academy,<br />
http://search3.openobjects.com/kb5/hea/evidencenet/resource.page?record=12nH2AFIYcc [accessed 12<br />
August 2011]<br />
Harden, R and Gleeson F (1979) “Assessment of clinical competence using an objective structured clinical<br />
examination (OSCE)”, Medical Education, Vol 13, No.1, pp39-54.<br />
Love, T and Cooper, T (2004) “Designing online information systems for portfolio-based assessment: Design<br />
criteria and heuristics”, Journal of Information Technology Education, Vol 3, pp65-81.<br />
98
Tim Cappelli<br />
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.77.6803&rep=rep1&type=pdf [accessed 15<br />
August 2011]<br />
Nicol, D. & Macfalane-Dick, D. (2006) Formative assessment and self-regulated <strong>learning</strong>: a model and seven<br />
principles of good feedback practice. published in Studies Higher Education 2006, Vol 31(2), pp99-218.<br />
http://tltt.strath.ac.uk/REAP/public/Resources/DN_SHE_Final.pdf [accessed 12 August 2011]<br />
Ridgway, J. and McCusker, S. and Pead, D. (2007) ’Literature review of e-assessment.’, [online].<br />
Futurelab, Bristol http://dro.dur.ac.uk/1929/1/Ridgway_Literature.pdf?DDD29+ded0kmt [accessed 15 August<br />
2011]<br />
99
Digital Educational Resources Repositories in Lower and<br />
Middle Education in Portugal: Quality Criteria in the<br />
International Context<br />
Cornélia Castro 1 , Sérgio André Ferreira 1 and António Andrade 2<br />
1<br />
School of Education and Psychology, Portuguese Catholic University, Oporto,<br />
Portugal<br />
2<br />
School of Economics and Management, Portuguese Catholic University,<br />
Oporto, Portugal<br />
corneliacastro@gmail.com<br />
sergioandreferreira@gmail.com<br />
aandrade@porto.ucp.pt<br />
Abstract: The thematic of Digital Educational Resources (DER) and digital educational repositories at primary<br />
and secondary education levels is of growing centrality in the educational themes in the international debate. In<br />
order to make it easier for teachers and students to find the best and relevant <strong>learning</strong> resources and to<br />
encourage teachers’ uptake of innovative materials and <strong>learning</strong> styles, authorities and/or companies in<br />
European countries have launched web based educational repositories. Following this trend, about two years<br />
ago, three major institutions in Portugal have launched their repositories: Portal das Escolas (ME), Casa das<br />
Ciências (FCG) and BOA (INESC-ID). After examining the state of the art of educational repositories in Europe, it<br />
was found that those three Portuguese repositories are not mentioned in recent European reports and so the<br />
study here presented seemed to be very relevant. This work aims to develop a descriptive and comparative<br />
study, in order to characterize the reality of the Portuguese repositories and compare it with European examples<br />
of good practices. To guide this investigation, the following question was theorized: “How are the Portuguese<br />
institutional repositories, for primary and secondary education levels, positioned in what concerns international<br />
quality criteria?” Our methodology design is a qualitative one and of type i) descriptive, as it is intended to<br />
characterize the main Portuguese institutional DER repositories for primary and secondary school education,<br />
aiming to assess the key dimensions that affect their quality and ii) comparative since it has in view comparing<br />
those repositories with reference European ones. In this sense, Lektion in Sweden and Scoilnet, Portal for Irish<br />
Education in Ireland are considered. The study was undertaken from the view of the DER producers and users.<br />
As a tool for data collection it was used a framework of analysis built on the Guía para la Evaluación de<br />
Repositorios Institucionales e Investigación (GERII), a joint publication of the Spanish Ministry of Science and<br />
Innovation and other organizations in the fields of Education, Science and Technology (Villar et al, 2010). The<br />
GERII guide includes a set of guidelines for the creation and evaluation of a repository, bringing together a<br />
battery of 31 evaluation criteria, distributed over seven dimensions that any repository must accomplish to meet<br />
quality and therefore was considered an excellent basis for our analysis. Comparing the three Portuguese<br />
repositories, target of this study, with two reference European repositories in order to contextualize the national<br />
reality in the international arena will stand for the Portuguese institutions that hold the repositories as an<br />
opportunity for improvement. Thus, Portuguese teachers will have an increased chance to access an eclectic<br />
collection of quality resources, in order to diversify their teaching strategies.<br />
Keywords: comparative study; digital educational resources; evaluation; quality; repositories<br />
1. Introduction<br />
The 2010 edition of the Horizon Report (Johnson et al., 2010) refers to the Open Content as an<br />
emerging trend. In fact, we are witnessing a growing volume of digital content from different sources<br />
(OECD, 2007) in the education field. It is shared in the cloud DER which: i) contain inherently<br />
educational purposes; ii) are framed on the needs of education systems, iii) have identity and<br />
autonomy in relation to other objects and iv) meet the defined quality standards (Ramos et al., 2010).<br />
This environment of sharing and reuse makes the access, by teachers, to a wide range of resources<br />
easier, increasing the exploitation of more diversified contents (OECD, 2007), which is considered as<br />
an important condition for the improvement of quality in education.<br />
In this study we aimed to develop a descriptive and comparative study in order to characterize the<br />
reality of three Portuguese repositories and compare it with European examples of good practices in<br />
order to evaluate how Portuguese institutional repositories stand in the international arena. To<br />
achieve that we established the following research question: “How are the Portuguese institutional<br />
repositories, for primary and secondary education levels, positioned in what concerns international<br />
quality criteria?”<br />
100
Cornélia Castro et al.<br />
After examining the state of the art of educational repositories in Europe (EdReNe 2009, Poulsen,<br />
2008; Dumitru et al, 2009), we verified that the variety in DER repositories in Europe is huge and the<br />
approaches taken by the various countries (22, altogether) differ. However, all countries and<br />
developers share the objective of making resources available and visible to the users and face the<br />
same challenges of reaching many users, managing rights and adapting standards (EdReNe, 2009:<br />
4). The same state of the art showed us that three major Portuguese repositories: Portal das Escolas<br />
(Schools Portal – ME), Casa das Ciências (House of Sciences – FCG) and BOA (Learning Objects<br />
Exchange – INESC-ID), were not mentioned in recent European reports although the 2 nd Strategic<br />
Seminar of EdReNe took place in Lisbon on June 9 th – 11 th , 2008 [exception for Portal das Escolas<br />
once, under the Portuguese Technological Plan for Education (PTE) and its three axes, “(…) the<br />
content area will be supported by a Schools Portal to be used by teachers and students in schools all<br />
over the country (…)” (EdReNe 2009: 68)]. We considered these to be the main Portuguese<br />
repositories for the following reasons: 1 st – a varied typology of resources; 2 nd - curriculum areas<br />
covered; 3 rd - number of publications; 4 th - number of downloads made; 5 th - number of visits.<br />
Lektion, a web community for teachers, created in 2002, was chosen because it is Sweden’s largest<br />
teachers’ site, a so called “(…) meeting place for teachers and all who work in Swedish schools (…)”<br />
(Lektion.se, 2011). Scoilnet is the department of Education and Skills official portal for Irish education<br />
(Scoilnet.ie, 2011) and was selected because it is an institutional repository whose resources are<br />
constantly being reviewed by a team of over 30 curriculum experts. Besides, it allows the users to<br />
meet the repository at their local school web portal. Both Lektion and Scoilnet were still selected<br />
because i) their url was available; ii) their diversity of resources and iii) they were launched well before<br />
the Portuguese ones.<br />
The relevance of this study is also justified by the fact that the results could prove to be inputs to the<br />
three institutions holding the Portuguese repositories analyzed, with the goal to continue acting<br />
towards the improvement of their quality.<br />
2. Repositories drivers and barriers<br />
The repositories are online systems intended to archive, preserve, make available and disseminate<br />
the intellectual output of a community, available in a digital space where data and information are<br />
stored and updated (Ramos, 2010; Poulsen et al., 2008 ) or "digital store boxes that host collections<br />
of digital resources in a <strong>learning</strong> object format: i. e. resources that are designed to be integrated,<br />
aggregated and sequential in an efficient way to produce “units of <strong>learning</strong>” that are meaningful to<br />
learners” (Margaryan, 2007: 3).<br />
2.1 Drivers<br />
As there are several potentialities in a repository, we have decided to stress some of those<br />
established by the Organization for Economic Co-Operation and Development (OECD, 2007):<br />
To facilitate the change of teaching practices;<br />
To stimulate more interactive and constructivist teaching practices;<br />
To promote the use of DER as a complement or substitute of teaching in the classroom;<br />
To make collaborative approaches in teaching easier and faster (OECD, 2007; GEPE, 2009);<br />
To induce and streamline the production and usage of tools, content, resources and information<br />
on digital support;<br />
To minimize the digital divide, allowing remote access at low cost to content, modules and<br />
courses;<br />
To enhance the inclusion in teaching and <strong>learning</strong> of people with special needs;<br />
To develop and strengthen a culture of lifelong <strong>learning</strong><br />
2.2 Barriers<br />
Studies identify several barriers that limit the access and use of repositories (Collis, 1995; Davis et al.,<br />
2010; OECD, 2007, Pawlowski and Zimmermann, 2007), such as:<br />
Technical: lack of broadband availability;<br />
101
Cornélia Castro et al.<br />
Economic: lack of resources to invest in hardware and software required to develop and share<br />
DER;<br />
Social: lack of skills to use technical inventions; communication style;<br />
Cultural: resistance to share or use the resources produced by other teachers or other institutions,<br />
cultural perceptions of the teacher and student’s roles;<br />
Political;<br />
Legal: intellectual property and copyrights.<br />
3. Digital Educational Resources repositories<br />
The objects of this study are the three most visible institutional repositories of DER in Portugal, in<br />
primary and secondary education: Portal das Escolas, Casa das Ciências and BOA, which are<br />
compared with the Swedish Lektion and the Irish Scoilnet. The three countries – Ireland, Portugal and<br />
Sweden – are members of EdReNe thematic network (edrene.org, 2011) which was established in<br />
2007 with support from the European Union's eContemplus programme (ec.europa.eu, 2011). The<br />
overall goal of EdReNe "is to improve the provision of and access to <strong>learning</strong> resources" (Poulsen and<br />
Lund, 2009: 86).<br />
3.1 Portugal<br />
3.1.1 Portal das Escolas<br />
The Portal das Escolas, launched on the 22 nd June 2009 and addressed to the educational<br />
community (teachers, students, parents and non-teaching), intends to establish itself as the<br />
institutional platform par excellence in the educational community, through which the Portuguese<br />
teachers can produce and access, anytime and anywhere, a wide range of online services that add<br />
value to the teaching and <strong>learning</strong> process. This repository provides "material and reliable<br />
information", adopting the ME mechanisms for validation of educational resources (ME, 2010).<br />
3.1.2 Casa das Ciências<br />
The Casa das Ciências project was created by the Fundação Calouste Gulbenkian (FCG), a<br />
Portuguese private institution of public utility whose statutory aims are Art, Charity, Science and<br />
Education.<br />
It stands as "an integrator and amplifier vehicle of current efforts in using Information Technology in<br />
teaching and <strong>learning</strong>, made by many different agents and whose results are scattered” (FCG, 2010),<br />
establishing itself as a subject repository. This repository "aims to give visibility and usefulness to the<br />
efforts of many teachers, recognizing the merit they actually have and becoming a website of<br />
reference for all Portuguese Science teachers" (FCG, 2010). All the materials submitted are assessed<br />
by referees from a scientific and educational point of view, in an approach to peer-review. This quality<br />
control of the submitted materials complies with a set of rules stated in a regulation, which considers<br />
the awarding of prizes to the DER considered of higher quality, as a way to encourage the<br />
submission.<br />
3.1.3 BOA<br />
The BOA/VemAprender (INESC-ID, 2010) project is a research initiative sponsored by the Information<br />
Systems Group of INESC-ID at Lisbon with the aim of designing, developing and evaluating the<br />
implementation of collaborative platforms within teaching and <strong>learning</strong> contexts. Its main purpose is<br />
the creation of a web platform to which the educational community of various levels of education can<br />
access. This way it aims to enable everybody to participate collaboratively in building a repository of<br />
<strong>learning</strong> objects, with a significant set of associated information. This initiative intends to foster the<br />
creation of a community of practice that is a facilitator and an enhancer of quality in the teaching<br />
profession. In this repository, when submitting a DER, the author gives a certain value – credits – to<br />
its object. The value initially given is regularly updated according to the number of downloads made,<br />
this being the mechanism that allows the evaluation of the object submitted.<br />
102
3.2 Sweden<br />
Cornélia Castro et al.<br />
3.2.1 Lektion<br />
Lektion, the largest Swedish teacher site, created in 2002, is a repository owned and developed by a<br />
web company named Lektion Sweden AB. This company, located at Helsingborg, took the initiative of<br />
starting the Lektion.se site to connect Swedish teachers in an internet community. Lektion.se is an<br />
idea with suggestions for lessons by teachers, for teachers and the lesson database is referred to be<br />
gigantic, containing thousands of tips and ideas for teaching. This concept of community is favoured<br />
by the fact that the teachers can share ideas about the DER through a forum. There is a Vendor<br />
Archive where teachers can find everything to do relating school and even find a job in the Jobs<br />
Archive.<br />
3.3 Ireland<br />
3.3.1 Scoilnet<br />
Scoilnet is the official education portal of the department of Institutional Education and Science in<br />
Ireland. This repository is responsible for the promotion and use of the internet in education under the<br />
Government Information and Communication Technologies in Schools Programme. Launched in<br />
1998, the website is managed by the National Centre for Technology in Education (NCTE, located at<br />
Dublin City University). Its current interface was launched in 2003 with design tweaks in September<br />
2009 (scoilnet.ie). Scoilnet contains a database of over 11 800 digital resources (theme pages,<br />
reviewed websites, quizzes, lesson plans, crosswords and other multimedia). The content of the<br />
Encyclopaedia Britannica Online and of Word Book Online was specifically authorized for educational<br />
use within the site Scoilnet. Curriculum focused, it acts as a support to teachers, students and<br />
parents. Scoilnet promotes actively the integration of Information and Communication Technologies<br />
(ICT) in teaching and <strong>learning</strong>.<br />
4. Methodology<br />
The type of study followed in this research fits the descriptive and comparative ones. The main<br />
institutional repositories of DER for primary and secondary levels of education in Portugal are<br />
characterized, the key dimensions that affect the quality are assessed and they are compared with<br />
other European repositories. To do this study it was required to previously identify the dimensions to<br />
be evaluated (Vilelas, 2009).<br />
With that purpose, the Guia para la Evaluación de Repositorios Institucionales de Investigación<br />
(GERII), published jointly by the Ministry of Science and Innovation of Spain and other organizations<br />
in the field of Education, Science and Technology, was adapted (Villar et al., 2010). The GERII<br />
includes a series of guidelines for the creation and evaluation of a repository. For that function it<br />
gathers together a set of 31 evaluation criteria, distributed over seven dimensions that any quality<br />
repository must meet and therefore it was considered an excellent base for our analysis.<br />
In the present study, we chose to make some adjustments to the evaluation model followed in GERII.<br />
Thus, we considered five of its seven dimensions and it was decided not to include the dimensions:<br />
Interoperability and Security, Authenticity and Integrity of Data because: i) since they are essentially<br />
technical dimensions, they do not fit in the methodology of data collection defined and ii) it is intended<br />
to perform an evaluation of the repositories only in the perspective of producer and user.<br />
In table 1, and according to the guide GERII, we present the criteria to be considered in any<br />
evaluation of the five dimensions considered in this study:<br />
Table 1: Dimensions to be evaluated<br />
Dimension Evaluation Criteria<br />
Visibility<br />
Visibility in the major search engines; existence of a standard name and a<br />
proper url; presence of incentives for participation and information on the<br />
mode of submission.<br />
103
Cornélia Castro et al.<br />
Dimension Evaluation Criteria<br />
Policies<br />
Disclosure of the mission, goals and function; information on who can submit,<br />
what can be submitted and in what formats; information on documents<br />
archive policies and preservation of content; forms of contact and support.<br />
Legal aspects Information on the intellectual property of DER present in the repository.<br />
Metadata<br />
Statistics<br />
Information on formats used; use of classifiers or definition of a policy of<br />
indexing content.<br />
Mechanisms to record access logs to the server and access, use and<br />
download of DER.<br />
The methodology of data collection was based on the analysis of the repositories websites. Thus,<br />
each repository was classified for each criterion of the five dimensions and the evaluation was made<br />
from the point of view of the producer and user of DER.<br />
5. Results<br />
The study results allowed a global assessment of the quality of the five repositories of DER for<br />
primary and secondary levels of education, from the perspective of the author and the user of DER<br />
and are presented in a comparative summary for each of the dimensions assessed.<br />
The following table 2 summarizes the Visibility dimension results.<br />
Table 2: Dimension: Visibility<br />
Advantages Limitations<br />
Excelent visibility in the search engines.<br />
The website homepage of the Portuguese<br />
Standardized name of repositories (except BOA, institutions makes no reference to the<br />
facilitating its identification).<br />
repository.<br />
Apropriated url.<br />
Lack of public disclosure in the case of BOA<br />
Incentives for participation in the repository.<br />
(limitations not applicable to Lektion).<br />
Clear information on how to submit RED.<br />
In this dimension, there aren’t significant differences between the Portuguese repositories and the foreign<br />
ones.<br />
In a search for repositories, by "denomination", in the three most used search engines (Google, Bing,<br />
Yahoo) (SEO, 2011), they all appear as the first result. The exception is Lektion that in Yahoo<br />
appeared in third place.<br />
The fact that the ME, FCG and INESC-ID homepages do not make reference, in their internal search<br />
engines, to their repositories, may lead the target audience to consider the repositories as not worthy.<br />
This may be evidence that the repositories are not central in the policies of the institutions that hold<br />
them.<br />
In all the analyzed repositories, information on how to submit resources is published and messages<br />
encouraging the participation and collaboration in them are promoted. It is noteworthy the dynamics of<br />
the repositories Lektion and Scoilnet which present a video explaining how to find a resource and/or<br />
how to make a submission. The Scoilnet repository also presents additional information on 10 ways to<br />
use Scoilnet.<br />
The main results of the dimension Policies are summarized in Table 3.<br />
104
Table 3: Dimension: Policies<br />
Cornélia Castro et al.<br />
Advantages Limitations<br />
The mission, goals and functions of the repository are present.<br />
It is established how users can upload the content, what kind of<br />
content is accepted and in what formats.<br />
There is a requirement to register in order to download a<br />
resource (Portal das Escolas and Scoilnet are<br />
exceptions 1 ).<br />
Several ways of contact and support are available.<br />
There isn’t established a policy of<br />
preserving content.<br />
Unlike the Portuguese, the Swedish and Irish repositories have a video demo of their features.<br />
1 Navigation is not subject to registration but some features are only accessible to teachers who are registered users.<br />
It appears that institutions do not declare their commitment to make content available on an ongoing<br />
basis and to take measures for their preservation (as in the case of migration, for example) to ensure<br />
access to them and create and maintain file formats. This condition can contribute to the reduction in<br />
the confidence of authors, inhibiting their collaboration. This less favorable aspect occurred in the<br />
Portuguese repositories as well as in Lektion and Scoilnet.<br />
The Legal Aspects were compared and with regard to compliance with the fundamental requirements<br />
of authors’ rights guarantee, we present a summary in table 4.<br />
Table 4: Dimension: Legal Aspects<br />
Advantages Limitations<br />
Confirmation by the author during submission, that the material<br />
does not violate any intellectual property right (BOA is an<br />
exception and it was not possible to confirm in Scoilnet).<br />
Request for an authorization from the author that allows the<br />
Not identified.<br />
distribution of content (not possible to confirm in Scoilnet).<br />
Existence of information on copyright like Creative Commons or<br />
any other.<br />
Like Lektion, the Portuguese repositories adopt the Creative Commons philosophy in response to the<br />
protection of copyright. Scoilnet refers to Irish and European legislation, not materializing the type of license.<br />
The safeguard policy of intellectual property and copyright encourages the dissemination of ethically<br />
correct behaviour in the production and use of DER.<br />
The enormous amount of DER available in the repositories makes it important that they are described<br />
by metadata (table 5). These allow the recognition of the repositories both by humans and computers,<br />
so that they can be located by using different criteria. This is a capital issue since the resources will<br />
be of little use if they cannot be easily found.<br />
Table 5: Dimension: Metadata<br />
Advantages Limitations<br />
There are standard procedures for indexing DER (indication<br />
of the author, description, topic, subject, school grade,<br />
keywords, target audience, resource type or time).<br />
Not identified.<br />
The metadata allow search by standard criteria.<br />
Metadata facilitate research of DER into the archives of the<br />
repository.<br />
In this dimension, there were no significant differences between the Swedish and Portuguese repositories<br />
(couldn't be confirmed in Scoilnet).<br />
Upon uploading the authors are asked to categorize the resources through a series of standard<br />
criteria such as author, description, topic, subject, school grade, keyword or time, for example.<br />
Similarly, the repositories enable a search of DER by the same criteria.<br />
Finally, the dimension Statistics (table 6) is also important, both from the perspective of the user or<br />
from the author.<br />
When you use a repository, the statistics for a given DER, allow you to realize, for example, which<br />
resources were the most voted, information that can help you on the decision to download a particular<br />
resource, helping to give it visibility. From the author’s point of view, this knowledge will give him<br />
105
Cornélia Castro et al.<br />
feedback on the interest that his own DER created in the community of peers and will encourage him<br />
to produce and share more resources. Knowing the number of visitors to a repository helps the<br />
teacher/user to understand its importance and so he starts seeing it as a tool that he might consider<br />
using to facilitate his work and diversify his practices. The existence of various statistics relating to the<br />
access to the repository is also valuable to the institution that holds the repository, as it provides<br />
information on the dynamics of the repository which is very important to its management.<br />
Table 6: Dimension: Statistics<br />
Advantages Limitations<br />
They report on the number of available and/or published objects.<br />
They refer to the number of downloads and/or visits made.<br />
They indicate the number of visitors (Casa das Ciências and Portal<br />
das Escolas have several historical data).<br />
Not identified.<br />
They present the latest, top rated, most popular and/or most visited<br />
DER.<br />
The authors that share more <strong>learning</strong> objects are highlighted (BOA).<br />
In this dimension, there were no significant differences between repositories, although it was noted that<br />
Scoilnet does not indicate some of the items evaluated in this dimension; in addition, by not admitting the<br />
login, it did not allow the identification of possible limitations.<br />
6. Conclusions<br />
Repositories contribute to the opening of the classroom to the world. The DER online availability<br />
allows them to be accessed from anywhere, anytime and by anyone who has appropriate training and<br />
an Internet connection.<br />
The data presented in table 7, concerning the implementation of each one of the repositories<br />
evaluated in this study, as well as the discussion of the results, allowed us to answer to our initial<br />
research question: “How are the Portuguese institutional repositories, for primary and secondary<br />
education levels, positioned in what concerns international quality criteria?”.<br />
Table 7: Data on the implementation of the evaluated repositories<br />
PORTUGAL SWEDEN IRELAND<br />
Portal das<br />
Escolas<br />
Casa das Ciências BOA Lektion Scoilnet<br />
Inhabitants 10 647 763 1<br />
9 354 462 2<br />
4 470 700 3<br />
Teachers 115 680 4<br />
Registered Users 83 000 7<br />
Published DER 2 620 7<br />
Teachers/registered<br />
teachers<br />
Registered<br />
teachers/published<br />
DER<br />
1,4:1<br />
8 226 7<br />
1 407 7<br />
Not applicable for being a<br />
subject repository<br />
326 7<br />
168 7<br />
355:1<br />
32:1 6:1 2:1<br />
78 400 5 55 052 6<br />
229 123 7 4 000 7<br />
Not available 11 800 7<br />
No possible<br />
evaluation<br />
No answer from<br />
request to<br />
Lektion<br />
14:1<br />
1 Estimate, National Statistical Institute, February, 2011; 2 2009, most recent census; 3 estimate<br />
2010, CSO; 4 GEPE, ME, 2009; 5 2003; 6 2007; 7 Data repositories, May, 2011.<br />
The data shows that Portugal is, as far as population size and number of teachers is concerned,<br />
closer to Sweden than to Ireland. Although Ireland is the smallest country in terms of population and<br />
number of teachers, it has the highest number of published DER. The reasons for that might be: i) the<br />
fact that the repository is quite user-friendly, allowing the teacher to file his preferences in the tab My<br />
stuff, ii) it is possible to submit any resource without peer-review (the teacher is the sole responsible<br />
for the material he submits and he can leave it in the repository for as long as he wants or take it off<br />
when he wants) and iii) the age of the repository.<br />
The number of registered users in the Portuguese repositories is substantially smaller than in the two<br />
foreigner ones. This probably happens because of the youth of our repositories. However it is<br />
106<br />
1:3
Cornélia Castro et al.<br />
noticeable that there has been an effort to keep increasing levels of quality since their creation. The<br />
ME Portal das Escolas requires the registration of teachers so that they can be certified in terms of<br />
ICT skills, which possibly explains the ratio 1,4:1.<br />
Taking into consideration the few limitations detected for the Portuguese repositories in the<br />
dimensions evaluated on the basis of GERII, it appears that the solution of some of them<br />
(unavailability of broadband and the lack of resources to invest in hardware and software, for<br />
example) does not depend directly from the institutions that control them and others (the institutions<br />
homepages that make no reference to their repositories or the lack of definition of a public policy of<br />
preserving data) can be easily minimized.<br />
The Portal das Escolas has recently launched (March 10 th , 2011) an area of educational blogs and on<br />
May 30 th , 2011 the Casa das Ciências launched a bank of images and the first scientific encyclopedia<br />
written in Portuguese: the WikiCiências with a collaborative basis and constantly quotable on the<br />
Internet. We should highlight the uniqueness of this repository which, being related to a subject, is<br />
totally different from any of the two European repositories analyzed. For the sustainability of DER<br />
repositories in Portugal it will be important to foster communities of practice of which the Casa das<br />
Ciências seems to be the most paradigmatic one.<br />
As a result of this comparative study, it was found that the foreign repositories meet certain<br />
requirements not found in the Portuguese ones. Thus, we present the following proposals for<br />
inclusion in the Portuguese repositories:<br />
Video demonstration on the process of submitting a resource and/or all the repository functions;<br />
Different tabs with information for parents and students;<br />
Forum for sharing ideas among users;<br />
Diversify the user generated content like blogs, wikis, twitter and podcasts in a logic of Web 2.0<br />
technology;<br />
Engage in the provision of a web hosting service to all schools (in Portal das Escolas). This would<br />
enable schools to easily develop and host their website on a school’s broad band network and to<br />
assume as a repository that actively promotes the integration of ICT in teaching and <strong>learning</strong>,<br />
under the Portuguese PTE.<br />
Navigation in the five analyzed repositories could only be made in the language of the country of<br />
origin. It would be important to consider the development of interfaces, at least bilingual, in the<br />
language of the country of origin and in English. The same applies to metadata. We consider this to<br />
be a relevant step since in the European Union there are several projects of mobility of both teachers<br />
and students and exchange programmes between schools. This would also enable all teachers to<br />
access and adapt DER from other countries other than their own, thus creating a European network<br />
of repositories.<br />
The comparative study held, whose goal was to verify how the Portuguese institutional repositories, in<br />
lower and middle education, are positioned in what concerns international quality criteria, allowed us<br />
to conclude that the Portuguese repositories are facilitators of authors collaboration and general<br />
users’ access. They also contribute to promoting the pursuit of new practices in the classroom by<br />
teachers in primary and secondary schools in Portugal, as they make accessible DER with scientific<br />
and teaching quality.<br />
In short, it seems that the Portuguese experience, in spite of being a recent one, when compared with<br />
other European experiences, is on the right track to achieve educational innovation. Repositories<br />
focused on people, on the community of education stakeholders and on teaching practices will help to<br />
overcome individual and organizational inertia and other barriers, towards the success among its<br />
users.<br />
References<br />
Dumitru et al. (2009) “D1.2 European State-of-Art-Report”, [online], Lifelong Learning Programme, Education and<br />
Culture DG,<br />
http://www.ea.gr/ep/metaschool/deliverables/Metaschool_WP1_EuropeanStateofArtReport_V1.0_30Apr200<br />
9_EUN.pdf (acessed on 20/01/10).<br />
107
Cornélia Castro et al.<br />
EdReNe (2009) “State of the art – II Educational Repositories in Europe”, [online] eContentplus,<br />
www.edrene.org/results/deliverables/EdReNe D 26 SoA-II.pdf (accessed on 20/01/10).<br />
EdReNe (2011) “EdReNe”, [online], http://edrene.org/ (acessed on 20/05/2011).<br />
European Commission – CIP (2011). “Competitiveness and Innovation Framework Programme (CIP)”, [online],<br />
http://ec.europa.eu/cip/index_en.htm (acessed on 20/05/2011).<br />
Fundação Calouste Gulbenkian - FCG, "Casa das Ciências", [online],<br />
http://www.casadasciencias.org/index.php?option=com_content&view=article&id=16&Itemid=46&menu=2&i<br />
ntro=1 (acessed on 07/12/10).<br />
GEPE (2009), Portal das Escolas. Estudo de Implementação, ME, GEPE, Lisboa.<br />
Grupo de Sistemas de Informação do INESC-ID (2010, Dez, 17), [online], "Bolsa de Objectos de Aprendizagem,<br />
BOA", http://www.vemaprender.net/Acerca@12.aspx (acessed on 17/12/10).<br />
Johnson, L., Levine A., Smith, R. and Stone S. (2010) The 2010 Horizon Report, Austin, Texas, The New Media<br />
Consortium, USA.<br />
Margaryan, A., Milligan, C. and Douglas, P. (2007) “Structured Guidelines for setting up Learning Object<br />
Repositories”, [online], CDLOR, http://www.academy.gcal.ac.uk/cd-lor/documents/CD-<br />
LOR_Structured_Guidelines_v1p0_001.pdf (accessed on 17/05/11).<br />
Lektion.se (2011) “Lektion.se”, [online], http://www.lektion.se/index.php (acessed on 20/05/2011).<br />
Milligan C. (2007) “CD-LOR Final Project”, [online], http://www.academy.gcal.ac.uk/cd-lor/documents/CD-<br />
LOR_Final_Report_v1p0.pdf (accessed on 17/05/11).<br />
Ministério da Educação da República Portuguesa - ME, "Portal das Escolas", [online],<br />
https://www.portaldasescolas.pt/portal/server.pt/community/p%C3%A1ginas/243/perguntas_frequentes/152<br />
12#Funcionalidades%20do%20Portal%20das%20Escolas (acessed on 07/12/10).<br />
OECD (2007) Giving Knowledge for Free. The Emergence of Open Educational Resources, Paris. France.<br />
Pawlowski J. and Zimmermann V. (2007) Open Content: A Concept for the Future of ELearning and Knowledge<br />
Management, in Conference Proceedings Knowtech.<br />
Poulsen, L. J., et al. (2008) “State of the art – I Educational Repositories in Europe”, [online], eContentplus,<br />
http://edrene.org/results/.../EdReNe%20D%202.5%20SoA%20-%20I.pdf (acessed on 20/01/10).<br />
Poulsen, L. H. and Lund, T. B. (2009) “EdReNe – Educational Repositories Network”, Vol. 12, No. 1, pp 85-91.<br />
Ramos, J. L., Teodoro, V. D., Fernandes, J. P. S., Ferreira, F. M. and Chagas, I. (2010) Portal das Escolas:<br />
Recursos Digitais para Portugal. Estudo Estratégico, ME, GEPE, Lisboa.<br />
Scoilnet (2011). “Scoilnet Portal for Irish Education”, [online], http://www.scoilnet.ie/Default.aspx (acessed on<br />
20/05/2011).<br />
SEO Consultants Directory (2010) "Top search engines for 2010", [online], http://www.seoconsultants.com<br />
(acessed on 18/05/11).<br />
Vilelas J., (2009) Investigação – O processo de construção do conhecimento, Edições Sílabo, Lisboa.<br />
Villar C. A., et al. (2010) "Guía para Evaluación de Repositorios Institucionales de Investigación", [online],<br />
Ministério da Ciencia e Innovación de España, FECYT, Recolecta e CRUE,<br />
www.acessoaberto.usp.br/guiavaliacaorepositorios2011/ (accessed on 11/02/11).<br />
108
eLearning: Roles in Distance Tertiary Education<br />
Ivana Cechova, Dana Zerzanova and Jana Berankova<br />
University of Defence, Brno, Czech Republic<br />
ivanacechova@unob.cz<br />
dana.zerzanova@unob.cz<br />
jana.berankova@unob.cz<br />
Abstract: Information and communication technologies (ICT) have undergone a rapid development in<br />
accordance with advances in the society. This state has to be reflected in the teaching process, while teaching<br />
languages is no exception. Language training at the University of Defence (UoD) is carried out on both a full-time<br />
as well as a distance study basis. E-<strong>learning</strong> is an integral part of both forms of language study programmes. The<br />
possibilities of students’ involvement into the process of creating study supports, especially for teaching and<br />
<strong>learning</strong> specific terminology were employed to determine the influence of technology-supported materials on the<br />
students’ language <strong>learning</strong>. The co-operation with students enables their more active participation in study<br />
material preparation and during the lesson itself, the content of which responds to the needs of particular<br />
specialisations. This approach opens a different perspective to the teachers who prepare the study materials for<br />
teaching special terminology in co-operation with individual departments of the faculty. The students’ involvement<br />
motivates both sides of the educational process and influences them qualitatively. The basis for the research<br />
were semi-structured interviews the aim of which was to find out how Information and Communication<br />
Technologies (ICT) help the distance students in their study, how distance students assess ICT in their language<br />
training and especially how the students assess LMS Barborka. The second part of the research was a<br />
questionnaire survey aimed at creating a distance learner profile dealing with technology usage for language<br />
<strong>learning</strong>. We also wanted to find out if our distance students prefer to: use up-to-date technologies, the Internet,<br />
an in-house Study Portal and on-line sessions in language lessons be taught in a classical way (without computer<br />
technology programs). The paper describes our research, the methods used and our respective findings and<br />
suggests implications of the research for a broad range of educational fields. It will be of interest to stakeholders<br />
in education and those who seek to be informed users and or promoters of computer technology for <strong>learning</strong><br />
purposes.<br />
Keywords: distance <strong>learning</strong>, LMS Barborka, students’ participation, electronic study material<br />
1. Distance education<br />
“Foreign language is crucial to a nation’s economic competitiveness and national<br />
security. Multilingualism enhances cognitive and social growth, competitiveness in the<br />
global marketplace…, national security, and understanding of diverse people and<br />
cultures. As we approach a new century where global communication will be essential for<br />
survival, we cannot afford the luxury of international ignorance…”– The United States<br />
Congress, Foreign Language Assistance Act of 1994<br />
Advances in information and communication technologies (ICT), coupled with changes in society,<br />
have had a tremendous impact on educational systems. This provides an opportunity to develop new<br />
flexible <strong>learning</strong> environments to facilitate the <strong>learning</strong> process by providing more efficient ways of<br />
<strong>learning</strong> and teaching. Many learners, teachers, stakeholders and institutions are coming to distance<br />
education as a tool allowing access to education to a number of those adult students for whom the<br />
full-time study is not possible for many different reasons (e.g. time, distance or financial constraints).<br />
Distance education aims at delivering a quality tertiary education to students who are not able to be<br />
physically present on campus. Although the label "distance <strong>learning</strong>" could be applied to any situation<br />
where students are <strong>learning</strong> at remote sites, the term is normally restricted to teaching via satellite or<br />
other long-distance telecommunication technologies. Shelley defines distance education as: “an<br />
educational system in which learners can study in a flexible manner in their own time, at the pace of<br />
their choice and without requiring face-to-face contact with a teacher” (Shelley 2000). According to<br />
Cassell distance <strong>learning</strong> is "a field of education focusing on the andragogy and pedagogy,<br />
instructional systems, and technology which endeavor to deliver an education to students who are not<br />
physically in a classroom or campus setting" (Cassel, 2010). However, White emphasizes the<br />
following starting points for a definition of distance education: most begin with structural concerns, a<br />
few begin with pedagogical concerns and fewer begin with learner-based perspectives.<br />
Organisational or structural concern includes:<br />
The separation of teacher and learner in time and/or place.<br />
109
Ivana Cechova et al.<br />
The influence of an educational organisation, especially in preparing, delivering and providing<br />
support materials.<br />
The use of a range of media, such as print, audio, video and computer-based applications, CD or<br />
MP3 recordings or Webcasts.<br />
The use of communication devices to facilitate an effective communication (Internet or corporate<br />
Intranet).<br />
The possibility of face-to-face contact usually in the form of tutorials and courses (online courses,<br />
web-conference courses).<br />
The provision of a range of support services, which may include opportunities for interaction and<br />
response to individual learners, as well as guides (White, 2003).<br />
Pedagogical concerns comprise the extent to which “written teaching dominates in contrast to spoken<br />
teaching and so <strong>learning</strong> by reading is stressed rather than <strong>learning</strong> by listening” Peters (1998). This<br />
has a significant impact on the development of oral and aural skills as well as interactive skills in<br />
distance language <strong>learning</strong>.<br />
The learner concern includes the characteristics that a learner brings to <strong>learning</strong>. This includes the<br />
learners’ beliefs, attitudes, experiences and any prior knowledge how to learn.<br />
Traditional forms of distance <strong>learning</strong> that used print materials in combinations with materials on CD<br />
ROMs are being replaced by opportunities for collaboration and interaction online (Cohen and Lippert<br />
1999). In the case of distance language <strong>learning</strong>, course designers must consider not only how to<br />
help learners gain communicative competence but also language learner strategies that support<br />
success.<br />
According to Wohlert the keys to successful distance <strong>learning</strong> are:<br />
Live interaction between the instructor and the students during the course.<br />
The presence of a classroom teacher in the remote sites who is involved in the <strong>learning</strong> process.<br />
In some cases, these cooperating or coordinating teachers are studying the language along with<br />
the students with the intention of completing a teaching minor.<br />
The regular use of other media, such as computers, speech recognition devices, audiotapes, and<br />
workbooks in a comprehensive approach to distance <strong>learning</strong>.<br />
The involvement and support of school administrators.<br />
The use of electronic mailboxes (which all students and cooperating site teachers use), or a tollfree<br />
phone number with recording machine (Wohlert 1989).<br />
Grubb adds that successful distance <strong>learning</strong> is impossible without honesty, responsiveness,<br />
relevance, respect, openness and empowerment (Grubb. 2010).<br />
Distance education allows a student to arrange the study around work and family commitments.<br />
Understanding of the subject can be enhanced by access to online lessons, CDROMs, DVDs, text<br />
books and other reference sources. Support and communication between a teacher and students is<br />
possible via email, online sessions and telephone and provides web-based facilities which allow a<br />
student to send questions, submit assignments, check the grades and access other resources, 24<br />
hours a day, seven days a week.<br />
2. Distance language <strong>learning</strong><br />
Distance language <strong>learning</strong> includes a lot of elements and practices ranging from print-based<br />
correspondence courses to online courses with extensive opportunities for interaction, feedback,<br />
support between a teacher and learners as well as support among learners themselves (White 2003).<br />
Distance language <strong>learning</strong> consists of synchronous and asynchronous <strong>learning</strong>. Asynchronous<br />
distance language <strong>learning</strong> is commonly facilitated by media such as eMail and discussion boards,<br />
thus supporting work relations among learners and learners with teachers, even when participants<br />
cannot be online at the same time. It is thus a key component of flexible e-<strong>learning</strong>. In fact, many<br />
people take online courses because of their asynchronous nature, combining education with work,<br />
family, and other commitments. Asynchronous e-<strong>learning</strong> makes it possible for learners to log on to<br />
an e-<strong>learning</strong> environment at any time and download documents or send messages to teachers or<br />
110
Ivana Cechova et al.<br />
peers. Students may spend more time refining their contributions, which are generally considered<br />
more thoughtful compared to synchronous communication (Hrastinski 2008). Synchronous <strong>learning</strong>,<br />
commonly supported by media such as videoconferencing and chat, has the potential to support<br />
learners in the development of <strong>learning</strong> communities. Learners and teachers experience synchronous<br />
<strong>learning</strong> as more social and avoid frustration by asking and answering questions in real time.<br />
Synchronous sessions help learners feel like participants rather than “isolates”.<br />
The debate about the benefits and limitations of asynchronous and synchronous <strong>learning</strong> seems to<br />
have left the initial stage, in which researchers tried to determine the medium that works “better”—<br />
such studies generally yielded no significant differences (Gunawardena, McIsaac 2004). According to<br />
Hrastinski instead of trying to determine the best medium, the e-<strong>learning</strong> community needs an<br />
understanding of when, why, and how to use different types of e-<strong>learning</strong>. Note also that the users<br />
decide how to use a medium (Hrastinski 2008). For example, in some instances eMail is used nearsynchronously<br />
when users remain logged in and monitor their eMail continuously. Thus, the<br />
difference between asynchronous and synchronous e-<strong>learning</strong> is often a matter of degree. A solution<br />
is offered by Mason, who uses the new term multi-synchronous <strong>learning</strong>. The term multi-synchronous<br />
<strong>learning</strong> refers to the combination of both synchronous and asynchronous <strong>learning</strong> with the emphasis<br />
on advantages of both of them. Multi synchronous <strong>learning</strong> may include the following elements:<br />
Web-based conferencing/chats to hold online lessons<br />
eMail to send messages, homework and feedback and for announcements<br />
Face-to-face lessons/meetings to start and finish a course (Mason 1998b, Cassell, 2011).<br />
However, language teachers must be fully aware of the following facts: who their students are, what<br />
their needs and wants are, and what their preferences in the language training and their motivation<br />
might be.<br />
3. LMS Barborka<br />
The teachers at the Language Training Centre (LTC) of the University of Defence (UoD) are aware of<br />
the contemporary trends in distance language <strong>learning</strong> and the involvement of ICT in teaching. We<br />
have implemented the Learning Management System (LMS) Barborka into the managing and<br />
teaching of distance study programme students. LMS Barborka allows the creation, administration,<br />
and use of the study supports included, which are distributed by the Internet or on the intranet of the<br />
UoD. In addition to tools for the creation, administration and distribution of courses, LMS contains<br />
tools for communication both among students and between students and the teacher. It consists of<br />
the following four basic subsystems:<br />
Author – it serves for designing appropriate study supports, which include texts with various<br />
pictures, animations, video sequences, or sounds to give the students structured material as<br />
comprehensible and clear as possible.<br />
Student – this subsystem enables learners to go through the texts, communicate with other<br />
students and teachers/tutors via the Barborka mail system. Students can use electronic study<br />
supports at any time and place if there is Internet access available. It is also possible to print the<br />
materials and study without Internet access.<br />
Tutor – this subsystem enables the managing of the teaching process; it serves for administration<br />
of students and courses, assigning homework, giving basic information to students about the<br />
study course, and communicating with them.<br />
Administrator – it enables the registering of courses, students and their results, it also records<br />
study groups and teachers.<br />
Electronic study supports deal with courses of general English, professional English – military,<br />
economic and management topics and vocabulary, and for the higher grades also specific vocabulary<br />
of their specializations. The electronic supports are supplemented with exercises both for reflection as<br />
well as those which enable feedback, and thus allow monitoring of the comprehension of the studied<br />
materials, either in communication with the tutor, or with independent work. Instantaneous feedback is<br />
ensured by the interactive exercises created by the ‘Hot Potatoes’ program. Study supports are<br />
appreciated by students because they help them to prepare both for the final tertiary exams as well as<br />
for the STANAG 6001 (SLP 2-3) standardized exams.<br />
111
4. Methodology<br />
Ivana Cechova et al.<br />
In academic year 2010/11 we designed a research study the task of which was to find out if and how<br />
ICT help the distance students in their study, how distance students assess ICT in their language<br />
training and especially how the students assess LMS Barborka. For the first step of our research we<br />
chose the method of interviews which were carried out via Skype. All respondents were our distance<br />
students and graduates. The interviews were semi-structured, using an 'interview plus' approach to<br />
enhance discussion (Bloom 1953; JISC 2009). The interview questions focused on four main themes:<br />
likes and dislikes dealing with English study; language <strong>learning</strong> and ICT; LMS Barborka as a tool for<br />
language <strong>learning</strong>; LMS Barborka and its evaluation from the students’ point of view. Ten interviews<br />
were undertaken, with each lasting approximately one hour. Data were fully transcribed and entered<br />
into INVivo. Initial coding was inductive (Thomas 2006) and categories that reflected the question<br />
schedule and main trends were developed. The coded data were then reviewed by the research team<br />
and a series of key themes were determined by consensus. The data were then re-coded according<br />
to the agreed themes, and through this iterative process the project's findings emerged (Strauss,<br />
Corbin 2004). We tried to achieve answers to the following research questions:<br />
What are the students’ likes and dislikes dealing with English?<br />
How do the students evaluate LMS Barborka as a tool for studying English and how do they<br />
evaluate the study supports presented in LMS Barborka?<br />
What are the advantages and disadvantages of ICT in language <strong>learning</strong>?<br />
The second part of the research is a questionnaire aimed at creating a distance learner profile by<br />
establishing baseline data on technology use. The questions elicited prior experience and<br />
expectations of using technology for <strong>learning</strong>, and the impacts of e-<strong>learning</strong>. Then we wanted to find<br />
out if our distance students prefer to use up-to-date technologies (the Internet, an in-house Study<br />
Portal and on-line sessions in language lessons) or to be taught in a classical way (without computer<br />
technology programs), furthermore we wanted to know the students’ preferences and needs while<br />
using ICT in the language classes and the LMS Barborka utility. We wanted either to confirm or reject<br />
the following hypothesis:<br />
The students prefer the use of up-to-date technologies to a classical way of <strong>learning</strong>.<br />
ICT can influence English training in a positive way.<br />
Study materials and communication via LMS Barborka can influence English training in a positive<br />
way.<br />
The respondents were distance students of the University of Defence, the Faculty of Economics and<br />
Management.<br />
5. Findings<br />
In some answers the opinions of distance study students do not differ from those of full-time ones<br />
(research carried out in 2004-2009). All students highly appreciate the access to LMS Barborka,<br />
which offers interesting texts from different economic and military areas full of various specialized<br />
terminology. They like most especially those texts which differ from commercial textbooks and the<br />
Internet materials. In this context they often express a demand for topics regarding their<br />
specializations or military-political ones.<br />
“Topics dealing with military and terrorism, as I can’t find them anywhere else.” (R4,<br />
March 2011)<br />
“Study supports for individual topics, they are well arranged and written in readable<br />
English.” (R1, May 2011)<br />
Students asses positively the communication with their tutors, tutors’ fast response to the questions,<br />
the correction and evaluation of their homework and they also appreciate the possibility of electronic<br />
testing, mainly in terms of the necessity to pass the STANAG 6001 exam.<br />
“The evaluation of homework was a big surprise for me, it is accomplished at a high<br />
technical level.” (R9, April 2011)<br />
“The study materials enable to revise, possibly to deepen knowledge of the topics. They<br />
are useful for the STANAG 6001 exam preparation as well.”(R10, April 2011)<br />
112
Ivana Cechova et al.<br />
Using ICT while studying language the students see as the necessity, and moreover it breathes fresh<br />
life into teaching. The choice of the appropriate tool is connected with the activities which it facilitates.<br />
They use the Internet for eMails, while working with texts they use a dictionary, they download study<br />
materials from web pages and they keep in touch with their peers. They appreciate electronic<br />
dictionaries, electronic testing and interactive forms of most study materials. The possibility to choose<br />
the place, time and pace of study plays a very important role when choosing the ICT tool.<br />
“In the electronic dictionary you look everything up much quicker than in the paper one.<br />
There are lots of teaching programs.”(R2, March 2011)<br />
“In my job I don’t use English very often; it means that ICT is the only means how to<br />
brush up the language (at least dealing with listening and writing).” (R1, May 2011)<br />
Studying English with the help of ICT is assessed positively by 7 out of 10 respondents. One<br />
respondent is negative about anything which is connected with e-<strong>learning</strong>; another one prefers the<br />
classical teaching method, i.e. face-to-face one.<br />
“Studying English with the help of Barborka is the new way of study for me; no doubt it is<br />
more up-to-date and comfortable. Textbooks are not bad. Nevertheless, I prefer<br />
ICT.”(R1, May 2011)<br />
“I appreciate assigning and evaluating tasks in electronic form. It is quicker and more<br />
comfortable for me.”(R2, March 2011)<br />
“I prefer teaching in a classical way.” (R4, March 2011)<br />
Students underline the study of English as a prerequisite of their own personal development and<br />
future practice as well. Acquiring the language is understood as a necessity for the future and a<br />
means of promotion at a job.<br />
“In the case of mastering the language, there are possibilities which the knowledge offers<br />
(job - career ladder, etc.)” (R1, May 2011)<br />
“English is nowadays used in a wide range of areas.” (R7, April 2011)<br />
While studying English, the students appreciate mainly the variety, interestingness and topicality; on<br />
the other hand the least attractive activity is studying grammar and vocabulary, mainly idioms and<br />
phrasal verbs<br />
“Present perfect and past perfect tenses are difficult for me.“(R3, April 2011)<br />
“Some texts are demanding in terms of vocabulary.”(R9, May 2011)<br />
According to the analysis of the students’ interviews it is possible to state that distance students can<br />
see ICT/LMS Barborka as a tool which facilitates their study and at the same time as a simpler form of<br />
teaching. They like especially the chance to study any time/anywhere and the offer of unique<br />
materials, which are accessible only in this application (LMS Barborka). On the other hand they are<br />
fully aware of some limitations, e.g. the lack of communication and listening exercises. They focus<br />
repeatedly on the increase of military and management materials accompanied with suitable<br />
exercises and glossary, which can be emphasized as the most important information from all<br />
interviews.<br />
6. Recommendations<br />
The findings of the research suggest the following to improve the students' use of ICT/LMS Barborka<br />
for maximum educational benefit:<br />
Involve students in the creation of military materials. This will help teachers and students to<br />
cooperate closely and to use the students’ special military knowledge.<br />
Encourage students to use web-based support by clearer signposting of online tutorials, guidance<br />
and discussion boards. This will help to deepen comprehension.<br />
Enhance students' awareness and understanding of meta-<strong>learning</strong> strategies in an online<br />
environment (and elsewhere).<br />
Motivate students to search the Internet for updated reading and listening materials (news, video,<br />
radio reports, written briefings, etc,) and share them with other students as well as teachers.<br />
Reward Internet-based research skills within assessment, ensuring that students use critical<br />
thinking skills to evaluate Internet resources.<br />
113
Ivana Cechova et al.<br />
Although this research was undertaken with distance students, the recommendations suggest several<br />
possibilities that could include other student groups. The research determines the strengths and<br />
weaknesses of ICT/LMS Barborka in language study and identifies the specific skills which distance<br />
students possess, enabling them to use technology effectively for <strong>learning</strong> English. However, the<br />
fundamental question remains: How to achieve balanced communicative competence of distance<br />
students and how to motivate them to practice all skills (speaking, listening comprehension, writing<br />
and reading comprehension) via the Internet in order to feel confident in English?<br />
7. Material development<br />
The research findings show that the students predominantly miss integrated file of materials of military<br />
and management terminology with easy access. First we prepared authentic reading materials<br />
according to students needs. In the process of choice we have cooperated with teachers from<br />
specialized departments and methodologically processed them. We supplemented interactive<br />
exercises for vocabulary practice and comprehension check to provide students with appropriate<br />
feedback. This was only the first step. The following one in accordance with constructivist theory is to<br />
enable students’ active participation in the lessons. It is reflected by their own choice of vocabulary<br />
areas needed for their future specializations. Students of the 1st form of the master study programme<br />
were asked to use their knowledge of specialized military subjects as well as military training and help<br />
us with the creation of study supports for the courses of Military English and English for Management.<br />
They agreed with great enthusiasm because they can utilize the gained information and the<br />
knowledge from practical military training. In addition the chosen students have a very high level of<br />
English (Stanag SLP 3), they are interested in the military and have access to a plenty of authentic<br />
literature. Teachers and these students co-operate on a methodological processing of the authentic<br />
texts to tailor them for use in e-courses and a future e-book. The students’ involvement motivates<br />
both sides of the educational process and influences them qualitatively. Teachers profit from students’<br />
specialized updated knowledge of the topics; students combine their knowledge from specialized<br />
subjects with English and ICT, and thus their knowledge becomes complex. From the pedagogical<br />
point of view, teachers and students become active partners in the educational process, so they both<br />
share the responsibility for students’ study, which is one of the basic goals of tertiary education<br />
(Berankova et al. 2011). In Figures 1, 2, and 3 there are examples of students’ involvement in<br />
creating study supports.<br />
Figure 1: LMS Barborka – military text<br />
114
Figure 2: LMS Barborka – page with exercises<br />
Figure 3: LMS Barborka - Exercise<br />
Ivana Cechova et al.<br />
115
8. Conclusion<br />
Ivana Cechova et al.<br />
This paper has documented the experience of a group of distance students. The findings demonstrate<br />
that the majority of students in this study are able to use technology to meet their own <strong>learning</strong> needs.<br />
The distance students assessing their <strong>learning</strong> opportunities often mention quality, speed, diversity,<br />
timeliness, clarity, transparency, efficiency, openness, frequency to achieve their <strong>learning</strong> and<br />
professional goals.<br />
It is clear that the teaching of foreign languages presents special instructional challenges. The<br />
teachers and researchers at the LTC of the UoD continue to address such issues as the development<br />
of communicative competence, as well as participation and interaction in the distance <strong>learning</strong><br />
environment. However, current trends within the distance <strong>learning</strong> may affect many decisions,<br />
processes and emphasis within the distance <strong>learning</strong> environment.<br />
For further development of the monitored area it is recommended to carry out an additional research<br />
to prove the above mentioned findings and to find out if the achieved information is also valid outside<br />
the sample of the UoD respondents. The teachers at the LTC of the UoD are fully aware of this<br />
issue´s complexity and are trying to implement distance <strong>learning</strong> strategies sensitively, bearing in<br />
mind Plato’s quotation: “Someday, in the distant future, our grandchildren’s grandchildren will develop<br />
a new equivalent of our classrooms. They will spend many hours in front of boxes with fires glowing<br />
within. May they have the wisdom to know the difference between light and knowledge.”<br />
References<br />
Berankova, J. Cechova, I. and Zerzanova, D. (2011) Contemporary trends in language training - practical<br />
implementation of ICT in tertiary education. Distance, Learning, Simulation and Communication 2011. Brno,<br />
pp. 36 -41.<br />
Bloom, B.S. (1953) ‘Thought processes in lectures and discussions’, Journal of General Education, vol. 7, pp. 55-<br />
71.<br />
Cassell D. Distance Learning. (2010) [online]. Available at: [Accessed 3<br />
October 2011].<br />
Davis, J.N. (1988) Distance education and foreign language education: Towards a coherent approach, Foreign<br />
Language Annals, vol. 21 (6), pp 547-550.<br />
Grubb A. Building Cyber-Learning Communities in WebCT. [online]. Available at:<br />
[Accessed 1 October<br />
2011].<br />
Gunawardena Ch. N., McIsaac, M. S. (2004) Distance Education. In: David H. Jonassen, ed. Handbook of<br />
Research on Educational Communications and Technology. Mahwah, NJ: Lawrence Erlbaum, pp. 355–395.<br />
Hrastinski, S. (2008) Asynchronous and Synchronous e-<strong>learning</strong>. [online]. Available at:<br />
[Accessed 6 June 2011].<br />
JISC (2009) JISC learner experience phase 2: Interview plus. [online]. Available at:<br />
[Accessed 3 June 2011].<br />
Mason, R. (1998b) Models of online courses. Journal of Asynchronous Learning networks 2 Available at:<br />
[Accessed 30 May 2011].<br />
Peters, B.G. (1998) Comparative politics: Theory and methods. New York University Press.<br />
Shelley, M.A. (2000) Distance education. In M. S. Byram, ed. Encyclopedia of Language teaching and Learning.<br />
London: Routledge, pp. 183-185.<br />
Strauss, A. and Corbin J. (2004) Basics of Qualitative Research: Grounded Theory Procedures and techniques.<br />
Newbury Park: California Sage.<br />
Thomas, R. (2006) ‘A general inductive approach for analysing qualitative evaluation data’, American Journal of<br />
Evaluation, vol. 27 (2), pp. 237-246.<br />
Thinkexist.com (2008) [online]. Available at: <br />
[Accessed 27 May 2011].<br />
White, C. (2003) Language Learning in Distance Education. Cambridge University Press.<br />
Winter, J. et.al. (2010) ‘Effective e-<strong>learning</strong>? Multi-tasking distractions and boundary management by graduate<br />
students in an online environment’. Research in Learning Technology, March 18 (1), pp 71 – 83.<br />
Wohlert, H. S. (1989) German by satellite: Technology-enhanced distance <strong>learning</strong>. In: W.F. Smith, ed. Modern<br />
technology in foreign language education: Applications and projects. Lincolnwood, IL: National Textbook<br />
Co, pp 107-115.<br />
116
Independent Learning in Need or in Crisis? Independent<br />
Learning Under the new Four-Year Undergraduate<br />
Curriculum in Hong Kong<br />
Yin Ha Vivian Chan, Delian Dawn Gaskell, Mei Ah Tan and Lip Yan Felix Chao<br />
The Chinese University of Hong Kong, China<br />
chanyh@cuhk.edu.hk<br />
b123499@mailserv.cuhk.edu.hk<br />
meiahtan@cuhk.edu.hk<br />
felixchao@cuhk.edu.hk<br />
Abstract: Accompanied with the rapid development of technology, independent <strong>learning</strong>, especially in the format<br />
of multi-media, has inspired a great deal of enthusiasm and energy in academia over the past few decades. This is<br />
certainly the case in Hong Kong since the number of university students has continued to increase dramatically.<br />
One particular point that has placed Hong Kong in the spotlight is that there will soon be a complete switch from<br />
the previous 3-2-2-3 education curriculum to the 3-3-4 curriculum in 2012. Under this new system, university<br />
students will be required to spend four years rather than three to obtain an undergraduate degree. In response to<br />
this, Hong Kong teachers and administrators have been propelled to reconsider the role self-access centres<br />
should play in this situation. The eight tertiary institutions currently funded by the University Grants Committee<br />
have all been running language centre or self-access centre, sometimes both. The majority of these centres<br />
provide language <strong>learning</strong> resources and language support, some of them are directly under language centres.<br />
The Independent Learning Centre (ILC) of the Chinese University of Hong Kong is a special case. It serves more<br />
or less as a standalone unit, and there have been instructions from higher authorities to broaden its scope from<br />
language to subject-related matters. The ultimate goal is to create an online platform that contains comprehensive<br />
teaching/<strong>learning</strong> materials on various disciplines for students to study on their own. This paper uses the ILC as a<br />
point of investigation. Through discussing the challenges and difficulties it faces in the process of transition, this<br />
study explores the role self-access centres should play, including what is practical and reasonable, and what is<br />
theoretical and idealistic. It begins with a review of the concept of independent <strong>learning</strong> and the ideology behind it,<br />
followed by a discussion of autonomous <strong>learning</strong> in Hong Kong and the functioning of the self-access centres,<br />
proceeds with the challenges the ILC faces, and concludes with possible solutions to these challenges in the face<br />
of the upcoming four-year curriculum. It is hoped that this research can shed light on what independent <strong>learning</strong><br />
means, if technology is the ultimate solution to budget strain, and how self-access centres should perhaps function<br />
with the benefits of students in mind.<br />
Keywords: independent <strong>learning</strong>, autonomy, education reform, financial imperatives<br />
1. Independent <strong>learning</strong>: A contested concept<br />
Independent <strong>learning</strong> has inspired a great deal of enthusiasm and taken up a large amount of energy in<br />
academia in the past few decades. In Hong Kong, this has been driven by a combination of the fact that<br />
an increasing number of students have been admitted to local universities, while the development of<br />
technology has become ever more influential in local education. However, independent <strong>learning</strong> is a<br />
thorny concept because though it is not new, there is no universal understanding of what exactly it<br />
means. In fact, researchers such as Broad (2006) list several major definitions. The earliest may be<br />
Moore’s (1984: 27) “broad and accessible” definition from the Open University: “working with<br />
increasingly less structured teaching materials and with less reliance on traditional kinds of tutor<br />
support”, which is understandable when considering the nature of adult education. There are also the<br />
social factors behind the surge of independent <strong>learning</strong>. Instead of pointing to the structure and setting,<br />
Kesten (1987) emphasizes the autonomy of the learner to make the decisions necessary to meet one's<br />
own needs. She also notes the social aspect of <strong>learning</strong> by iterating that “[t]hese decisions ought to be<br />
made within the bounds of social acceptability and by self-directed, self-motivated, willing learners.”<br />
(1987: 5) Broad (2006: 121) himself states that the ultimate goal of independent <strong>learning</strong> is to teach<br />
students “to learn for themselves and in turn empower them in their <strong>learning</strong> whatever the context."<br />
Summing up these definitions may give us a rough idea of independent <strong>learning</strong> but what exactly is<br />
understood by “independent” is still ambiguous and open to many interpretations. In fact it is interesting<br />
to view the plethora of results from Kesten’s (1987) audit of the terms used synonymously to describe<br />
this type of <strong>learning</strong>: autonomous <strong>learning</strong>, independent study, self-directed <strong>learning</strong>, student initiated<br />
<strong>learning</strong>, project orientation, discovery and inquiry, teaching for thinking, <strong>learning</strong> to learn, self<br />
instruction and lifelong <strong>learning</strong>.<br />
117
Yin Ha Vivian Chan et al.<br />
What is shared by these interchangeable terms is the concept of autonomy. Researchers generally<br />
agree that that there should be a certain degree of autonomy which learners enjoy or acquire in their<br />
<strong>learning</strong> that these terms describe. Unfortunately, to define what autonomy in <strong>learning</strong> means is equally<br />
difficult. Among the many different definitions of autonomy in <strong>learning</strong>, there are two primary schools of<br />
thought: one maintains that the term should describe the “capacity of the learner”, inspired by Holec<br />
(1981: 5), while the other (Riley and Zoppis 1985; Dickinson 1987) refers to autonomy as more of a<br />
situational context. Researchers on autonomy are aware that this situational definition goes against the<br />
fact that learners who are forced to study in isolation do not necessarily develop autonomy. Instead,<br />
autonomy should entail some interdependence (Kohonen 1992), and collaboration (Little 1996) as the<br />
learner works with others toward shared goals, following a more communicative approach to <strong>learning</strong>.<br />
When autonomy is seen as the capacity to take control of one’s own <strong>learning</strong>, the question arises of how<br />
much control being referred to? Holec (1981: 3) draws an almost ideal picture of an autonomous learner<br />
by saying that he “is himself capable of making all these decisions concerning the <strong>learning</strong> with which<br />
he is or wishes to be involved.” These decisions include determining the objectives, defining the<br />
contents and progress, selecting methods and techniques, and monitoring and evaluating any progress<br />
and achievements. Benson (2001: 103) adds that autonomy without control over content is actually an<br />
inauthentic form of autonomy. Few would refuse this tempting image of the ideal autonomous learner.<br />
The question is this: Is it realistic to expect to have such an ideal autonomous learner in any present<br />
education system? If this type of ideal learners do exist, what is the proportion of them at an university?<br />
When education has become more and more an industry focused on accommodating thousands of<br />
students, does such an individualistic concept of <strong>learning</strong> have a place? How can this individual control<br />
over one’s <strong>learning</strong> be possible in collective education practices? Benson (1966: 33) in an earlier paper<br />
argued that: “[c]ontrol is a question of collective decision-making rather than individual choice.”<br />
However, in reality, perhaps such absolute control over one’s <strong>learning</strong> is theoretical and may serve as<br />
more of an ultimate goal to pursue.<br />
Although there is still no concrete agreement of what learner autonomy means, there are some<br />
common basic beliefs. Benson (2001: 183) summarises three hypotheses: first, autonomy is available<br />
to all; second, learners who lack autonomy are capable of developing it; and third, autonomous <strong>learning</strong><br />
is more effective than non-autonomous <strong>learning</strong>. The first two are less disputable after the paradigm<br />
shift of a genuine belief in students' ability and responsibility occurred several decades ago. Yet the<br />
third hypothesis remains debatable. Researchers try to argue for its validity but invariably admit that it is<br />
difficult to measure the effectiveness of autonomous <strong>learning</strong>. Some research collects positive<br />
feedback for self-access centres as a means to prove this effectiveness, but unfortunately the success<br />
of a self-access centre does not guarantee the enhancement of learner autonomy. Nor does the<br />
improvement of learner autonomy necessarily result in more effective <strong>learning</strong>. On the contrary, some<br />
research findings even work against learner autonomy. Ferugson and Tryjankowski (2009) conducted a<br />
study comparing online and face-to-face <strong>learning</strong> in Master’s-level courses. The research looked at<br />
whether these two modes of the same course resulted in two equivalent groups of MA students<br />
achieving the same level of <strong>learning</strong> success. Results did not show this to be the case. In fact, students<br />
in the face-to-face courses did better than those in the online courses. This was especially the case<br />
when students were struggling academically.<br />
The setting up of independent <strong>learning</strong> centres when funding is abundant may illustrate foresight and is<br />
often ambitious. However, in the case of Hong Kong, the higher education system is in the process of<br />
returning to a four-year undergraduate curriculum and funding is tight, so the major concern of the<br />
funding authorities right now is whether these centres can share the increased face-to-face teaching<br />
load in a more economical way with an emphasis on providing online resources for students.<br />
2. Autonomous <strong>learning</strong> in Hong Kong higher education<br />
The concept of autonomy in <strong>learning</strong> can be traced back as early as to the education philosophy of<br />
Jean-Jacques Rousseau (1712–1778) and John Dewey (1859–1952). However, the concept did not<br />
enter into the field of language <strong>learning</strong> until the late 1960s. Early interest in autonomy in language<br />
<strong>learning</strong> is said to have been part of a response to the social and ideological changes of the time. A<br />
respect for the individual in society (Holec, 1981), an emphasis on the meaning and value of personal<br />
experience (Gremmo and Riley, 1995), and the notion of student power (Cockburn and Blackburn,<br />
1970) all have informed the research and practice of autonomy in the field. Self-access resource<br />
centres that encourage self-directed <strong>learning</strong> have proliferated over the last three decades in higher<br />
education throughout the world. It is also understandable that autonomy arouses interest in the field of<br />
118
Yin Ha Vivian Chan et al.<br />
language <strong>learning</strong> as language is one of the primary tools used to acquire knowledge, and is thus an<br />
essential aspect of independent <strong>learning</strong>.<br />
This emphasis on languages within self-access centres is particularly true for Hong Kong education. As<br />
an international city, Hong Kong takes pride in its bilingualism: both English and Chinese (the latter<br />
often referring to written Chinese and the local dialect of Cantonese for everyday spoken<br />
communication). Since the handover of Hong Kong’s sovereignty, Hong Kong’s Education Bureau has<br />
promoted various approaches for the population to cope with the new role the city plays in relation to<br />
China and the rest of the world. One of the major tactics has been the encouragement of “bi-literacy and<br />
tri-lingualism” in education. While the written languages refer to standard English and Chinese, the<br />
spoken refer to English, Cantonese, and Putonghua. In 1998, the government decided to require that<br />
the “mother-tongue” be implemented as the medium of instruction in schools, which triggered a<br />
vigorous debate about whether such a change would lead to a decline in the English proficiency of local<br />
students. This concern reveals the public awareness that the mastery of languages can lead to success<br />
in study and vice versa.<br />
The importance of language in Hong Kong was further reinforced in 2005 when the Education Bureau<br />
presented the document “Reforming the <strong>Academic</strong> Structure for Senior Secondary Education and<br />
Higher Education – Actions for Investing in the Future.” It announced the full implementation of a new<br />
high school education system in 2009 and the complete switch from the 3-2-2-3 to 3-3-4 curriculum in<br />
2012 1 . The switch poses great challenges to all parties concerned. As a result, large-scale curriculum<br />
revisions at the high school level have been ongoing since 2009; the most significant of which is<br />
probably that of Chinese language and the incorporation of liberal studies into the curriculum. By<br />
broadening the scope for these two subjects, the Bureau aims to encourage extensive reading and<br />
nurture youngsters to become well-informed, well-educated, and well-cultured members of society.<br />
The emphasis on language enhancement is not only at the high school level. Various universities have<br />
already launched pilot courses on languages and general education to cope with the change in the high<br />
school curriculum. Take, for example, The Chinese University of Hong Kong where newly admitted<br />
students will have to take 21 credits for general education, six credits for Chinese, and nine credits for<br />
English. These new language requirements are meant to prepare students to handle the voluminous<br />
reading requirements in both Chinese and English and to sharpen their language abilities in preparation<br />
for the future job market. According to Bachman’s 2008/09 review of the efficiency of language<br />
education in Hong Kong tertiary institutions, language enhancement is an urgent need for local<br />
university students. Bachman notes:<br />
Virtually all the research literature on achieving high-levels of proficiency in a second or<br />
foreign language suggests that once learners have reached a given level, increasing larger<br />
amounts of time are required for them to move from intermediate to advanced levels.<br />
(2010: 2)<br />
His remark illustrates why universities need to invest more resources to cater for students’ language<br />
enhancement needs in preparation for serious academic studies and research. Bachman (2010: 2)<br />
further notes that the present resources available to students are insufficient and suggests that local<br />
universities should radically increase the number of language education courses they offer in order to<br />
help students to advance their language proficiency.<br />
What makes the situation in Hong Kong even more daunting is that the number of first year university<br />
students will increase by an estimated 40 per cent in 2012. This “double cohort” includes the last group<br />
of students under the former education system as well as the new group under the revised curriculum,<br />
both of whom will enter local universities in the same year, thus creating more budgetary strain. In view<br />
of this, it is not surprised that the university administration and those in charge of the budgets see<br />
independent <strong>learning</strong> as the most simple and direct means to deal with this surge of student intake and<br />
alleviate potential financial pressure. However, their understanding of independent <strong>learning</strong> remains<br />
unclear. For many, a popular view is that independent <strong>learning</strong> simply means transferring face-to-face<br />
teaching to self-study and thus the responsibility would be put onto universities’ self-access centres.<br />
However, this view has been deeply questioned by the recent research in the field. Sturtridge (1997: 70)<br />
calls attention to this intention of funding authorities to see one-off capital expenditures on self-access<br />
1 Originally, students had to spend three years to complete junior high, two years for high school, two years for university<br />
preparatory courses, and three years for university. After the reform, students will take three years to complete junior high, three<br />
years for high school and four years for university.<br />
119
Yin Ha Vivian Chan et al.<br />
as a viable alternative to more expensive ongoing expenditures on teacher salaries. Ferugson and<br />
Tryjankowski (2009) in their research conclude that simply transferring a face-to-face syllabus to an<br />
online medium does not result in an equal <strong>learning</strong> environment. The lack of support, community and<br />
connectedness, and a dearth of specific skills and attitudes needed for online teaching and <strong>learning</strong> in<br />
particular tend to lead to this unfavourable result. In addition, Gardner and Miller (1999:31) remind us<br />
that “a key element in success SALL [self-access language <strong>learning</strong>] is pedagogical input and this is<br />
relatively expensive”. Littlejohn and O’Dell both have similar observations. Littlejohn (1985) suggests<br />
that the effective use of self-access centres depends on the extent to which each teacher views<br />
self-access work as valuable. O’Dell (1992) observes that students who made full use of the resources<br />
of the Eurocentre Cambridge in the United Kingdom were those in classes where the teacher was<br />
confident and well-informed of the centre. These researches indicate that attempts to use self-access<br />
centres as a means to cut costs is the result of a misunderstanding of their purpose, and also brings<br />
about little improvement in student <strong>learning</strong>, which should be the focus of a university.<br />
The eight local tertiary institutions currently funded by the University Grants Committee (UGC) have all<br />
established either a language or self-access centre, many times both. The establishment and facilitation<br />
of these centres are both in line with the goal set down by Chief Executive, Donald Tsang, and the UGC<br />
to increase Hong Kong’s global competitiveness. This blatant embrace of the global economy is deeply<br />
disturbing as research has pointed out that in these circumstances the concern of language education<br />
becomes “a service to global economy in which language skills represent a form of economic capital”<br />
(Benson, 2001: 19), and that “[b]roader political concerns about autonomy are increasingly replaced by<br />
concerns about how to develop strategies for learner autonomy” (Pennycook, 1997: 41). The push for<br />
the economically-driven nature of these centres illustrates a larger problem in terms of the<br />
commercialization of Hong Kong education and is worth more research and attention.<br />
The major role of Hong Kong’s self-access centres is to help students to master language. So, it makes<br />
sense that their two common features are the provision of language <strong>learning</strong> resources in different<br />
formats and language consultations. Approximately half of these centres also run workshops. In<br />
addition, the teachers at these centres act as facilitators and mentors rather than simply as teachers.<br />
They direct students to relevant self-<strong>learning</strong> materials and guide them through their plan to improve<br />
their language abilities, which includes providing advice at opportune times. These centres are<br />
therefore significantly different from libraries in the sense that they not only provide materials, but also<br />
provide tutorial support to help users develop language and study skills and design a suitable<br />
independent study strategy.<br />
The practice of providing human support in the form of teachers has been implemented with discretion<br />
in these centres. In 1994, instructors from different Hong Kong institutions published research on the<br />
self-access centres at their respective universities. This research revealed that self-access centres run<br />
most effectively with the guidance of teachers and most properly utilized when the teaching and<br />
<strong>learning</strong> behaviors changed from the giver-receiver mode to that of trainer-trainee. Star’s (1994)<br />
research at the Hong Kong Institute of Education revealed three major points. 1) Users prefer some<br />
guidance to save time and effort in searching for suitable materials; 2) they prefer some training in<br />
cognitive as well as meta-cognitive strategies; and 3) they express the need for materials to help check<br />
progress. She concludes that “learner training” is a key to the success of a self-access centre (Star,<br />
1994: 165). While Star proposes various ways to promote self-access centres, she also notes that it is<br />
necessary to improve consultations “by explaining their purpose to both students and tutors and asking<br />
tutors to be more proactive in the area.” (1994: 166) Contrary to the nonprofessional belief that a<br />
self-access centre does not need teachers, in fact research and actual practice reveal that tutors play<br />
an important role in facilitating and promoting self-access centres in tertiary education.<br />
Star’s findings do not represent an isolated case. Pang (1994: 37) from Lingnan University comments<br />
on the potential of self-access centres as follows: “The SAC [self-access centre] acts to provide much<br />
needed consultation and support — moral, pedagogical, linguistic and tactical.” While Miller and<br />
Gardner (1994) from the City Polytechnic of Hong Kong and the University of Hong Kong respectively at<br />
that time both point out the importance of learner training and teacher training, how the learner functions<br />
as an independent learner and the teacher as learner trainers.<br />
At present, in preparation for the new curriculum in 2012, some local self-access centres seem to be<br />
narrowing their focus to writing, which may result in their apparent similarity to writing centres. However,<br />
120
Yin Ha Vivian Chan et al.<br />
the Independent Learning Centre (ILC) of The Chinese University of Hong Kong (CUHK) is also worthy<br />
of mention due to recent developments and transition of its focus.<br />
3. Independent Learning at CUHK: Challenges and solutions<br />
The ILC of CUHK was originally run by the English Language Teaching Unit (ELTU) when it first started<br />
in October 1993 and became a bilingual centre shortly after its opening. Staff from the Chinese<br />
Department together with those from the ELTU formed a task force to help students enhance their<br />
language proficiency through independent <strong>learning</strong>. However, the ILC became an independent unit in<br />
2005 and has its own staff. This independence gives the ILC more freedom in fostering autonomous<br />
<strong>learning</strong> among students, but at the same time poses significant challenges to it.<br />
Challenge 1:<br />
The fact that most of the self-access centres (SAC) affiliate to their university language centre(s) is not<br />
without reasons. Some of them may even share teaching staff like the ILC did. Such close connection<br />
gives the language teachers a better position to understand the missions and resources of the SAC.<br />
Research has already shown that teachers’ attitude towards the centre and their familiarity with the<br />
resources of the centre are crucial to the effective use of it (Littlejohn, 1985; O’Dell, 1997). Therefore, it<br />
is not surprising to know that the major challenge the ILC faces seems to come from the Centre’s<br />
autonomous status outside the university’s language units. Though many efforts have been made to<br />
establish routine and productive engagement between the ILC and the two largest language units, there<br />
have been some issues that seem to delay or negate this effort. These issues seem to stem from the<br />
differences in understanding of what “self-access” or “independent <strong>learning</strong>” is among the parties<br />
concerned. This is perhaps understandable considering the difficulty practitioners have in defining<br />
these terms. Many within the language units and top administration appear to be of the opinion that the<br />
ILC’s role should be restricted to that of a resource centre, where students can take out resources or<br />
receive lists of links to use in their language study. Though this kind of provision is definitely one of the<br />
aspects of assisting students in enhancing their language ability and <strong>learning</strong> autonomy, simply<br />
providing these resources without being involved in discussions with the instructors from the language<br />
units in terms of how to best support their students’ needs within specific courses is not making full use<br />
of the ILC’s capacities, nor is it fully supporting the students’ needs. Therefore, there have been a<br />
number of attempts to work more closely with the language units, of which a recent example may serve<br />
to illustrate the current challenge.<br />
In response to the upcoming four-year undergraduate curriculum, ten per cent of the assessment of a<br />
few required language courses will go to online <strong>learning</strong>. In order for this to take place, a website will be<br />
set up and the ILC and the corresponding language unit are meant to collaborate to design part of the<br />
associated online <strong>learning</strong> materials. The language unit head and the instructors concerned have been<br />
invited to communicate with the ILC to share their goals and ideas for the website and online <strong>learning</strong>,<br />
and in preparation, the ILC has introduced its services and resources in extensive detail. Suggestions<br />
from the ILC have included the inclusion of interactive language exercises for the students, learner<br />
support for assigned readings, and the possibility of linking up the ILC’s website to this new site for easy<br />
access to more independent <strong>learning</strong> materials. The idea behind this exchange of opinions and<br />
information is to encourage instructors who may still practice face-to-face teaching predominantly to<br />
understand more about the concept of learner autonomy and enable the Centre to gather more<br />
information about what sort of independent <strong>learning</strong> activities these instructors would consider effective<br />
in enhancing students’ language development.<br />
One of the key factors Sturtridge (1997) identifies in the effectiveness of self-access is whether activities<br />
of the SAC are integrated into the curriculum and teaching timetable. The ten per cent online <strong>learning</strong><br />
requirement written in the new language curricula could be an advantage to foster <strong>learning</strong> autonomy, if<br />
the online <strong>learning</strong> activities are carefully designed to achieve the goal. However, a couple of meetings<br />
have been held but no serious discussion on the concept of autonomy has been successfully started.<br />
The message implies by the top authorities of the university that independent <strong>learning</strong> simply means to<br />
shift paper-based materials to a digital format, and ultimately a way to cut cost, seems predominant.<br />
The online <strong>learning</strong> component is seen mainly as a means to cut down contact hours and to save<br />
marking time. In addition, mutual trust in certain academic circles seems difficult to establish. In a subtle<br />
and indirect way, there seem to be reservations expressed by some parties involved about whether and<br />
how the ILC can academically facilitate language <strong>learning</strong>, though our instructors all come from the<br />
same academic background and experience level as those in the language units. In other occasions,<br />
121
Yin Ha Vivian Chan et al.<br />
the ILC has been seen as a threat and rival. Perhaps it is difficult to expect standalone language<br />
teaching-related units to work smoothly and with complete trust if they are not officially linked.<br />
Ultimately, the progression of this collaboration is at a standstill. There is currently no working schedule<br />
to move the collaboration forward.<br />
The potential solution to this challenge could perhaps come from a change in the culture currently in<br />
existence between the ILC and the language units, and in the University in general. This may be best<br />
facilitated by the involvement of the related directors and higher administration to require the staff in all<br />
the units to work together to integrate the principles of independent <strong>learning</strong> into the new four-year<br />
curriculum for the relevant language courses that will be offered. This would require the staff to meet on<br />
a routine basis, rather than simply leaving it up to the staff to participate if and how they see fit.<br />
Sturtridge’s (1997) argument that all staff should be involved as stakeholders in the development of<br />
self-access facilities perhaps is not too ideal or ambitious. In the long run, if we see the need for the<br />
independent <strong>learning</strong> facilities and learner autonomy to develop and grow, a deeper understanding of<br />
the concept and widespread support must be established within the University.<br />
Challenge 2:<br />
The ILC has experienced some recent developments and a transition of focus over the last few years in<br />
the run-up to the new curriculum that will be put in place after September 2012. Over the previous two<br />
years, the Centre has experienced numerous changes initiated from management above. In 2009,<br />
these higher authorities decided to both drop the word yuwen 語文 from the ILC’s Chinese name and to<br />
encourage staff members to focus more on study skills rather than language improvement. The ultimate<br />
goal has been for the ILC to develop an online resource platform where students can access<br />
information and <strong>learning</strong> resources of their programmes of study. It is hoped that by doing so the ILC<br />
can assist the development of their study skills and <strong>learning</strong> strategies while also promote lifelong<br />
<strong>learning</strong>. This requires the ILC to work in partnership with teachers and units across the University in<br />
providing resources and activities to support students’ various <strong>learning</strong> needs. This type of independent<br />
<strong>learning</strong> would be much broader than the original focus on language enhancement and is undoubtedly<br />
ambitious. If successful, it could expand the practice of independent <strong>learning</strong> to various academic<br />
programmes in the University. However, this plan faces many obstacles, the most apparent of which is<br />
to gain teachers’ support. The discussion of student’s <strong>learning</strong> needs is a sensitive, sometimes political,<br />
issue, especially when it is initiated by an outside unit as the ILC not to mention that language teachers<br />
are traditionally less respected in the academic circle.. Despite the various attempts at different levels<br />
by the ILC Director and the higher authorities to introduce the ILC support to students’ <strong>learning</strong> needs to<br />
different departments, there are very few responses that suggest genuine interests.<br />
Apparently it is not that students do not have problems in their studies, nor do the teachers not care. It is<br />
the question of whose responsibility it is to take care of students’ <strong>learning</strong> needs. Perhaps it is like<br />
opening up a Pandora's box of problems of responsibility which entails the investment of extra funding,<br />
manpower, time, etc. There may also be an unwillingness to upset the status quo. The intended move<br />
away from the traditional role the ILC to assist students in their independent language enhancement<br />
work has also had the secondary effect of causing confusion amongst university staff, who are quite<br />
rightly uncertain of the educational role the ILC currently plays, and will play in the future.<br />
The solution to this challenge would seem to involve an open discussion amongst all the stakeholders<br />
involved: the ILC, the heads of all the different academic departments, student union representatives<br />
and directors of the two language units. Perhaps if this sort of a dialogue could take place on a routine<br />
basis, it would be possible to determine what the academic departments require most from their<br />
students, what the students feel they need in terms of support and what the language units and what the<br />
ILC is capable of offering by the 2012 deadline before the new four-year curriculum begins.<br />
4. Conclusion<br />
Hong Kong’s university system will face an unprecedented challenge when it switches to a four-year<br />
curriculum in 2012. The pressure will be high on academic staff and freshmen entering universities in<br />
the next few years when the new curriculum is in effect. Therefore, there is the potential for independent<br />
<strong>learning</strong> and self-access centres to play a beneficial role in alleviating some of this pressure, though<br />
what this role will be exactly must be thought through carefully.<br />
122
Yin Ha Vivian Chan et al.<br />
The role of the CUHK’s ILC has been under some scrutiny in the run-up to the new curriculum’s<br />
implementation. Traditionally, the Centre has functioned as a centre to support students’ independent<br />
<strong>learning</strong> of Chinese and English, as these are the two modes of communication required for students to<br />
acquire and communicate knowledge in a local tertiary environment. In addition, the Hong Kong<br />
government has continued to emphasize the importance of becoming bilingual in at least these two<br />
languages, and outside experts such as Bachman have reported that students need more support in<br />
these languages, not less. However, the ILC faces challenges from different levels within the University<br />
to abandon its traditional focus on language enhancement, and to act as more of a resource centre<br />
while working with a wide variety of departments to create online versions of course materials for<br />
students to access without teacher support. There are also challenges to overcome in surmounting<br />
some misunderstandings of what independent <strong>learning</strong> is and how it is best implemented for students to<br />
improve their ability to learn and improve their language skills independently. Potential solutions have<br />
been suggested, but it should be remembered that all of these challenges exist while the university<br />
readies itself for the upcoming influx of students who will represent a 40 per cent increase in student<br />
intake in September 2012, so the stakes are high.<br />
References<br />
Bachman, Lyle F. (2010) Language enhancement in Hong Kong universities: Some observations and<br />
recommendations, review report on language enhancement activities, Hong Kong: University Grants<br />
Committee, pp. 1–19.<br />
Benson, P. (1996) “Concepts of autonomy in language <strong>learning</strong>”, in R. Pemberton et al. (eds) Taking Control:<br />
Autonomy in Learning Learning, Hong Kong: Hong Kong University Press, pp. 27–34.<br />
Benson, P. (2001) Teaching and researching autonomy in language <strong>learning</strong>, Harlow: Pearson Education <strong>Limited</strong>.<br />
Broad, J. (2006) “Interpretations of independent <strong>learning</strong> in further education”, Journal of Further and Higher<br />
Education, vol. 30, No. 2, May, pp. 119–143.<br />
Cockburn, A. and Blackburn, R. (eds.) (1970) Student power: Problems, diagnosis, action, London: Penguin.<br />
Dickinson, L. (1987) Self-instruction in language <strong>learning</strong>, Cambridge: Cambridge University Press.<br />
Gardner, D. and Miller, L. (1999) Establishing self-access: From theory to practice, Cambridge: Cambridge<br />
University Press.<br />
Gremmo, M.-J. and Riley, P. (1995) “Autonomy, self-direction and self-access in language teaching and <strong>learning</strong>:<br />
the history of an idea”, System , 23 (2), 151–64.<br />
Holec, H. (1981) Autonomy and foreign language <strong>learning</strong>, Oxford: Pergamon.<br />
Kesten, C. (1987) Independent <strong>learning</strong>: a common essential <strong>learning</strong>, A study completed for the Saskatchewan<br />
Department of Education Core Curriculum Investigation Project, Saskatchewan Department of Education,<br />
University of Regina.<br />
Kohonen, V. (1992) “Experiential language <strong>learning</strong>: Second language <strong>learning</strong> as cooperative learner education”,<br />
in D. Nunan (ed.) Collaborative Language Learning and Teaching, Cambridge: Cambridge University Press,<br />
pp. 14–39.<br />
Little, D. (1996) “Freedom to learn and complusion to interact: Promoting learner autonomy through the use of<br />
information systems and information technologies”, in R. Pemberton et al. (eds) Taking Control: Autonomy in<br />
Language Learning, Hong Kong: Hong Kong University Press, 203–18.<br />
Littlejohn, A. (1985) “Learner choice in language study”, ELT Journal, 39 (4): 253–61.<br />
Miller, Lindsay and Gardner, David, “Directions for research into self-access language <strong>learning</strong>” in Gardner, David<br />
and Miller, Lindsay (eds.) (1994) Directions in Self-access Language Learning, pp. 167–175.<br />
Moore, M. G. (1984) “Concepts of adult education”, in: R. Bell (ed.) Education for Adults, Milton Keynes: The Open<br />
University Press.<br />
O’Dell, F. (1992) “Helping teachers to use a self-access centre to its full potential”, ELT Journal, 46 (2): 153–9.<br />
O’Dell, F. (1997) “Confidence building for classroom teachers working with self-access resources”, in P. Benson<br />
and P. Voller (eds.) Autonomy and Independence in Language Learning, London: Longman, pp. 150–63.<br />
Pang, Terence T. T. (1994) “A self-directed project: A critical humanistic approach to self-access”, in Gardner,<br />
David and Lindsay Miller (eds.) Directions in Self-access Language Learning, Hong Kong: Hong Kong<br />
University Press, pp. 29–38.<br />
Pennycook, A. (1997) Cultural alternatices and autonomy, in P. B. (eds.) Autonomy and Independence in<br />
Language Learning, London: Longman, pp. 35–53.<br />
Riley, P. and Zoppis, C. (1985) “The sound and video library”, in P. Riley (ed.) Discourse and Learning, London:<br />
Longman, pp. 286–98.<br />
Star, Marian. (1994) “Learning to improve: Evaluating self-access centres”, in Gardner, David and Miller, Lindsay<br />
(eds.) Directions in Self-access Language Learning, Hong Kong: Hong Kong University Press, pp. 157–166.<br />
Sturtridge, G. (1997) “Teaching and language <strong>learning</strong> in self-access centres: Changing roles?” in P. Benson and<br />
P. Voller (eds.) Autonomy and Independence in Language Learning, London: Longman, 66–78.<br />
123
The Development and Application of a web Based<br />
Metacognitive Mapping Tool<br />
Serdar Çiftci 1 and Mehmet Akif Ocak 2<br />
1<br />
MS. Department of Educational Technology, Institute of Educational Studies<br />
of Gazi University, Turkey<br />
2<br />
PHd Department of Computer and Instructional Technologies Teaching, Gazi<br />
University, Turkey<br />
serdar@gazi.edu.tr<br />
maocak@gazi.edu.tr<br />
Abstract: Teachers support metacognition by following different teaching methods in face to face <strong>learning</strong><br />
environments. However, literature shows that web based <strong>learning</strong> environments lack of the use of tools which<br />
support and guide the students’ metacognitive activities. The primary purpose of this study is to give insight about<br />
development of a metacognitive mapping tool (MMT) supporting the college students’ metacognitive <strong>learning</strong><br />
abilities. The participants of this study were 25 third grade college students selected from Gazi Faculty of<br />
Education, Gazi University. The students participated in a six week qualitative study. They studied an online<br />
Database Management Course by getting support from the tool. The tool was developed on the four components<br />
of metacognitive skills which were planning, monitoring, evaluation and revision. The tool was utilized for an<br />
asynchronous web based course. The tool allowed students to plan their studies, monitor and evaluate their<br />
progress. After monitoring and evaluation, students could revise their planning and change their thinking patterns.<br />
A content analysis was carried out to examine students’ interviews. The focus of the interviews was to investigate<br />
the students’ perceptions and purpose of utilization. This study argues that using the tool gives the students an<br />
opportunity to choose exact content of the course and to plan the content for individual aims. The tool used in this<br />
study can assist time management and tracing of the students’ personal study progress.<br />
Keywords: metacognition, metacognitive mapping tool, web based education, eLearning, metacognitive <strong>learning</strong><br />
process<br />
1. Introduction<br />
The biggest advantage of web based <strong>learning</strong> is that it gives students active <strong>learning</strong> opportunities<br />
(Tsai 2009). Together with the increase in the demand for web based <strong>learning</strong> environments, today,<br />
this kind of environments shows the rapid development. The causes such as regional distances,<br />
equal educational opportunities and time constraints are among of the causes of this increase in<br />
demand. While the studies conducted in the first place dealt with the development of <strong>learning</strong><br />
environments and transfer of <strong>learning</strong> activities from the form of face-to-face environment to the web<br />
based environments. They, now, focuses on how web based <strong>learning</strong> can be given effectively. To<br />
provide an effective presentation of <strong>learning</strong> environments, supporting tools and advanced<br />
technologies in context must be provided and integrated. Collis and Meeuwsen (1999) indicated that<br />
students need to take responsibility for increased control of <strong>learning</strong> process and show positive<br />
attitude to achieve active <strong>learning</strong> in such an open-ended <strong>learning</strong> environment. Visual and written<br />
representation of the content on the web is not just enough to teach and learn. Students may be<br />
required for new <strong>learning</strong> strategies and skills so that they can become effective and successful online<br />
learners (Tsai 2009). Using <strong>learning</strong> tools help students to increase interaction with each other, the<br />
content and the teacher and are used more effectively. Over time, difficulties of web based <strong>learning</strong><br />
were overcome and studies have largely concentrated on how additional gains can be achieved from<br />
web based <strong>learning</strong>.<br />
2. Metacognition<br />
Metacognition, first, was defined by Flavell (1976) as knowledge about knowledge. Subasi (2003)<br />
defined the metacognition as the individual's own cognitive processes and knowledge. Schraw and<br />
Brooks (1998) and Zimmerman (1989) collected metacognitive processes under the heading of<br />
<strong>learning</strong> based on self-regulation. Prior research suggests that metacognitive strategies in face-toface<br />
<strong>learning</strong> improve students' achievements. Metacognitive knowledge of strategies consists of<br />
knowledge about cognitive strategies for enhancing <strong>learning</strong> and performance (Pifarre and Cobos<br />
2009) (Figure 1). For constructivist <strong>learning</strong>, students' better use of self-regulation skills facilitates the<br />
process of information processing and the recall. Learners having deficiencies in key metacognitive<br />
and self-regulatory skills will not learn much from open-ended <strong>learning</strong> environments without the<br />
implementation of scaffolds to help them along (Azevedo, Moos, Greene, Winters and Cromley,<br />
124
Serdar Çiftci and Mehmet Akif Ocak<br />
2008). Thus, students' metacognitive awareness is important. Jones, Farquhar and Surry (1995)<br />
suggest that with improved learner awareness of metacognitive knowledge, <strong>learning</strong> efficacy could<br />
also be enhanced. Especially, in web based <strong>learning</strong> environments, the students have a sense of<br />
loneliness. In order to overcome these difficulties, different supporting tools for this purpose might be<br />
structured. Learning in a hypermedia environment requires from the learner to regulate his or her own<br />
<strong>learning</strong> in order to construct higher and deeper levels of knowledge; that is, it provides to make<br />
decisions about what, how, how much to learn, how much time to spend on it, how to access other<br />
educational materials, how to realize whether he or she understands the material, how to modify<br />
plans and strategies to learn better and when to increase one’s effort (Azevedo et al. 2008). This<br />
research is shaped on the basis of the metacognitive grounds.<br />
Figure 1: Metacognition development strategies used in face-to-face <strong>learning</strong> environments<br />
2.1 Metacognitive mapping tool<br />
Metacognitive mapping tool (MMT) was developed based on the principles of the students’ planning<br />
of a web based content, monitoring of their activities and evaluation cycle (Figure 2). Students<br />
engaging in web based <strong>learning</strong> environments plan their own <strong>learning</strong> activities. This planning, mostly,<br />
takes place in the student's mind and cannot be followed actively. One of the aims of the MMT is to<br />
record planning, monitoring and evaluation activities shaped in the students' mind while guiding the<br />
students in a metacognitive way at the same time. In this way, students’ behavior can be observed.<br />
As a result of the monitoring, based on the students’ process, <strong>learning</strong> environment, mapping tool and<br />
the content can be formed in a more effective way.<br />
Figure 2: Four components of metacognitive skills<br />
Four components of metacognitive skills are explained below:<br />
Planning: Students in this section will be able to choose which headings to cover, which sequence to<br />
use and how much time they will spend on each topic. Planning consists of two sections. In the first<br />
section, students choose their topics that they will study and determine the study sequence. In the<br />
second section, students decide time planning of each topic.<br />
Monitoring: The students will be able to monitor the process during the instruction. The stage of the<br />
process which was planned, how much time was spent and remaining time might be accessible to all<br />
students from this section. The current situation might be supported with visual help and it is visible<br />
whether the student follows the plan which is part of the whole process. This section provides the<br />
infrastructure to the next sections which are evaluation and revision.<br />
Self-evaluation: The students monitoring the current state with monitoring section of the tool will be<br />
able to evaluate themselves whether the situation they are in is correct. In this section, students<br />
evaluate whether they obey the plan (in terms of time, content and sequence) with "yes / no" answers.<br />
The answer, "Yes”, shows that the student followed the plan in the <strong>learning</strong> process and did not need<br />
any editing. "No" response offers students revision stage by which the students get the opportunity to<br />
correct mistakes and see the lacking points in the process.<br />
125
Serdar Çiftci and Mehmet Akif Ocak<br />
Revision: In revision stage, opportunities for changes are given to the students to plan the whole<br />
process again and allow them to see what they did before. Students finding themselves insufficient<br />
and are not confident with the current situation might return to previous topics and change their plans<br />
in revision.<br />
2.2 Development and application of the MMT<br />
In application, MySQL Database Management Course was administered to the students through the<br />
web based <strong>learning</strong> environment. Students, with a period of 6 weeks of training, received instruction<br />
via the web and used metacognitive mapping tool as a means of support. Learning environment and<br />
metacognitive mapping tool were constructed via PHP programming language and MySQL database<br />
management system. Throughout the development of the MMT, expert opinions on content, <strong>learning</strong><br />
environment and the MMT were obtained. Some revisions have been made based on the feedback<br />
obtained from the experts. Figure 3, 4 and 5 show screenshots of the MMT tool. Figure 3 shows the<br />
main screen of the web based environment and the MMT.<br />
Figure 3: MMT in the web based <strong>learning</strong> environment<br />
Figure 4 shows the first face of the planning section of the tool.In this section, students are allowed to<br />
choose the topics they will study from the existing course content by using drag and drop method.<br />
Figure 4: First phase of the planning section of the MMT<br />
126
Serdar Çiftci and Mehmet Akif Ocak<br />
Figure 5 shows the second phase of the planning section of the tool. Students decide on study time of<br />
chosen topics. For example, in Figure 5, there are 3 topics chosen by the student and 50 minutes<br />
were given to each topic. Totally, the student plans to work 150 minutes to study on chosen topics.<br />
Figure 5: Second phase of the planning section of the MMT tool<br />
3. Purpose of the study<br />
This research aims to give insight about development and implementation of MMT, planning to help<br />
college students' metacognitive activities to organize a web based course content, course work and<br />
activities. In web based <strong>learning</strong> environments, a <strong>learning</strong> tool is needed to improve students'<br />
metacognitive awareness and help them in planning, monitoring and self-evaluation procedures.<br />
Considering this requirement, the purpose of the study is to develop and implement a supporting tool<br />
to help students to arrange metacognitive activities.<br />
4. Method<br />
4.1 Participants<br />
The participants of the study were of 25 third grade college students studying in the Department of<br />
Computer and Instructional Technologies Teaching, Gazi Faculty of Education, Gazi University. The<br />
research was conducted in spring semester of 2010-2011 academic year. Of the students in the<br />
study, 16 (65.38%) were male and 9 (34.61%) were female.<br />
4.2 Learning environment<br />
Before the study begins, the researchers conducted one week orientation session for all students in<br />
the computer laboratory. In this orientation, students were explained to use the tool and web based<br />
<strong>learning</strong> environment. Additionally, there were no pre-information or sample planning provided to the<br />
students. Using the tool takes approximately 6 to 10 minutes to complete. Any additional information<br />
was not needed for utilization of the tool.<br />
Students were given <strong>learning</strong> tasks for each week. Total, 6 tasks were administered to students.<br />
Tasks were related to topics that students studied in accordance with <strong>learning</strong> goals. Students were<br />
asked to complete one task for each week. Aim of the tasks was to engage students to use the MMT<br />
and study the chosen topics in web based <strong>learning</strong> environment.<br />
The tool was used in the MySQL Database Management Course. Third grade college students<br />
planned their study in terms of content, time and sequence. In other words, “A plan” means that<br />
students choose which headings to cover, which sequence to use and how much time they will spend<br />
on each topic. This progress lasted 6 weeks. Students were able to monitor their study activities and<br />
check their plan if they follow the plan efficiently.<br />
127
4.3 Data collection<br />
Serdar Çiftci and Mehmet Akif Ocak<br />
In this study, in order to provide students a metacognitive support in a web based <strong>learning</strong><br />
environment, a MMT has been developed. The intent of developing a separate tool was to offer<br />
support to students while studying in the web based environment. In the data collection process, the<br />
students were asked open-ended questions by using the interview form. All students participating in<br />
the application answered every question. A total of 8 questions administered to students were asked<br />
online.<br />
Interview questions presented to students as follows:<br />
1. In the planning stage, please explain how MMT helped you?<br />
2. In the monitoring stage, to what extent MMT helped you?<br />
3. In the self-evaluation stage, to what extent MMT helped you?<br />
4. In the revision stage, to what extent MMT helped you?<br />
5. With the MMT, did your awareness towards planning, monitoring, self- evaluation and revision<br />
activities increase in your mind?<br />
6. With the MMT, did you arrange your own work activities more efficiently?<br />
7. Briefly describe the points that you find useful when using MMT.<br />
8. Briefly explain the missing points that you see in MMT.<br />
9. While studying in web based environment, do you need this kind of tool/ support / guidance?<br />
4.4 Data analysis<br />
The data obtained from the study were examined by content analysis. Common patterns were<br />
identified in content analysis. As a result, the frequency and percentage of given responses were<br />
obtained. Based on the frequency of students' usage of metacognitive mapping tool, common<br />
keywords were identified and patterns were classified.<br />
5. Results<br />
As a result of the interview responses of students on open-ended questions, frequency and<br />
percentage of responses were calculated and used. Results are presented separately for each<br />
question below:<br />
1. In the planning stage, please explain how MMT helped you?<br />
Most of the students indicated, with the help of planning phase of the tool, it was helpful to plan their<br />
<strong>learning</strong>. Students reported that, by using planning section of the tool, they prepared a plan in the web<br />
based environment according to their own <strong>learning</strong> styles. It shows that metacognition is not a linear<br />
process that moves from preparing and planning to evaluating (Anderson, 2002). When analyzing the<br />
answers given by the students, frequency and percentage of the prominent answers are shown in<br />
Table 1.<br />
Table 1: Students’ responses to question 1<br />
Keywords f %<br />
Planning 23 44,23<br />
For assistant 16 30,77<br />
For efficient study 6 11,54<br />
Adaptation 3 5,77<br />
I did not use the tool very much 3 5,77<br />
It provided guidance 1 1,92<br />
Total 52 100<br />
128
Serdar Çiftci and Mehmet Akif Ocak<br />
Some of the sample answers given by students to question 1 were as follows: “In order to choice the<br />
topics, it created a specific order and depending on this order, it provided easiness to study the topics.<br />
Therefore, with the planning phase of the MMT, it provided planning to study the topics effectively and<br />
efficiently with a particular sequence.”, “In a specified period of time, it helped to study the topics<br />
which I chose more effectively and in a disciplined way.”, “It helped me to follow the topics<br />
respectively; it assisted me on how much time I will spend for each subject.”, “It helped me to plan<br />
and arrange the topics according to the order of importance.”. Similarly, Simons (1994) found that one<br />
aspect of planning that has been studied is whether students plan different <strong>learning</strong> activities when<br />
<strong>learning</strong> for recall and when preparing for comprehension.<br />
2. In the monitoring stage, to what extent MMT helped you?<br />
Of the coded responses, 55.56% specifically indicated that the students were able to monitor their<br />
study progress. 29.63% of the coding indicated that the MMT helped students to go through the<br />
<strong>learning</strong>, to see <strong>learning</strong> deficiencies and to estimate the required time for each topic. Examining the<br />
data obtained from interviews, it was identified that the students, with the monitoring section of the<br />
MMT, could trace their own study activities and give direction to their study. The tool, also, allowed<br />
the students to monitor their study speed and re-shape their own work schedules. When analyzing the<br />
answers given by the students, prominent answers are shown in Table 2.<br />
Table 2: Students’ responses to question 2<br />
Keywords f %<br />
Monitoring 15 55,56<br />
It gives help 8 29,63<br />
Step by step <strong>learning</strong> 1 3,70<br />
Disciplined work 1 3,70<br />
I did not see its effect 1 3,70<br />
Guidance 1 3,70<br />
Total 27 100<br />
Sample answers given by students to question 2 were as follows: “I saw my deficiencies by looking<br />
the time spent on the topics and studied those topics again.”, “I watched the steps in a more<br />
disciplined way. I performed the work in a better way than I have done before.”, “I see the difference<br />
between my application and my plan. It helped me to see where I stayed behind on my work.”, “By<br />
seeing how much time I complete the topics, now I estimate the time required to split up for each<br />
subject better.”. Prior research also indicates that accurate monitoring is crucial in <strong>learning</strong> and<br />
training contexts where students have to master a great deal of new knowledge (Tobias, Everson and<br />
Laitusis, 1999).<br />
3. In the self-evaluation stage, to what extent MMT helped you?<br />
Of the coded responses, 76.19% indicated that the students received reminders for self-evaluation. In<br />
this way, students controlled their planning (14.28%) and study process (4.76%). When analyzing the<br />
answers given by the students, prominent answers are shown in Table 3.<br />
Table 3: Students’ responses to question 3<br />
Keywords f %<br />
Checking points for self-evaluation 16 76,19<br />
Re-planning 3 14,28<br />
Controlling my self-study 1 4,76<br />
Evaluation 1 4,76<br />
Total 21 100<br />
Sample answers given by students to question 3 are as follows: “In the evaluation stage, I got<br />
opportunity to control my own studies.”, “It gave me chance to re-plan when I needed”, “It remind me<br />
of self-evaluation by myself.”<br />
4. In the revision stage, to what extent MMT helped you?<br />
129
Serdar Çiftci and Mehmet Akif Ocak<br />
Of the coded responses, 72.73% specifically indicated that the students revised their study plan.<br />
13.64% of the coding shows that students did not see any benefit of the revision stage. Students who<br />
could not make proper planning in the first place processed their work more accurately. In particular,<br />
by starting to study the content, their study plan was much more clearer. When analyzing the answers<br />
given by the students, prominent answers are shown in Table 4.<br />
Table 4: Students’ responses to question 4<br />
Keywords f %<br />
Revision 16 72,73<br />
I did not see any benefit 3 13,64<br />
Plan arrangement 1 4,55<br />
Guidance 1 4,55<br />
Time saving 1 4,55<br />
Total 22 100<br />
Sample answers given by students to question 4 are as follows: “My pre-planning was random,<br />
because I did not know the content of the topics. After seeing and studying a little, I re-edited.”, “I saw<br />
unnecessary time planning on some topics and did re-planning to avoid loss of time by using the<br />
MMT.”, “I had the opportunity to re-edit the time spent more or less on the topics which I studied.”, “It<br />
helped me to arrange the topics which I had to study and arrange the time which must be spent.”.<br />
Thomas (2001) found that the effect of intervention on students’ metacognition was variable. Some<br />
students became increasingly metacognitive and reported evidence of revision of their <strong>learning</strong><br />
processes.<br />
5. With the MMT, did your awareness towards planning, monitoring, self-evaluation and revision<br />
activities increase in your mind?<br />
The answers to this question showed that students’ metacognitive activities in planning, monitoring,<br />
self-evaluation and revision stages with MMT converted their mental plan into writing. Concrete plans<br />
laid out by the students have enabled them to plan more accurately. When analyzing the answers<br />
given by the students, prominent answers are shown in Table 5.<br />
Table 5: Students’ responses to question 5<br />
Keywords f %<br />
It raised my awareness 16 53,33<br />
Helped to plan 5 16,67<br />
I converted my mental plan into writing 4 13,33<br />
My awareness increased partially 3 10,00<br />
I saw my deficiencies 1 3,33<br />
It was useless 1 3,33<br />
Total 30 100<br />
Sample answers to question 5 are given as follows: “Normally, when I study a lesson, I make a plan in<br />
my head. Since this plan was in my head, it was difficult for me to implement. With this tool, I know I<br />
used this plan in my mind; otherwise, trying to implement my plan would be very difficult. This issue<br />
has increased my attention more.”, “With MMT; my awareness has increased towards planning,<br />
monitoring, evaluation and revision activities. While <strong>learning</strong> the topic, it needed to draw the map from<br />
the beginning. While processing and also at the end, it was useful to use this tool to increase<br />
efficiency.”<br />
5. With the MMT tool, did you arrange your own work activities more efficiently?<br />
The students found MMT useful to do the plan. Deficiencies in the first planning have been corrected<br />
by the revision in the next stages. When analyzing the answers given by the students, prominent<br />
answers are shown in Table 6.<br />
130
Table 6: Students’ responses to question 6<br />
Serdar Çiftci and Mehmet Akif Ocak<br />
Keywords f %<br />
It supported my arrangement 20 64,52<br />
Problems with time arrangement 3 9,68<br />
There should be orientation 2 6,45<br />
I did not need it 2 6,45<br />
It provided adaptation 1 3,23<br />
Supportive for efficient planning 1 3,23<br />
It should be improved 1 3,23<br />
It helped partially 1 3,23<br />
Total 31 100<br />
Sample answers to question 6 are given as follows: “I did not have difficult time to choose the topics<br />
in order; however, there were different from the first done. I reduced these differences over time.”, “It<br />
helped me to edit my studies. Thanks to this kind of tools that made it easier to plan and implement.”,<br />
“I could not arrange in my first planning. With the help of revision, I was able to arrange efficiently.”<br />
7. Briefly describe the points that you find useful when using MMT.<br />
The students found planning section of the tool most useful. Monitoring and evaluation stages,<br />
recognizing their own metacognivite abilities and adaptation were some of the other important points<br />
indicated by the students. When analyzing the answers given by the students, responses are shown<br />
in Table 7.<br />
Table 7: Students’ responses to question 7<br />
Keywords f %<br />
Planning 16 61,54<br />
Monitoring 6 23,08<br />
Self-evaluation 2 7,69<br />
Revision 2 7,69<br />
Total 26 100<br />
Some students' answers given to question 7 are as follows: “By providing me an opportunity to sort<br />
the content according to my pre-knowledge, I completed my own studies and this allowed me to<br />
recognize myself in the planning.”, “To present my study plans saved me from the chaos of<br />
complexity.”, “The most important point was time I saved.”, “It helped me to see how much time I<br />
spend to which content, how much I studied each content and whether I passed the time that I<br />
specified for each content.”, “For me, the most useful point was re-editing, because I had trouble time<br />
to spend the time given to chosen contents. Although I understand that content, I repeated two or<br />
three times.”<br />
8. Briefly explain the missing points that you see in MMT.<br />
Although some of the students pointed that MMT does not contain any deficiency, the data was<br />
obtained about some technical problems such as visuality and need of development of<br />
personalization points. When analyzing the answers given by the students, some of the important<br />
points are shown in Table 8.<br />
Sample students' answers given to question 8 are as follows: “I think MMT is enough for students;<br />
however, I believe it should contain more information for students.”, “In the course planning section,<br />
sample times might be specified.”, “Parts of the subject content studied might be presented visually.<br />
In other words, visuals might be added.”<br />
131
Table 8: Students’ responses to question 8<br />
Serdar Çiftci and Mehmet Akif Ocak<br />
Keywords f %<br />
There is no deficiency of the MMT 5 29,41<br />
There are problems with design 2 11,76<br />
Some constraints exist 2 11,76<br />
Time management problems 2 11,76<br />
Technical problems 2 11,76<br />
Detailed information is needed 2 11,76<br />
It should be developed 1 5,88<br />
Personalization problems 1 5,88<br />
Total 17 100<br />
9. While studying in a web based environment, do you need this kind of tool/ support / guidance?<br />
Findings show, while students study in web based environment, it was observed that they need such<br />
a tool. When studying in an individual lesson via the Internet, one of the important considerations was<br />
distracting variables affecting the students’ interest and attention. The MMT made a positive effect on<br />
planning to deal with such issues. When analyzing the answers given by the students, responses are<br />
shown in Table 9.<br />
Table 9: Students’ responses to question 9<br />
Keywords f %<br />
Yes (For <strong>learning</strong>) 15 65,22<br />
Yes (For the web) 4 17,39<br />
Partially yes 2 8,70<br />
No 2 8,70<br />
Total 23 100<br />
Sample answers given by students to question 9 are as follows: “I have not used this type of a web<br />
based tool before. It caught my attention and I started to use. I think it is great for planning and<br />
monitoring.”, “When using web based course, it is very easy to lose my interest since the internet is<br />
always under my hand. But thanks to this tool that helped to develop a plan and according to the plan,<br />
I was able to focus on to what I really have to do.”, “I think it is necessary to have such a tool not only<br />
in web based environments but also in other <strong>learning</strong> environments. So far, I was not aware of the<br />
need to use such a tool. Moreover, I think everyone needs such a tool.”<br />
6. Conclusion and recommendations<br />
The purpose of the study was to examine students’ metacognitive activities in a web based <strong>learning</strong><br />
environment. The MMT was developed to guide and allow students to achieve structured <strong>learning</strong><br />
task. Results found that the MMT constructed according to metacognitive components (planning,<br />
monitoring, self-evaluation and revision) was used by the students efficiently and positively.<br />
Thinking about online courses, Henning (2003) indicated that we need “more than data of<br />
performance in the technology is needed to interpret the learners’ social position comprehensively” (p.<br />
304). Hence, supporting students’ metacognitive activities with <strong>learning</strong> tools would be a solution to<br />
this obstacle. The American Psychological Association (APA), in its report for a framework for school<br />
redesign and reform (APA, 1997), concluded that metacognition is one of the most important<br />
principles for <strong>learning</strong>. It is important to note that that basic concept behind the metacognition differs<br />
from adaptive instructional systems. In this study metacognition mainly deals with students’ selfregulated<br />
<strong>learning</strong> strategies and perceptions on metacognitive skills. As indicated by Carrier and<br />
Jonassen (1988), adaptive <strong>learning</strong> includes following variables: (a) aptitude variables (b) prior<br />
knowledge (c) cognitive styles and (d) personality variables. The structure of an adaptive system<br />
needs collection of information about individual user. This study did not focus on the variables<br />
indicated above; however further research will be needed to investigate the relationship between<br />
metacognition and adaptive system in terms of <strong>learning</strong> tools.<br />
132
Serdar Çiftci and Mehmet Akif Ocak<br />
The results showed that the use of MMT resulted in supporting student awareness and monitoring of<br />
thinking during complex tasks. Martinez (2010) found that similar metacognitive tools provided<br />
common vocabulary to make students’ thinking explicit to teachers and other students and providing a<br />
generic but powerful process that could be used by students for other <strong>learning</strong> tasks. Hartman (2001)<br />
noted that metacognition is especially important because it affects comprehension and retention of<br />
what is learned, in addition to affecting <strong>learning</strong> efficiency, critical thinking and problem solving.<br />
In web based <strong>learning</strong> environments, computers and the Internet might lead to distraction of attention.<br />
In this regard, the MMT was observed to make a positive impact on students. In addition, to make the<br />
metacognitive activities in the students’ mind concrete was one of the best advantages provided by<br />
MMT. The visual development of the tool and some of the sample plans for students as a<br />
presentation were seen as limitation. Further research might concentrate on the improvement of the<br />
tool. To present a sample plan to students is a condition to be investigated in terms of limiting the<br />
students’ own plan activities. As limitations of the study, there was a limitation on generalizing the<br />
study result because of the number of participants. Also, the MMT was limited to principles of<br />
metacognition: planning, monitoring, self-evaluation and revision. Other principles about<br />
metacognition might be included for further research.<br />
References<br />
American Psychological Association (1997). Learner-Centered Psychological Principles: A Framework for School<br />
Reform & Redesign. Retrieved on 15/08/2011, from http://www.apa.org/ed/governance/bea/learnercentered.pdf<br />
Anderson, N.J. (2002). The role of metacognition in second language teaching and <strong>learning</strong>. ERIC Digest, April<br />
2002, 3-4.<br />
Azevedo, R., Moos, D. C., Greene, J. A., Winters, F. I. and Cromley, J. C. (2008). Why is externally regulated<br />
<strong>learning</strong> more effective than self-regulated <strong>learning</strong> with hypermedia?, Educational Technology Research<br />
and Development, 56, 45–72.<br />
Carrier, C., Jonassen, D. H. (1988). Adapting courseware to accommodate individual differences. In D. Jonassen<br />
(Ed.), Instructional desings for microcomputer courseware. Mahwah, NJ: Lawrence Erlbaum Associates.<br />
Collis, B. and Meeuwsen, E. (1999) Learning to learn in a WWW-based environment, In D. French, C. Hale, C.<br />
Johnson and G. Farr, (Eds.), Internet based <strong>learning</strong>: Higher education and industry, Sterling, VA: Stylus<br />
Publishing, pp. 25-46.<br />
Flavell, J. H. (1976), Metacognitive aspects of problem solving, In L. Resnick (Ed.), The nature of intelligence,<br />
Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 231-235.<br />
Hartman, H. J. (2001). Metacognition in Learning and Instruction: Theory, Research and Practice. Dordrecht:<br />
Kluwer <strong>Academic</strong> Publishers<br />
Henning, E. (2003). ‘I click, therefore I am (not)’: Is cognition ‘distributed’ or is it ‘contained’ in borderless<br />
eLearning programmes? International Journal of Training and Development, 7(4), 303-317.<br />
Jones, M. G., Farquhar, J. D. and Surry, D. W. (1995). Using metacognitive theories to design user interfaces for<br />
Computer-based <strong>learning</strong>, Educational Technology & Society, 35(4). 12-22.<br />
Pifarre, M. and Cobos, R. (2009), “Evaluation of the development of metacognitive knowledge supported by the<br />
knowcat system”, Education Tech Research Dev, 57, 787-799.<br />
Scraw, G. and Brooks, D. W. (1998), Helping Students Self-Regulate in Math and Sciences Courses:Improving<br />
the Will and the Skill, University of Nebraska-Lincoln, Lincoln, NE, Retrieved on 08/06/2011, from<br />
http://dwb.unl.edu/Chau/SR/Self_Reg.html.<br />
Simons, P. R. J. (1996). Metacognition. In: E. DeCorte and F. E. Weinert, Editors, International encyclopedia of<br />
developmental and instructional psychology, Pergamon Press, Kidlington, pp. 436–441.<br />
Subaşı, G. (2000), “Etkili Öğrenme: Öğrenme Stratejileri”, Milli Eğitim Dergisi, 146, Retrieved on 02/06/2011, from<br />
http://yayim.meb.gov.tr/yayimlar/146/subasi.htm.<br />
Thomas, G. P. and McRobbie, C.J., 2001, Using a metaphor for <strong>learning</strong> to improve students metacognition in<br />
the chemistry classroom, Journal of Research in Science Teaching, 38(2), 222-259.<br />
Tobias, S., Everson, H. and Laitusis, V. (1999). Towards a performance based measure of metacognitive<br />
knowledge monitoring: Relationships with self-reports and behavior ratings. Proceedings from the American<br />
Educational Research Association proceedings on Metacognitive Monitoring in Montreal, Canada.<br />
Retrieved August 10, 2011 from http://www.fordham.edu/gse/faculty/tobias/aera99.html<br />
Tsai, M. J. (2009). The Model of Strategic eLearning: Understanding and Evaluating Student eLearning from<br />
Metacognitive Perspectives, Educational Technology & Society, 12(1), 34-48.<br />
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic <strong>learning</strong>, Journal of Educational<br />
Psychology, 81(3), 329-339.<br />
133
An Exploratory Comparative Study of Distance-Learning<br />
Programmes<br />
Marija Cubric, Karen Clark and Mariana Lilley<br />
University of Hertfordshire, Hatfield, UK<br />
m.cubric@herts.ac.uk<br />
k.clark@herts.ac.uk<br />
m.lilley@herts.ac.uk<br />
Abstract: In this paper we describe an exploratory study of distance <strong>learning</strong> practice. We review five different<br />
distance <strong>learning</strong> programme models from five different schools at the University of Hertfordshire, each varying in<br />
production and presentation. We situate the programmes in an extension of Weller’s pedagogy-technology<br />
space, and we further qualify their pedagogy, using Chickering and Gamson’s principles as a basis for<br />
evaluation. The results of our analysis show that while the flexibility offered to students and economics of<br />
distance <strong>learning</strong> are indeed important drivers for implementation and adoption of distance <strong>learning</strong>, the quality of<br />
teaching and students’ <strong>learning</strong> experience is less well understood and frequently overshadowed by the abovementioned<br />
factors. Moreover, we found that certain principles for ‘good teaching’ become more important than in<br />
the face-to-face scenarios, some principles assume different meaning in distance <strong>learning</strong> situations and new<br />
principles related to effectiveness and ‘affordability’ of on-line communication emerge and gain in importance.<br />
The study aims to help develop a framework for analysis to be a tool for programme planners in a dynamic<br />
education environment. It is already helping in formulating implementation of the ambitious distance <strong>learning</strong><br />
strategy at University of Hertfordshire but can also help other higher education institutions that aspire to provide<br />
quality distance <strong>learning</strong> education in the future as well as in informing other providers of distance <strong>learning</strong><br />
materials and tools.<br />
Keywords: distance <strong>learning</strong>, distance <strong>learning</strong> programmes, distance <strong>learning</strong> pedagogy distance <strong>learning</strong><br />
models, Chickering and Gamson’s principles<br />
1. Introduction<br />
A simple, commonly agreed definition of distance <strong>learning</strong> is that of ‘planned <strong>learning</strong> that occurs in a<br />
different place from teaching…’ (Moore & Kearsley, 1996:2). This definition is broad enough to<br />
include all evolutionary stages of distance <strong>learning</strong> from early correspondence courses to the present<br />
generation based on increased use of the internet. The media used to present <strong>learning</strong> materials has<br />
changed over time and as a consequence so has communication between teacher and learner, with<br />
many more opportunities for shared and collaborative <strong>learning</strong> emerging in the modern age.<br />
In addition to the separation of learner and teacher in space (or time), Sherry (1996) identifies two<br />
more ‘hallmarks’ of distance <strong>learning</strong>: control of <strong>learning</strong> by students and non-contiguous<br />
communication between student and teacher.<br />
We argue that neither of the above characteristics is specific to distance <strong>learning</strong>, nor necessarily<br />
implied by contemporary distance <strong>learning</strong>. Active, student-controlled <strong>learning</strong> is desirable attribute of<br />
any type of <strong>learning</strong>, Also, more and more researchers and practitioners are emphasising the<br />
importance of ‘pacing’ of <strong>learning</strong> materials (Sherry, 1996, Galusha,1997, Cohen, 2002) as well as of<br />
increased teacher ‘presence’ (Garrison, Anderson and Archer, 2000) in distance <strong>learning</strong> processes.<br />
Advancing ICT capabilities such as audio and video conferencing meant from the 1980s onwards<br />
communication no longer had to be asynchronous. More and more synchronous interactions became<br />
feasible through the use of technologies such as Skype, virtual classrooms (e.g. Elluminate) and<br />
similar. Classroom-based courses are also increasingly encompassing asynchronous means of<br />
communication between the teacher and students, through the use of e-mails, chats, discussion<br />
forums etc.<br />
In addition to the medium used for <strong>learning</strong> materials and communication, further distinction can be<br />
made based on the unit of <strong>learning</strong>, that in the context of higher education can be a single module (or<br />
course), degree programme or an entire institution.<br />
Through the rest of this paper we will adopt Moore & Kearsley’s (1996:2) definition in the context of<br />
internet-enabled distance <strong>learning</strong> and we will consider five different distance-<strong>learning</strong> programmes<br />
offered at five different schools at University of Hertfordshire, that vary in scope, production and<br />
presentation.<br />
134
Marija Cubric et al.<br />
The aim of the study is to help develop a framework for analysis to be a tool for distance <strong>learning</strong><br />
programme planners. The main objective is to identify principle dimensions and attributes of distance<br />
<strong>learning</strong> programmes that can be used to compare different programmes and to contextualize them<br />
for evaluation purposes. Another objective is to assess applicability of some of the course (module) –<br />
based evaluation tools, such as Chickering and Gamson’s (1987) principles, in the context of distance<br />
<strong>learning</strong> programmes.<br />
The approach used in the paper is based on semi-structured interviews with programme tutors and<br />
teachers from the five programmes. Programmes were selected based on their match with the<br />
definition of distance <strong>learning</strong> adopted in this paper, and do not include some other approaches<br />
regarded as distance <strong>learning</strong> by the University administration, such as ‘dissertation top-up’<br />
programmes, ‘fly-in faculty’ programmes, or ‘work-based <strong>learning</strong>’ programmes.<br />
Pre-prepared interview questions, included basic programme descriptors, such as number of<br />
students, type of assessment, etc., as well as specific questions on the extent of implementation of<br />
specific Chickering and Gamson’s (1976) principles in teaching and programme implementation.<br />
Those questions were augmented with additional questions that came out as a result of the<br />
interviewees’ responses. These additional responses were ‘normalized’ and sorted into additional<br />
categories, such as, drivers and barriers for staff engagement, effectiveness of distance <strong>learning</strong> and<br />
measures of it, degree of interaction between classroom based (CB) and distance <strong>learning</strong> (DL)<br />
students etc.<br />
The paper is organized as follows. We start with a review of the literature in section 2. This is followed<br />
by a summary of cases (section 3), comparison of cases and relevant discussion (section 4) and<br />
conclusions (section 5).<br />
2. Background<br />
The literature review on distance <strong>learning</strong> (or distance education, as it is sometimes referred to)<br />
reveals a large body of work on issues, barriers, benefits and strategies for implementation. Equally,<br />
it reveals a lack of standards for classifying different modalities of distance <strong>learning</strong>.<br />
Dillenbourg (1999) argues that collaborative <strong>learning</strong> cannot be discussed unless the <strong>learning</strong> is<br />
contextualized: similarly, within distance <strong>learning</strong> the ‘space’ must be defined. In other words, an<br />
object in ‘distance <strong>learning</strong> space’ defines one type of distance <strong>learning</strong> with specific effects that<br />
cannot be automatically generalized to other objects in the space. For example, the effects of<br />
distance <strong>learning</strong> taking place with a small group of part-time local students with a residential tutoring<br />
option cannot be extrapolated to distance <strong>learning</strong> taking place with a massive 100+ globally<br />
distributed cohort. Therefore it is important to understand different defining characteristic<br />
(‘dimensions’) of distance <strong>learning</strong> approaches, in order to contextualize results.<br />
Bélanger and Jordan (2000) provide an overview of the <strong>learning</strong> variables applicable to distance<br />
<strong>learning</strong> that includes <strong>learning</strong> objectives from Bloom’s (1956) cognitive, psycho-motor and affective<br />
domains, interactivity (between learners, and between content and learner), content navigation and<br />
search capabilities of the <strong>learning</strong> materials, and synchronicity of communication.<br />
Weller’s (2002) framework for classifying distance-<strong>learning</strong> courses extends Bélanger and Jordan’s<br />
(2000) set of pedagogical variables with technology-related attributes. This approach is based on the<br />
premise that technology and pedagogy are ‘intertwined’ in any online course, i.e. actively and<br />
iteratively influencing each other. The framework has two dimensions representing the influence of<br />
technology or pedagogy on the course.<br />
The ‘technology’ axis represents the degree of technological sophistication in the design and delivery<br />
of a course. Weller (2002) proposes the following criteria for assessing the technology of a course:<br />
Range of media (audio, video, animation)<br />
Interactive tools such as quizzes, games, simulations<br />
Degree of personalization offered<br />
Sophisticated back-end (tracking progress, logs, annotations)<br />
Web-page design including navigation, interactivity, search<br />
135
Marija Cubric et al.<br />
Web 2.0 tools (wikis, blogs, RSS, sharing of slides, images etc.)<br />
Communication environment to facilitate dialogue e.g. discussion forums, chats, etc.<br />
Additionally, the following more recent net-centric applications (Anderson, 2009) should be added to<br />
the criteria for technical ‘richness’ of a course:<br />
Awareness mechanisms, such as notifications (e.g.RSS), online presence and status updates<br />
(e.g. Twitter) etc.<br />
Tools for supporting virtual communities, based on the ‘wisdom of the crowd’ idea such as wikis,<br />
Digg, Facebook, wePapers, Course Hero, Elgg, Ning, VoiceThread.com etc.<br />
Emerging network-centric applications for aggregating the information and extracting knowledge<br />
(e.g. Slashdot, Omgili)<br />
Mobile <strong>learning</strong><br />
Virtual worlds such as Second Life etc.<br />
Two different poles of the ‘pedagogy’ axis represent a ‘didactic’ or teacher-directed approach, and a<br />
‘constructivist’ or student-centred, collaborative approach. Rather than considering <strong>learning</strong><br />
objectives, Weller uses following criteria for characterising how close is the pedagogy of the course to<br />
either the didactic or constructivist end of the scale:<br />
Focus on content vs. focus on students’ interaction<br />
Assessing retention of content vs. assessing student’s interpretation<br />
Traditional lecture-based teaching vs. ‘conversational’ teaching and active <strong>learning</strong><br />
Teacher as an expert vs. teacher as a facilitator or mentor<br />
Learning as a knowledge acquisition vs. <strong>learning</strong> as a construction of knowledge through social<br />
activity.<br />
The extent to which the above attributes prevail in the course, will determine its proximity to either<br />
side of the pedagogy axis.<br />
The two-dimensional framework results in four different categories of distance <strong>learning</strong>:<br />
Low technology/didactic approach, based around streaming video lectures, and some form of<br />
CMC (computer mediated communication) such as email, suitable for initial adoption and low<br />
investment<br />
Low technology/constructivist approach, includes simple websites with more substantial CMC<br />
capabilities; suitable for small scale university courses in non-technical subjects that involve<br />
discussions and debating e.g. online courses in theology, philosophy, history and similar<br />
High technology/didactic approach, also know as ‘web-based training’, often aimed at individuals,<br />
who may or may not be supported by a tutor; suitable for CPD, professional certifications,<br />
accreditations, life-long <strong>learning</strong> and work-based <strong>learning</strong><br />
High technology/constructivist approach, encompasses virtual environments, different online<br />
spaces that promote collaboration; particularly useful for engineering and scientific subjects.<br />
Weller (2002) adds that the total cost, including production of materials (production cost) and staff<br />
time for course delivery (presentation cost) differs significantly between the four models, with the<br />
lowest cost being associated with low technology/didactic approach and highest with the high<br />
technology/constructivist approach. He concludes that balancing the trade-off between the cost and<br />
technological and pedagogical sophistication of the course is the key factor for success of online<br />
courses.<br />
One interesting observation with cost implications noted by Lozier, Oblinger, and Choa (2002) is that<br />
centralized services are generally used to support development and technology solutions while<br />
responsibility for core academic decisions, including course content, conferring degrees and faculty<br />
workload remains within individual departments.<br />
Similarly to Weller (2002), Cohen (2002) proposes a model for evaluation of distance <strong>learning</strong><br />
courses, based on the combination of pedagogical and technological factors. He further distinguishes<br />
136
Marija Cubric et al.<br />
between different pedagogical criteria, such as: the process of <strong>learning</strong> and teaching, the community<br />
of learners, the role of teacher, the role of students and implementation of the course.<br />
Gaspray, Dardan and Legorreta (2008) consider the effectiveness of distance <strong>learning</strong> through the<br />
‘lens’ of different <strong>learning</strong> theories, such as objectivist, constructivist, collaborative, socio-cultural,<br />
cognitive and computational models of <strong>learning</strong>. They suggest that each of the <strong>learning</strong> models,<br />
implies different meaning of <strong>learning</strong> ‘effectiveness’ i.e. student grades, student satisfaction,<br />
perceived interaction difficulty, perceived flexibility, <strong>learning</strong> climate, perceived knowledge and skill<br />
development respectively. They conclude that distance <strong>learning</strong> has characteristics that are important<br />
to all <strong>learning</strong> models, and while, the first three models are well-understood and widely accepted by<br />
practitioners, the last three need more attention, in order to ‘enable minority perspectives (sociocultural<br />
model) as well as individualized perspectives (cognitive and computational models)’<br />
(Gaspray, Dardan and Legorreta, 2008:58).<br />
This suggests that constructivism in Weller’s framework should be placed somewhere in the middle of<br />
pedagogy axis, while the pole opposite the ‘didactic’ end of the scale should in fact be characterized<br />
as socio-cultural, cognitive and computational.<br />
Sherry (1996) identifies two main models of distance education, based on the starting point or<br />
‘philosophy’ of the design. The ‘Iowa model’ starting point in design is classroom–based teaching, and<br />
here distance <strong>learning</strong> tries to recreate this classroom environment via mediating technologies such<br />
as virtual classrooms, audio-visual interactions etc. Alternatively, the ‘Norwegian model’ starts with<br />
distance teaching that can be computer-mediated and combined with some local ‘face-to-face’<br />
support. A third model, not mentioned by Sherry, is based on the Iowa model but in addition provides<br />
local face-to-face support in a form of residential sessions or local tutoring (branded as ‘tutored e<strong>learning</strong>’<br />
by University of Hertfordshire - http://www.herts.ac.uk/courses/schools-of-study/computerscience/online-courses/supported-e<strong>learning</strong>.cfm).<br />
Beldarrain (2006) explores the affordances of different new technologies, such as wikis, blogs<br />
podcasts etc. in the context of distance <strong>learning</strong>, and suggests the use of Chickering and Gamson’s<br />
(1987) principles as a starting point for defining the purpose and rationale of integrating specific<br />
technology into the distance <strong>learning</strong> curriculum.<br />
Chickering and Gamson’s principles were published in 1987 in a bulletin of the American Association<br />
for Higher Education & Accreditation, as a direct response to a call made by the association for easy<br />
to understand, practical and general principles that would guide further reforms of higher education<br />
and lead to better student experience. The principles state that good practice in undergraduate<br />
education:<br />
Encourages contact between students and faculty<br />
Develops reciprocity and cooperation among students<br />
Encourages active <strong>learning</strong><br />
Gives prompt feedback<br />
Emphasizes time on task<br />
Communicates high expectations<br />
Respects diverse talents and ways of <strong>learning</strong>.<br />
Our choice of the principles as a framework for comparison was based not only on their simplicity and<br />
practicality, but more importantly because they were founded on more than 30 years of research on<br />
how we teach and how students learn.<br />
Research findings especially support the principle of staff contact with students, active <strong>learning</strong> and<br />
delivery of feedback, all three of which are said to have a positive impact on students’ <strong>learning</strong> and<br />
engagement. Critiques of the principles argue that their importance varies across different disciplines,<br />
teaching methods, <strong>learning</strong> styles or organisations (e.g. Sorcinelli, 1991). For example, they are better<br />
suited for humanities and social sciences and are intended for traditional (18-21 year-old) students.<br />
Dalton & Tharp (2002) argue that the principles are incomplete and suggest additional constructivistbased<br />
requirements such as that teacher and students should join in productive activities and that<br />
‘meaning’ should be generated by linking the curriculum to students' lives.<br />
137
Marija Cubric et al.<br />
Despite all the issues, the principles have survived the test of time, and remain a popular tool for<br />
guiding curriculum design (Beldarrain, 2006) and evaluating the quality of <strong>learning</strong> and teaching in<br />
online courses (Graham et al., 2001).<br />
As observed by Merisotis and Phipps (1999) the research literature is focused on modules (or<br />
courses) and lacks in evaluation of distance <strong>learning</strong> programmes. In this paper we start to address<br />
that question.<br />
We use Weller’s framework as a basis for classification of distance <strong>learning</strong> programmes and we<br />
extend it with some other categories that appeared in course of discussions with the interviewees.<br />
Once the programmes are ‘situated’ in this ‘extended’ Weller’s space (Figure 2), we further qualify<br />
their pedagogy, using Chickering and Gamson’s principles as a basis for evaluation and comparison<br />
(section 4).<br />
3. Summary of cases<br />
In this section we summarize (anonymously) details of five programmes from five different subject<br />
areas, based on data collected in December 2009, excluding any subsequent programme changes.<br />
Case 3.1<br />
This MA programme was established in 2007/8 and initially targeted international students though it<br />
also includes a few home students. It is relatively small (less than 20 students). The <strong>learning</strong> and<br />
teaching model of the programme (including assessment) is based on a similar classroom-based<br />
programme (Iowa model) and employs a range of technologies such as (proprietary) MLE, Flickr (for<br />
uploading students’ work), discussion forum, email, Skype and Facebook for discussions and<br />
feedback, as well as recorded guest lectures. Teaching is done via guest lectures and students are<br />
supported in developing projects through frequent interaction with a tutor. Learning effectiveness<br />
measures are not decisive due to the small size of the cohort. The entire programme is supported by<br />
one (enthusiastic) member of staff! The main barriers for further staff engagement are the fear of<br />
extended workload, inadequacy of the current workload model for distance <strong>learning</strong> provision as well<br />
as a doubt among staff members that distance <strong>learning</strong> is an adequate method for <strong>learning</strong> creative<br />
subjects. In terms of Weller’s (2002) classification, the model can be described as low<br />
technology/constructivist approach, with low production cost (as there are no teaching materials<br />
specifically developed for the programme) and high presentation cost, due to the intensity and<br />
frequency of interactions between the students and staff.<br />
Case 3.2<br />
This BSc programme was established in 2004, and since then more than 1,200 students have<br />
enrolled and more than 600 have graduated. The programme runs in two different modalities: online<br />
(Iowa model) and online with local (face-to-face) tutoring support (‘tutored e-<strong>learning</strong>’). The student<br />
population is derived from over 35 countries across the world.<br />
Online students are tutored by UH staff predominantly through the University’s MLE (purpose-built<br />
<strong>learning</strong> materials, discussion forums, blogs, wikis, group work, electronic journals, e-books). The use<br />
of the MLE is supplemented by a suite of applications to support synchronous collaborative work,<br />
presentations, vivas and online tests. Tutored e-<strong>learning</strong> students have access to the same online<br />
facilities and resources but also receive local tutor support from staff at a partner institution.<br />
Independent study is supported through sequenced <strong>learning</strong> activities. The programme’s pedagogical<br />
framework (Pyper, Lilley & Hewitt, 2009) is based on <strong>learning</strong> activities comprised of tasks and<br />
resources and a narrative component to provide rationale for the work.<br />
Although there is no formal interaction between classroom-based and distance learners, some online<br />
<strong>learning</strong> materials are currently being used by classroom-based modules, enhancing the flexibility for<br />
these learners. Learning effectiveness is high compared to similar classroom-based programmes,<br />
with distance learners achieving slightly better grades and expressing higher level of satisfaction with<br />
the course. The retention rate is comparable to similar classroom-based programmes. Drivers and<br />
barriers for staff engagement are related to individual preferences i.e. some staff prefer face-to-face<br />
teaching, others prefer the flexibility offered by distance <strong>learning</strong>.<br />
138
Marija Cubric et al.<br />
Both modalities can be classified as medium to high on the technology scale, and in the ‘middle’ on<br />
the (constructivist) pedagogy scale. The production and presentation cost will also be somewhere in<br />
the middle, with the strong economy of scale effect, as the programme continues to grow.<br />
Case 3.3<br />
This postgraduate programme offers a flexible route, where students can choose the duration<br />
(12/15/18 months) and mode of study (classroom-based or distance <strong>learning</strong>). The student population<br />
consists of more than 70 early career professionals. It is designed as a combination of classroombased<br />
teaching (induction weekend and another extended weekend) with online activities. All<br />
technologies used are based on the (proprietary) MLE extended with additional features for online<br />
delivery, designed by a dedicated member of staff. Pedagogy is content-driven with some elements of<br />
experiential <strong>learning</strong> (activities) and collaborative <strong>learning</strong> (discussions).<br />
Case 3.4<br />
The postgraduate programme is aimed at working practitioners who need to complete specific<br />
professional training. It was established in 1996, with currently more than 150 students enrolled of<br />
whom 50% are home with the rest mainly from EU. It is designed as a combination of distance<br />
<strong>learning</strong> (14 weeks) and residential 3-day workshop/conference after the first six weeks. It can be<br />
classified as the ‘Norwegian’ model. Apart from residential sessions which are based on tutor-led<br />
problem-based and collaborative <strong>learning</strong> activities, tutor support is limited to on-demand contact and<br />
extra support for weaker students. Technologies used are limited to (proprietary) MLE and telephone<br />
communication. Although the <strong>learning</strong> effectiveness data are not provided, students are highly<br />
motivated as they are expected to report back at their workplace. The majority of teaching staff are<br />
visiting lecturers i.e. working practitioners from industry.<br />
Case 3.5<br />
This postgraduate programme is aimed at students who wish to convert a first degree into a<br />
professional qualification. The distance-<strong>learning</strong> route runs in tandem with the classroom-based and<br />
was established in 2007. It is relatively small with around 60 students enrolled about one third of<br />
whom are distance learners based mainly in the UK. It follows the Iowa model: lectures and plenary<br />
sessions are recorded in the classroom and uploaded on the (proprietary) MLE. Initially pedagogy<br />
was didactic, but a staff project established in 2010 was set to move closer to the constructivist end of<br />
the scale. Technologies used include (proprietary) MLE, audio podcast, virtual classroom (rarely),<br />
discussion forums, and email. Achievement and retention for distance-<strong>learning</strong> students is<br />
comparable to that of classroom-based peers. Student satisfaction is hard to administer, but<br />
comments are usually highly positive. All students are enrolled on the same programme and have<br />
access to all materials.<br />
4. Comparison of cases<br />
We start to differentiate between the five programmes (Figure 1) using the following attributes for<br />
comparison:<br />
Level of study<br />
Student population<br />
Years running to-date<br />
Scale (number of students)<br />
Learning and teaching model (Iowa, Norwegian, face-to-face%)<br />
Technologies used<br />
Leading pedagogy i.e. the pedagogical approach used by the majority of modules in the<br />
programme<br />
Learning effectiveness (measured through students’ performance, satisfaction and retention rate)<br />
Interactions between classroom-based (CB) and distance <strong>learning</strong> (DL) students<br />
Drivers and barriers for engaging teaching staff<br />
139
Marija Cubric et al.<br />
Staff profile i.e. permanent (CB), visiting lecturers, specially employed staff etc.<br />
While the first four attributes correspond to simple ‘demographic’ aspects of a programme, others<br />
result either from the reviewed literature (5-7) or emerge as important aspects of DL programmes in<br />
discussions with the interviewees. An additional criterion for differentiation is the subject discipline,<br />
which is not considered in this paper because of anonymised data.<br />
Case Scale/Student<br />
population<br />
3.1 Low/<br />
UK + some<br />
O/S<br />
3.2<br />
3.3<br />
3.4<br />
3.5<br />
Very high/<br />
35 different<br />
countries<br />
Low-medium/<br />
UK<br />
Medium/<br />
UK+ some EU<br />
Medium/<br />
mainly UK<br />
Basic Face-toTechno- Pedagogy Staff Learning<br />
model face % logy<br />
effectiveness<br />
Iowa 0 Low Constructivist Dedicated<br />
to DL<br />
programme<br />
NA<br />
Iowa 0/50% Medium<br />
- high<br />
Iowa Inductio<br />
n day +<br />
Norwegia<br />
n<br />
3 days<br />
3 days<br />
per<br />
module<br />
Middle<br />
between<br />
didactic/<br />
constructivist<br />
Combinatio<br />
n of CB<br />
staff, tutors<br />
at partner<br />
institutions<br />
and visiting<br />
vecturers.<br />
Low Didactic Same as for<br />
CB<br />
Very low Middle<br />
between<br />
didactic/<br />
students<br />
Mainly<br />
visiting<br />
lecturers<br />
Iowa 0 Low Didactic Same as for<br />
CB<br />
students<br />
Better results<br />
and higher<br />
satisfaction<br />
than CB<br />
students<br />
NA<br />
Highly<br />
motivated<br />
students<br />
Comparable<br />
results; CB<br />
students<br />
switching to<br />
DL mode<br />
Figure 1: Comparison of distance <strong>learning</strong> programmes<br />
While the programmes clearly differ in scope, use of technology and pedagogy, they each offer<br />
specific value to a specific, target population of learners: continuous conversation guiding<br />
development of student’s practice (3.1), focus on self-study for professional qualifications (3.3 and<br />
3.4), support for large diverse cohort of international students (3.2) and integrating distance learners<br />
with CB students into a more cohesive <strong>learning</strong> community (3.5).<br />
By comparing and contrasting the pedagogy and technology of programmes under investigation, we<br />
are able to situate each programme in Weller’s distance <strong>learning</strong> space (Figure 2), extended with new<br />
dimensions such as subject discipline, scope (number of students) and Sherry’s (1996) distance<br />
<strong>learning</strong> categories, represented by shapes of different size in Figure 2.<br />
In the rest of this section, we continue to differentiate between programmes’ pedagogies using<br />
Chickering and Gamson’s (1987) principles as a framework for discussion. To refer to a specific case,<br />
we use the corresponding section number i.e. 3.1, 3.2 etc.<br />
4.1 Good practice encourages contact between students and faculty<br />
Examples supporting this principle include: daily staff availability via emails (all cases), Facebook chat<br />
(3.1), tutor’s feedback on Flickr uploads (3.1), tutor engagement in discussion forum (3.3), personal<br />
tutoring via e-mail, phone or chat (3.3), and use of Skype/Elluminate/Facebook/individual online work<br />
areas (3.2).<br />
In case 3.4, a three-day residential session is setup as a conference, providing formal and informal<br />
interactions. As learners are adults, there are fewer barriers between students and teachers.<br />
Additional social activities are organised in case 3.5, where locally-based DL students are invited to<br />
public formal and social events with staff.<br />
140
Marija Cubric et al.<br />
4.2 Good practice develops reciprocity and cooperation among students<br />
This principle is supported with mechanisms for awareness (‘see who is online’), use of (online)<br />
discussion forums (3.1), discussion group ‘meeting rooms’ (3.2) etc.<br />
Figure 2: Distance <strong>learning</strong> programmes space<br />
In case 3.3, students are encouraged to post on discussion boards, with some less frequent use of<br />
wikis and blogs.<br />
Residential workshops offer opportunities for collaboration and cooperation – some assessment in<br />
case 3.4 is a group-based workshop: students are purposely not introduced, encouraging them to find<br />
out about their colleagues.<br />
In case 3.5, students are asked to post an online biography to create a sense of community. Group<br />
formative assessments require collaborative activities between CB and DL students.<br />
Although the literature emphasises the importance of the social component of distance <strong>learning</strong><br />
(Garrison, Anderson, & Archer, 2000) , it can be difficult to achieve (at least synchronously) in cases<br />
where the majority of students on programme do not share same time zone (3.2).<br />
4.3 Good practice encourages active <strong>learning</strong><br />
In case 3.1, the tutor indicates various (online) information sources required for a live project.<br />
In case 3.2 this is facilitated through a pedagogical framework (Pyper, Lilley & Hewitt, 2009) that<br />
guides tutors in designing various online tasks to keep students active.<br />
Case 3.4 is based on extensive self-directed study periods before and after the residential session.<br />
In case 3.5, online quizzes offer structured feedback and encourage cooperation between learners,<br />
discussing particular questions or answer rationale etc<br />
4.4 Good practice gives prompt feedback<br />
Examples include: feedback on Flickr images (3.1), annotated essays or sample answers after the<br />
residential session in case 3.4, response to email enquiries and use of audio for feedback i.e.<br />
recorded comments in case 3.5.<br />
141
Marija Cubric et al.<br />
In case 3.2, a ‘service level agreement’ is established regarding the expectations of staff and<br />
students’ engagement with discussion forums and emails. In addition to that, automated tools,<br />
annotations and sample answers are used to provide feedback.<br />
4.5 Good practice emphasizes time on task<br />
This principle was not considered in cases 3.1 and 3.4, where students are expected to conduct selfdirected<br />
study, supported by subject tutors, (when requested). In case 3.5, module guides specify<br />
session outlines, including formative activities planned for the session. More granular time on task is<br />
implemented in cases 3.2 and 3.3, where students are asked to complete weekly individual or group<br />
tasks with deadlines, using variety of methods and technologies (proprietary MLE groupware,<br />
Facebook etc). In addition, in case 3.3 all units are broken into timed tasks.<br />
4.6 Good practice communicates high expectations<br />
According to most interviewees, the implementation of this principle relies on the type of <strong>learning</strong><br />
activities used in different modules. For example, in case 3.1, students are expected to work on a ‘live<br />
project’ and to publish the results of their work (as well as work in progress) on a public website.<br />
4.7 Good practice respects diverse talents and ways of <strong>learning</strong><br />
With regards to this principle, various programmes offer various approaches. In case 3.1, programme<br />
tutor acts as a personal tutor, and provides one-to-one support to individual students. Students are<br />
accepted based on the portfolio and set of <strong>learning</strong> objectives that they want to achieve. The<br />
programme does not provide teaching of techniques but instead aims to support students in<br />
developing individual practice.<br />
In case of 3.2, from its inception, the programme was designed to support diversity and flexibility, by<br />
offering two modalities of distance <strong>learning</strong> (with or without face-to-face contact).<br />
In case 3.3, students are offered flexibility not only in where and how they learn, but also in the<br />
duration of their studies (12/15/18 months). In case 3.5 students can choose to study wholly or partly<br />
by distance over one or two years.<br />
In cases 3.2 and 3.4 assessment variety helps support different types of learners.<br />
4.8 Discussion<br />
The principles used for comparison are useful in situating the pedagogy of different DL models,<br />
however they seem to mean different things to different people. These differences seem to have been<br />
even greater across programmes.<br />
Weller (2002), Chickering and Gamson (1987), and indeed most evaluation studies, concentrate on<br />
specific courses (e.g. Graham, 2001; Buckley, 2003) or usability of technological solutions (e.g.<br />
Tselios et al, 2001). This, perhaps inevitably, means that the usefulness and applicability of some of<br />
the principles varies when extrapolated for DL programme analysis.<br />
Based on the data from our study, it may be anticipated that some principles are usable at<br />
programme level (e.g. 4.1, 4.4 and 4.7) while others are perhaps expected to be more<br />
module/teacher-centric (4.3, 4.5, 4.6) or cohort-dependent (4.2).<br />
Amongst the ‘programme-level principles’, principle 4.1 (‘staff-student contact time’) and 4.4 (‘prompt<br />
feedback’) could be formulated within the programme service level agreement, while 4.7 may be<br />
answered by the intrinsic flexibility of distance education.<br />
Although Weller’s framework was intended for classifying online modules, it can be applied equally<br />
well in the context of distance <strong>learning</strong> programmes, as they tend to exhibit higher ‘cohesion’ with<br />
respect to pedagogies and technologies used across different comprising modules (Lozier, Oblinger,<br />
and Choa, 2002). Indeed the experience in case 3.5 is that this consistency is not just a feature but a<br />
requirement for student satisfaction in a DL model at the didactic end of the spectrum.<br />
142
5. Conclusions<br />
Marija Cubric et al.<br />
Comparing distance <strong>learning</strong> programmes is clearly non-trivial; for example, different pedagogies<br />
imply different meanings for <strong>learning</strong> effectiveness, as suggested by Gaspray, Dardan and Legorreta<br />
(2008). There are also issues with gathering and interpreting appropriate primary data, which our<br />
own experience highlights; the design of different research instruments – for example, a questionnaire<br />
to be consistently administered across programmes – could be considered. It is very difficult to draw<br />
definitive conclusions from the current data analysis under Chickering and Gamson (1987), although it<br />
has been possible to identify which of their principles are useful at programme level. it is easy to see<br />
the value of an overarching framework for evaluation not least at the design and implementation stage<br />
of programmes. Such a framework could provide a mix of quantitative and qualitative data to support<br />
stronger conclusions in the future.<br />
So, the value of this work is in sketching such a framework for analysis to be a tool for programme<br />
planners in a dynamic education environment. In particular, one of its key strengths is the combination<br />
of technological and pedagogical axes – the literature indicates that historically there has been an<br />
over-emphasis on technology and this framework offers an integrated way to move forward. This<br />
framework, including accessibly-formulated principles also offers a chance for programme planners to<br />
expound a practical pedagogic rationale to staff who may well be, as many have noted, reluctant to<br />
reflect on their own teaching practice.<br />
References<br />
Anderson T (2009) ALT-C 2009 - Keynote Speech on 10 September 2009, Available online at:<br />
http://www.alt.ac.uk/altc2009/keynotes.html [07/06/2011]<br />
Bélanger, F. and Jordan, D.H. (2000) Evaluation and implementation of distance <strong>learning</strong>: technologies, tools,<br />
and techniques. UK: Idea Group Publishing<br />
Beldarrain Y. (2006) ‘Distance education trends : integrating new technologies to foster student interaction and<br />
collaboration’, Distance Education 27(2) 139-153<br />
Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David<br />
McKay Co Inc.<br />
Boyed-Barrett O (1999) ‘European and North American Models of Distance Education’, The Cal Poly Pomona<br />
Journal Of Interdisciplinary Studies<br />
Bullen M. (1999) ‘What’s the Difference?: A Review of Contemporary Research on the Effectiveness of Distance<br />
Learning in Higher Education (Authors: Jamie P. Merisotis and Ronald A. Phipps) ‘, Journal of Distance<br />
Education, 14:1, 102-114<br />
Buckley K.M., (2003) ‘Evaluation of classroom-based, Web-enhanced, and Web-based distance <strong>learning</strong> nutrition<br />
courses for undergraduate nursing’, Journal of Nursing Education, 42(8):367-70<br />
Cohen , V.L. (2003) ‘A model for assessing distance <strong>learning</strong> instruction’, Journal of Computing in Higher<br />
Education, 14(2), 98-120<br />
Chickering, A. W., and Gamson, Z. F. (1987) ‘Seven Principles for Good Practice in Under- graduate Education.’<br />
AAHE Bulletin, 39(7), 3–7<br />
Dalton S.S. & Tharp R.G. (2002) ‘Standards For Pedagogy: Research, Theory And Practice’, In Learning for life<br />
in the 21st century: Sociocultural perspectives on the future of education. G. Wells & G. Claxton (Eds.)<br />
Oxford: Blackwell, 181-194<br />
Dillenbourg P. (1999) ‘What Do You Mean By Collaborative Learning?.’, In P. Dillenbourg (Ed) Collaborative-<br />
Learning: Cognitive And Computational Approaches, 1-19. Oxford: Elsevier<br />
Galusha J. (1997) ‘Barriers to <strong>learning</strong> in distance education’, Interpersonal Computing and Technology, 1997<br />
Garrison, D. R., Anderson, T., & Archer,W. (2000) ‘Critical Inquiry In A Text-Based Environment: Computer<br />
Conferencing In Higher Education’,The internet And Higher Education, 2(2-3), 87-105<br />
Gaspay A., Dardan S., Legorreta L. (2008) ‘Distance <strong>learning</strong> through the lens of <strong>learning</strong> models’, Review of<br />
Business Research, 8(4)<br />
Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy T. M. (2001) ‘Seven principles of effective teaching: a<br />
practical lens for evaluating online courses’, The Technology Source Archives at the University of North<br />
Carolina. Available online at: http://bit.ly/atSX7<br />
Lozier, G., Oblinger, D., Choa, M. (2002) ‘Organizational Models for Delivering Distance Learning’, Educause<br />
Center for Applied Research Research Bulletin, 2002(2)<br />
Merisotis, J.P., Phipps, R.A. (1999) ‘What’s the Difference?: A Review of Contemporary Research on the<br />
Effectiveness of Distance Learning in Higher Education’, Washington, DC: The Institute for Higher<br />
Education Policy<br />
Pyper, A., Lilley, M. Hewitt, J. (2009) ‘A Framework to Support Students in Their Individual Studies’, In<br />
Proceedings of the First International Conference on Computer Supported Education, Lisboa, Portugal,<br />
March 23-26, 2009 - Volume 1 2009<br />
Sherry, L. (1996) ‘Issues in Distance Learning’, International Journal of Educational Telecommunications, 1(4)<br />
337-365<br />
143
Marija Cubric et al.<br />
Sorcinelli, M. D. (1991) ‘Research Findings on the Seven Principles’, New Directions for Teaching and Learning,<br />
47<br />
Tselios N., Avouris N., Dimitracopoulou A., Daskalaki S. (2001) ‘Evaluation of Distance-<strong>learning</strong> Environments:<br />
Impact of Usability on Student Performance’, International Journal of Educational Telecommunications,<br />
7(4), 355-378<br />
Weller, M. (2002) Delivering Learning on the Net, Routledge_Farmer<br />
144
The Optimal Teaching Style Based on Variability of Study<br />
Materials<br />
Blanka Czeczotková, Kateřina Kostolányová and Jana Šarmanová<br />
Ostravská univerzita, Ostrava, Česká republika<br />
D09434@student.osu.cz<br />
Katerina.Kostolanyova@osu.cz<br />
Jana.Sarmanova@osu.cz<br />
Abstract: Teaching by standard eLearning is being gradually replaced by a new form - personalized eLearning.<br />
Personalized eLearning is understood as not only an instruction tailored to each student according to his<br />
characteristics, but it is also adaptable according to the actual conditions under which the <strong>learning</strong> takes place.<br />
Pilot testing of students and subsequent analysis determined a group of student´s characteristics to which the<br />
eLearning study environment can be adapted. These characteristics must be put into accord with forms and<br />
variants of created <strong>learning</strong> materials. This paper deals with the assignment of an appropriate method of <strong>learning</strong><br />
management to students' individual <strong>learning</strong> styles.<br />
Keywords: adaptive eLearning, virtual teacher, teaching styles, <strong>learning</strong> styles, teaching methods<br />
1. Introduction<br />
Learning of an individual is affected by many factors - in particular by student <strong>learning</strong> styles, their<br />
personalities and motivation to learn. From this perspective, each student can be seen as an<br />
individual. During a standard instruction, the student is a part of a team and the teacher adapts<br />
teaching to suit the majority of students in the classroom. An experienced teacher notices during faceto-face<br />
instruction when the students lose their train of though.(Barker, 2005,2009)<br />
Let us move from the conventional instruction in a classroom to electronic education. Electronic<br />
education requires the teachers prepare all study materials for the students and organizational<br />
guidelines in order to minimize student's discomfort when following the course. Of course there is the<br />
possibility of consultations, either in person or by means of communication technologies (telephone,<br />
e-mail, chat, etc.) If such a kind of instruction is supported by information and communication<br />
technologies (ICT), we are talking about eLearning, managed by a Learning Management System<br />
(LMS). Even in such a case the <strong>learning</strong> process is not adapted to the specificity of each individual.<br />
The curriculum is presented to students in an integral coherent form, with the course walk through<br />
exactly planned. However, not every student may find such prepared procedures convenient.<br />
When we upgrade eLearning to the next quality level, we get to the personalized eLearning. Such an<br />
environment is predestined to consider each student’s individuality.(Brusilovský, 2001)<br />
By personalised eLearning we mean an automated way of teaching which adapts to different qualities<br />
of students which are characteristic for their <strong>learning</strong> styles.<br />
In case students learn without a direct contact with a teacher, they usually use textbooks. A good<br />
textbook can be regarded as another form of “a teacher”. Its author has put in it his optimal<br />
instructional approach, scope and detail of the information presented. It is known that the classic<br />
textbooks supplement direct interpretation given by a teacher, whereas textbooks for independent<br />
<strong>learning</strong>, so-called “distance education”, are supposed to substitute not only an interpretation of a new<br />
topic itself, but also the communication with the teacher, practicing and reviewing of the content, etc.<br />
To create an adaptive eLearning environment that will ensure a personalized (individual) approach for<br />
each student, we need to provide study materials processed in different variants.<br />
The system we have designed may be illustrated by the following schema (see Figure 1: Adaptive<br />
eLearning system schema).(Šarmanová, 2009)<br />
145
Figure 1: Adaptive eLearning system schema<br />
2. Adaptive eLearning system<br />
2.1 Student module<br />
Blanka Czeczotková et al.<br />
The basic presumption for the proper functioning of the adaptive system is a student who has been<br />
appropriately and properly tested. To create a tuple of his basic characteristics, each student must be<br />
a subject of continuous testing at several levels.<br />
The most important characteristic is the student's <strong>learning</strong> style. Learning style consists of a student´s<br />
individual features and characteristics, bearing a significant impact on the <strong>learning</strong> process. This set<br />
of characteristics is obtained before the study commenced, using an appropriate questionnaire.<br />
Further testing is carried out during the study and focuses mainly on the level of his current<br />
knowledge. A final set of characteristics is obtained by a long-term monitoring of the student´s study<br />
activities. This set of characteristics may contribute not only as a feedback during the current <strong>learning</strong>,<br />
but also can be used for an adjustment of the characteristics of the student, or to monitor his<br />
development.(Dunn, 2004; Entwistle, 1996; Gregorc, 1979).<br />
The choice of suitable student characteristics that we will observe was carried out on the basis of an<br />
initial analysis of several well-known and verified studies that focused on the student <strong>learning</strong> styles.<br />
We have selected such characteristics that can be taken into account when creating an eLearning<br />
environment. (Kostolányová, 2010a, 2010b). The analysis of the forecasting of the characteristics of<br />
students is described in more detail (Takács, 2009 ). The student analysis was conducted at 400 high<br />
schools and universities. Those are the following:<br />
Sensual perception: visual - auditive - kinaesthetic - verbal;<br />
Social aspects: likes to work alone - in a pair - in a group;<br />
Affective aspects: motivation to study - internal and external;<br />
Learning tactics:<br />
Systematic approach : studies and solves problems systematically, according to the instructions,<br />
needs regulations/ heuristic style (requires freedom, solves problems creatively);<br />
Method: prefers theoretical derivation / experimentation;<br />
Learning approach: perception of details / holistic;<br />
Learning tactics: deep - strategic – surface;<br />
Self-regulation: high, the student is able to organize his own <strong>learning</strong> low, needs teacher’s<br />
guidance.<br />
2.2 Author module<br />
A possible approach to create the variable supports may be when the author creates several books to<br />
teach the same topic, applying, however, each time a different teaching style. Such a method is not<br />
very suitable. First, the author with his own distinctive style of teaching may find it difficult to use a<br />
146
Blanka Czeczotková et al.<br />
number of quite different <strong>learning</strong> styles. The second reason is the fact that there are too many<br />
students with various <strong>learning</strong> styles.<br />
2.3 Three bases for the creation of adaptive <strong>learning</strong> supports<br />
When designing the methodology for creating adaptive <strong>learning</strong> materials, we have based our<br />
approach on general principles of teaching and <strong>learning</strong> and teaching methodology of distance<br />
education textbooks.<br />
The principles were adapted from J. A. Komenský,(Lánský, 1997) the events of instruction by Gagne<br />
(Gagné, 1975) and the Bloom's (Schaink, 2002) taxonomy of <strong>learning</strong> objectives. As few other works<br />
by renowned scholars werw also referred to.<br />
J. A. Komenský (Čapková, 1992) defined a number of pedagogical principles in his work that need to<br />
be applied so that the process of teaching is carried out successfully and efficiently. Out of these<br />
principles, we have selected the following five that are considered to be the most important: the<br />
principle of sense-impression, awareness, steady and systematic approach, adequacy and durability<br />
(see Table 1: Application of the principles by J. A. Komensky when creating an adaptable textbook).<br />
Table 1: Application of the principles by J. A. Komensky when creating an adaptable textbook<br />
principle way of application in an adaptable textbook<br />
senseimpression<br />
Using multimedia components and objects<br />
awareness Different ways of motivating, demonstrations of practical application of the learned matter<br />
steady and<br />
systematic<br />
approach<br />
Linking to the student’s prior knowledge, sequential and systematic <strong>learning</strong><br />
adequacy Content and extent of the curriculum according to the current levels of knowledge and<br />
competency<br />
durability Goals determination according to the study programme<br />
The instruction generally consists of <strong>learning</strong> (the activity of the student) and teaching (the activity of<br />
the teacher), and is carried out within an educational environment (in this case an electronic<br />
environment).<br />
Let us summarize the facts that are known to us: we know the student’s <strong>learning</strong> style (tested before<br />
the instruction itself).(Šarmanová, 2010; Takács, 2010) The student will be tested during the course of<br />
instruction (feedback of the presented content and its comprehension) and generally we are trying to<br />
create a user friendly environment (for the student not to feel helpless and isolated). The <strong>learning</strong><br />
environment´s ability to support a democratic way of teaching is still to be resolved. The democratic<br />
climate gives support to the independency development, self-control, planning and responsibility. All<br />
of these features are welcome and important in an electronic environment. Changing different<br />
teaching methods according to the student’s present mental state and his preferred <strong>learning</strong> style<br />
comes as an option. This area of adaptive environment design will be handled by adaptive algorithms,<br />
designed through cooperation of teams of psychologists, pedagogues and informatics.<br />
A necessary condition for the implementation of these algorithms is to structure the study material into<br />
small parts. They will be arranged and offered to the student based on the algorithms in concorde with<br />
his preferred <strong>learning</strong> style.<br />
Teaching methods consist of a sequence of elementary steps of instruction. In the process of<br />
instruction, there is a certain order in the sense of procedure: the beginning of instruction, lecturing,<br />
exercising, testing and ending of the process. To be able to apply this principle in the field of<br />
adaptable teaching, we based our efforts on the distribution of the <strong>learning</strong> process according to R.<br />
Gagné. On the basis of this principle, we have assigned to each teaching activity a subset of adaptive<br />
support - so called layers. The following table shows the distribution (see Table 2: Gagné’s theory<br />
applied on an adaptable study support).<br />
For the designed methodology to be as general and as universal as possible, we have also included<br />
the theory of Bloom's taxonomy of <strong>learning</strong> objectives. According to this theory, in adaptive <strong>learning</strong><br />
materials there are <strong>learning</strong> objectives defined with an emphasis on the cognitive domain of<br />
147
Blanka Czeczotková et al.<br />
knowledge with regard to curriculum level of detail and with regard to the particular subject. For<br />
example, the content of a course teaching office applications will be different for potential<br />
administrative staff from the one for analysts or programmers. Therefore, the objectives are<br />
formulated on the basis of the character of the taught content for 6 categories – to remember,<br />
understand, apply, analyse, evaluate, and create.<br />
Table 2: Gagné’s theory applied on an adaptable study support (Kostolányová, 2011)<br />
Event (according to the Gagné’s theory) Layer of the framework in an adaptive support<br />
Gain attention Motivational layer<br />
Inform learners of objectives Goals layer<br />
Stimulate recall of prior <strong>learning</strong> Entry knowledge testing, eventually demands of the framework<br />
Present the content Theoretical layer<br />
Provide „<strong>learning</strong> guidance“ Semantic layer<br />
Elicit performance (practice) Practical examples<br />
Provide feedback Auto testing – a layer of questions and tasks<br />
Assess performance Test results, key to unsolved tasks<br />
Enhance retention and transfer to the job Fixation layer<br />
When considering the features of teaching adaptability, we based our efforts on the methodology of<br />
creating distance <strong>learning</strong> materials. This is where the idea of dividing the study material into<br />
individual, smaller units comes from. The curriculum of the subject was divided into chapters and<br />
subchapters. Within subchapters, we call the thematically coherent elementary parts frameworks.<br />
A framework is the basic educational unit, which explains one sub-theme. And it is the framework we<br />
focus on most in structuring an adaptable textbook and to which more variants of instruction will be<br />
designed.<br />
Let us sum up the basic information on the author module. When designing an adaptive textbook, we<br />
build on the methodology used to create textbooks for distance education. The study material is<br />
divided into sub-chapters, called frameworks, and these are divided into layers representing particular<br />
instruction phases. As the most obvious characteristics of the <strong>learning</strong> style we have accepted the<br />
type of sense perception, and another quality is the depth of understanding. We chose two basic<br />
criteria for the creation of different variants of frameworks – the form of instruction for the student's<br />
sensory perception type and depth for the level of curriculum detail. There will be four sensory<br />
variants: one with a high amount of text (for the verbal type of student), with many pictures, graphs,<br />
tables, animations (for the visual type), with spoken word, audio recordings, communication activities,<br />
discussions (for the auditive type) or creative tasks, constructions, etc. (for the kinaesthetic type).<br />
The depth of instruction will be represented by three levels:<br />
Basic depth (which is used as the primary) contains the entire basic curriculum, the student is<br />
required master.<br />
Detailed depth for students who would not understand the curriculum presented in basic depth, a<br />
modified version (more detailed, from a different angle) will be ready.<br />
Extended depth for students with high motivation to the given topic, the variant will be enriched<br />
with matters of interest, peculiarities of the topic.<br />
3. Linking student´s module and the author´s module<br />
Now there is a question to be answered which teaching style best suits to each student. To be more<br />
specific which frameworks (sensory perception types of the student) and with layers (enhance,<br />
standard, detailed) will be the most suitable for the particular student. Let us have a look at what<br />
148
Blanka Czeczotková et al.<br />
reputable pedagogues say about teaching styles. Different pedagogues provide different definitions<br />
specifying the term “teaching”.<br />
J. Mares in his book Students Learning styles inclines to a definition by V. Kulic, who defines <strong>learning</strong><br />
as a process during which a man changes his complex of knowledge of natural and human<br />
environment, changes his forms of behaviour and procedures, characteristics of his personality and<br />
self-image. A man changes his relationships with people around him and society he lives in – and all<br />
of these towards development and greater efficiency. The fore mentioned changes are made mainly<br />
on the basis of experience, i.e. the results of previous activities, which are transformed into systems<br />
of known information - the knowledge. These experiences can be individual or assumed and adopted<br />
from the social experience.(Mareš, 1998)<br />
Teaching methods can be viewed from different perspectives - from a psychological perspective,<br />
procedural perspective, <strong>learning</strong> phase perspective, logical and organizational aspects perspectives.<br />
Currently the so-called activation methods have become popular, which include - discussions,<br />
<strong>learning</strong> games, the project method, the experiential <strong>learning</strong>, and others. Teaching methods are also<br />
divided according to the nature of the information sources. This division will be further dealt with.<br />
Among the best known teachers who have dealt with the division of teaching methods, according to<br />
the nature of the information sources were I. J. Lerner, Z. Pesek, J. Manak, D. O. Lordkipanidze, S.<br />
Vrana, E. Stracar M. Korinek, M. A. Danilov, B. P. Jesipov, O. Kadner, I. A. Kairov, A. Valisova, J.<br />
Valenta, E. J. Golant ...<br />
Some of the pedagogues were inspired by their colleagues and based their work on the already<br />
existing types of teaching methods. In 1938 D. O. Lordkipanidze divided teaching methods according<br />
to the nature of the information sources to-methods of verbal communication (monologic and dialogic<br />
methods), methods of work (practical work, lab work, ideal tasks of exploratory character) and<br />
methods of observation (demonstration, image and film observation, excursional demonstrations,<br />
illustrations, etc.). In 1959 S. Vrána divided teaching methods according to the nature of the<br />
information sources to: verbal methods (lecture, meeting, working with a book, written assignments,<br />
seminar method, and interview), visual methods (demonstration, observation) and practical methods<br />
(laboratory work, technical works, practical works, working with text, etc.). And so we can continue<br />
with a long list of the same teaching methods divided according to the nature of the information<br />
sources, until we get to J. Manak. J. Manak in 1967 divided the teaching methods into various<br />
aspects. The didactic division includes verbal methods (monologic, dialogic, methods of written work,<br />
methods of working with a textbook, a book, a text, a material), illustrative and demonstrative methods<br />
(observation of objects and phenomena, demonstrations, static images demonstration) and practical<br />
methods (physical training and working skills training, laboratory student activities, students, working<br />
activities, graphic and crafts activities).After mapping out the work of most pedagogues, we have<br />
decided to base our theory on J. Manak´s division of teaching methods.<br />
4. Rules for assigning a teaching style to the student individuality<br />
To keep a good orientation among the amounts of described views, classifications, types of teachers,<br />
and to be able to work with those teaching methods and <strong>learning</strong> styles, we have designed a table as<br />
a suitable working tool (see Figure 2: Linkage between the teacher and student characteristics and<br />
teaching methods). There are 3 columns containing teaching methods, student characteristics and<br />
characteristic features of teaching styles. Through an analysis of pedagogical principles and student<br />
characteristics, we have designed the assignment of appropriate teaching methods that best suit the<br />
various types of students. Figure 2 shows only a partial result of the whole analysis outcome.<br />
The result of this analysis is a set of rules of a following structure:<br />
If the student possesses (<strong>learning</strong>) characteristic ABC,<br />
then it is appropriate to use teaching methods XYZ and<br />
<strong>learning</strong> management type UVW.<br />
This set of rules will be used by the so-called virtual teacher, which is a <strong>learning</strong> management<br />
program, integrated into the adaptive LMS. The virtual teacher first assigns a teaching style<br />
appropriate to the student <strong>learning</strong> style, and then will follow this style and adapt the interaction with a<br />
student within the eLearning instruction.<br />
149
Blanka Czeczotková et al.<br />
Figure 2: Linkage between the teacher and student characteristics and teaching methods<br />
150
5. Conclusion<br />
Blanka Czeczotková et al.<br />
This paper deals with an analysis of existing teaching methods and teaching styles and their optimal<br />
choice for students with various <strong>learning</strong> styles.<br />
After the mapping of all teaching methods, we eliminated such teaching methods that are identical or<br />
very similar. We also unified analogical characteristics of teaching styles. Through another research<br />
student characteristics were assigned with appropriate forms of teaching methods and teaching<br />
styles, in order to be adaptable to each individual student while at the same time being usable in<br />
eLearning.<br />
The result of our efforts is a series of rules, assigning an optimal teaching style to a particular student.<br />
These rules will be the basis for the construction of several algorithms known as the virtual teacher, a<br />
<strong>learning</strong> management program, capable of adapting instruction to each student.<br />
Acknowledgments<br />
This publication is supported by the ESF project OP VK CZ.1.07/2.3.00/09.0019 titled Adaptive<br />
individualization of study through eLearning.<br />
References<br />
Barker, P. (2005), Knowledge management for eLearning , Innovations in Education and Teaching International.<br />
Vol. 42, No. 2, pp. 111-121.<br />
Barker, P., Schaik, P. (2009) Electronic Performance Support: Using Digital Technology to Enhance Human<br />
Ability, London, Gower Publishing.<br />
Blinov, V. M. (1976), Effektivnost´ obučenija. Moskva: Pedagogika, 1976.<br />
Brusilovsky, P., Kommers, P., Streitz, P. A. M., Berlin, N. (1996), Multimedia, Hypermedia and Virtual Reality;<br />
Models, Systems and Applications, Springer Verlag.<br />
Brusilovsky, P. (2001) Adaptive hypermedia. User Modeling and User Adapted Interaction, Ten Year Anniversary<br />
Issue (Alfred Kobsa, ed.) 11 (1/2), 87-110<br />
Dostál, A. M. (1959) Vyučovací metody. Praha: Krajský pedagogický ústav, 1959.<br />
Dunn, K., Dunn, R., Price, G.E. (2004), Dotazník stylů učení, Praha, Institut pedagogicko-psychologickeho<br />
poradenstvi.<br />
Entwistle, N., Tait, H. (1996) Approaches and Study Skills Inventory for Students, Centre for Research on<br />
Learning and Instruction, University of Edinburgh.<br />
Gagné, R. Podmínky učení. Praha: SPN, 1975.<br />
Gardner, H. (1999), Dimenze myšlení: teorie rozmanitých inteligencí, Portál, Praha.<br />
Gregorc, (1979) A. Learning/Teachins Styles: Their Nature and Effects, Reston, Christian Education Journal, Vol<br />
4, No.1, pp 62.<br />
Kolb, D. A. (1984) Experiental <strong>learning</strong>: Experience as the source of <strong>learning</strong> and development, Engelwood Cliffs,<br />
NJ, Prentice Hall.<br />
Čapková, D. Učiteľ učiteľov: J.A. Komenský a učiteľská profesia. Bratislava: SPN, 1992.<br />
Kostolányová, K. (2011). Design of study materials structure for adaptive instruction. ICTE2011;Ostrava.<br />
Kostolányová, K., Šarmanová, J., Takács, O. (2010a). Intelligent Tutoring in ELearning, ECEL2010, Porto, pp.<br />
794-796.<br />
Kostolányová, K., Šarmanová, J., Takács, O. (2010b), Adaptive individualized education in eLearning.<br />
CBLIS2010, Warsaw, Poland, pp. 119-125.<br />
Kucharev, N. V. (1979) , Effektivnost´ obučenija i vospitanija. Minsk: Izdatel´stvo BGU, 1979.<br />
Kulič, V. (1992), Psychologie řízeného učení. Praha: Academia, 1992. ISBN 80-200-0447-5.<br />
Lánský, M. (1997), Didaktická technologie ve světě idejí J. A. Komenského. In Technologické otázky vzdělávání.<br />
Praha: KAVA-PECH, 1997. ISBN 80-85853-28-0.<br />
Lerner, I. J. (1986), Didaktické základy metod výuky. Praha: Státní pedagogické nakladatelství, 1986.<br />
Maňák, J. (2003), Výukové metody. Brno: Paido, 2003. ISBN 80-7315-039-5.<br />
Mareš, J. (1998), Styly učení žáků a studentů. Praha: Portál, 1998. ISBN 80-7178-246-7 pp. 47<br />
Mojžíšek, L. (1988), Vyučovací metody. Praha: Státní pedagogické nakladatelství, 1988.<br />
Schaik, P., Pearson, R., Barker, P., (2002), “Designing Electronic Performance Support Systems to Support<br />
Learning”, Innovations in Education and Teaching International, Vol.39, No.4, p. 289-306.<br />
Šarmanová, J., Kostolányová, K., Takács, O. (2010), The use of adaptive individualized eLearning at teaching,<br />
IMSCI 2010, Florida, USA, pp. 147-152.<br />
Takács, O., Kostolányová, K., Šarmanová, J. (2009), Results of analysis of <strong>learning</strong> styles. ICTE2009, Ostrava.<br />
pp. 205-210.<br />
Takács, O., Kostolányová, K., Šarmanová, J. (2010), Adaptable educational supports. ICTE 2010, Ostrava, pp.<br />
117-121.<br />
151
Changing <strong>Academic</strong>s, Changing Curriculum: How<br />
Technology Enhanced Curriculum Design can Deliver<br />
Strategic Change<br />
Christine Davies<br />
University of Glamorgan, Pontypridd, UK<br />
cpdavies@glam.ac.uk<br />
Abstract: This paper describes a case study resulting from a JISC Building Capacity Project at the University of<br />
Glamorgan. The case study indicates that curriculum design can be used a vehicle to engage staff with<br />
technology for <strong>learning</strong> and teaching, and provide the means to initiate sustainable staff development. At the start<br />
of the project, a research survey revealed that whilst academic staff were reasonably proficient in the use of the<br />
institutional VLE, they were unaware of many of the tools and resources that could be useful within their subject<br />
disciplines. Some staff identified training needs, and indicated a preference for one-to-one support over group<br />
training. The survey also revealed evidence that some staff clear use technology in innovative and effective ways<br />
within their courses. In response to the research data the project team took a three-pronged approach to building<br />
the capacity of academic staff to use technology in their <strong>learning</strong> and teaching in a sustainable way: Remove the<br />
barrier of lack of awareness of useful technologies – this was undertaken through in-faculty drop-in sessions to<br />
demonstrate hardware and software and answer queries; and seminars and blogs to provide more detail of the<br />
ways in which the vast range of available technologies could fit into subject teaching and research activity<br />
Encourage staff already competent in using technology – by organizing regular ‘self-help’ groups to allow sharing<br />
of new ideas and good practice Reach staff with limited engagement with technology – by arranging one-to-one<br />
interviews using appreciative inquiry approaches to explore their subject <strong>learning</strong> and teaching traditions and<br />
identify the ways in which technology enabled tools and resources could be integrated Emphasis was placed on<br />
the curriculum rather than on the technology. Tools and resources –particularly Open Educational Resources<br />
(OER) - were discussed in the context of improved curriculum design and established tangible benefits (JISC<br />
Infonet, 2008). Learner benefits were also a focus, with emphasis on the key role of the tutor in guiding learners<br />
towards the effective use of technology in their <strong>learning</strong> (JISC, 2009). This case reveals that an institutional<br />
approach to curriculum design can be implemented through sustainable approaches at subject level. Thus a top<br />
down mandate for change can be implemented through a bottom-up engagement with practitioners in the<br />
language and approaches of their own discipline. Such an approach moves from central support into the common<br />
practice with departmental and course team debate.<br />
Keywords: technology-enhanced <strong>learning</strong>; staff development; curriculum design<br />
1. Introduction<br />
Over the past decade, The University of Glamorgan has followed a path from Blended Learning to<br />
Technology-Enhanced Learning, via several key projects including the HEA’s GWELLA (University of<br />
Glamorgan, 2011) and Change Academy (University of Glamorgan, 2010) projects which focused<br />
particularly on assessment and feedback methods. These initiatives, steered by the University’s<br />
Centre for Excellence in Learning and Teaching (CELT), have had a significantly positive effect on the<br />
institution, which serves an economically challenged geographical area and a student community that<br />
includes several minority and protected–status groups. These benefits linked to predicted<br />
expectations (Garrison and Kanuka, 2004) to some extent, but it was felt the institution needed to<br />
reflect on the impact resulting from previous interventions (Conole, 2006). It was also believed that<br />
barriers still existed in some cases that were preventing academic staff making full use of the benefits<br />
that technology can bring, and in some cases inhibiting them from making their first tentative steps<br />
across the digital ‘chasm’ (Rogers, 1962). The JISC Building Capacity Project at the University<br />
provided the resource to investigate these barriers more fully and to identify ways of removing them.<br />
The approach was strategic in that it covered the whole organization and was directed by the<br />
institutional Teaching and Learning Strategy (University of Glamorgan, 2006); however, it was also<br />
‘bottom up’ in the sense that there was significant direct contact with academic in the context of their<br />
teaching curricula. The project has constituted a key advance in an on-going programme to make<br />
best use of technology for the benefit of learners.<br />
2. Methods<br />
This paper outlines the methodologies adopted during the project in the form of a case study to<br />
enable the key project milestones to be viewed in context and to provide an ethnographic<br />
development of the project results and its outputs. This phenomenological research began by<br />
considering the barriers to engagement with technology for <strong>learning</strong>, teaching and assessment in<br />
152
Christine Davies<br />
more detail, and subsequently ways of removing these barriers were devised that were tailored to the<br />
constituencies of academic staff that we observed.<br />
2.1 Identifying barriers<br />
Butler and Sellbom (2002) observed that barriers to engagement with educational technology<br />
included inadequate or unreliable hardware; a belief that technology cannot make any difference; and<br />
lack of time to learn new techniques. We felt that these might also apply to the University of<br />
Glamorgan, but we believed it was important to ask more searching questions about the technologies<br />
in use and how they were viewed in the context of University policies and procedures, and individuals’<br />
teaching and/or research roles. A survey was therefore developed to ask about academics’ existing<br />
experience and perceptions of technology for <strong>learning</strong>, teaching and assessment which was sent out<br />
to academic staff in several faculties. We also had the intention of utilizing survey responses (with<br />
permission) as a pool of contact information to allow follow up of individuals for more detailed<br />
investigation.<br />
The survey response rate was disappointing at approximately 10%, but nonetheless included<br />
respondents from a wide range of disciplines and roles showing varying levels of experience of the<br />
use of technology. Responses indicated that technology was generally deemed to be important in<br />
education, but the questions that asked for specific examples of useful technologies were often left<br />
blank, as were suggestions for technologies that were perceived as desirable. Further answers<br />
showed that there was a good understanding of the institutional Virtual Learning Environment (VLE)<br />
and its capabilities, though a limited awareness of other <strong>learning</strong> technologies, especially ‘Web 2.0’<br />
and social media. There was also little experience of the many high-quality online Open Educational<br />
Resources (OER) available in virtually all subjects.<br />
Answers relating to training needs indicated that approximately 50% of respondents identified a need<br />
for training, and also showed a preference for one-to-one approaches. A question specifically asking<br />
for reasons for non-engagement was largely left unanswered, but follow-up interviews discovered that<br />
several of the anticipated barriers were involved, particularly lack of time, and issues such as<br />
imminent retirement, temporary posts, and worries about copyright were also raised. Another barrier<br />
identified was the noticeable lack of awareness of what technology could achieve.<br />
2.2 Raising awareness<br />
Given the results of the survey, we felt it was important to embark on a campaign of raising<br />
awareness, highlighting a wide variety of technologies that could be incorporated into the curriculum.<br />
This was implemented in several ways:<br />
2.2.1 Promotional campaign<br />
A colourful postcard was designed in-house to make lecturers and academics more aware of<br />
technologies that could benefit their students, and this was distributed to all academic staff. The card<br />
also notified them about forthcoming in-Faculty drop-in sessions<br />
2.2.2 In-faculty drop-in sessions<br />
Members of the Technology-Enhanced Learning (TEL) team at CELT visited each Faculty in turn for a<br />
three-hour period over lunchtime to enable academic staff to approach them for ideas, information<br />
and help relevant to their subject teaching. Though the proportion of the staff population who<br />
interacted at these events was quite low, in each Faculty several staff discussed issues affecting their<br />
courses, and subsequently further interviews and projects were arranged in several cases<br />
2.2.3 Accessible OER<br />
The survey indicated that few lecturers were making use of the many high-quality OER that are<br />
current available online. A key initiative within the JISC Building Capacity Project was therefore to<br />
produce templates for use on the institutional VLE to provide important information about key<br />
resources such as iTunesU and Openlearn to make them more accessible for academic staff.<br />
153
Christine Davies<br />
2.2.4 Seminars<br />
CELT regularly hosts seminars that are advertised to the academic community and well-attended by<br />
both external and internal staff. CELT seminars were used to disseminate information about the JISC<br />
Building Capacity Project, and the wide range of technologies relevant to teaching, ranging from<br />
interactive whiteboards to QR codes, and including OER. An interactive session during one seminar<br />
was also an useful opportunity to find out about technologies that the audience deemed useful and<br />
hence to discover some important pockets of innovative practice within the institution.<br />
2.2.5 Blogs and Twitter<br />
Information about the project and its key milestones was also disseminated via a series of blogs on<br />
the CELT website, and appropriately-times tweets.<br />
2.3 Staff already using technology<br />
A ‘Blended Learning Champions’ network was already in existence within the University, and as a<br />
result of the project many other individuals were identified as using technology in innovative and<br />
effective ways. Examples include the use of mobile <strong>learning</strong> resources for languages; screen-capture<br />
for feedback in science; audio-feedback in sports science. A groups of these ‘leaders’ in the use of<br />
technology were invited to meet and share their good practice, with the TEL team chairing and<br />
facilitating. The first meeting proved to be a great success, with new relationships forged and new<br />
ideas shared. Termly meetings of the group continue, and are effectively ‘self-help’ forums for anyone<br />
who already has some experience and interest in educational technology and wishes to explore<br />
further. The TEL team is available to provide support if necessary.<br />
2.4 Reaching staff with limited engagement in technology<br />
Survey information helped to inform a staff development plan for less-engaged staff that closely<br />
aligned to subject teaching and hence would directly inform curriculum re-design. Certain individuals<br />
were particularly targeted on the basis that they might be ‘key-holders’ in their subject areas or<br />
Faculties (JISC Infonet, 2006). The staff development plan featured a highly staff-focused (Mainka,<br />
2007) approach comprising one-to-one interviews lasting approximately one hour. The interviews<br />
were informal in context and interviewees were asked to suggest where and when they wished to<br />
meet. Appreciative inquiry techniques (Cooperinder et al, 2008) were used to explore lecturers’<br />
teaching methodologies and ascertain any personal barriers to engagement with technology, and care<br />
was taken to respect their expertise both in the context of their subject and their learners. There was<br />
also a discursive element to the interviews following the example of Laurillard’s work (2002), with the<br />
interviewer making suggestions about technologies that would meet specific needs. Technologies<br />
were selected on the basis of their previous tangible benefits as outlined by JISC Infonet publications,<br />
for example (JISC Infonet 2009), and included: online mind-mapping as a tool for planning and<br />
collaboration; blogs within Blackboard for reflection on practice; text-to-speech software to assist<br />
literacy and extend accessibility. The interviewer also made reference to the important role the<br />
teacher plays in guiding learners in their use of technology for their <strong>learning</strong> (JISC, 2009).<br />
After each interview, an eMail was sent to the interviewee with more information about the<br />
technologies discussed and a request for feedback after one or more had been tried out. As a result<br />
of these activities, a project has been initiated to use YouTube integrated into the institutional VLE as<br />
evidence for a course in the performing arts, and introductory modules in Business subjects will adopt<br />
mind-mapping as a key tool for revision and planning. Many of the comments and ideas resulting from<br />
these interviews are being further explored as part of an on-going action plan for the TEL team.<br />
3. Conclusion<br />
This project was ambitious in its scope and aimed to build capacity for the use of technology for<br />
<strong>learning</strong> and teaching. It could not be said to have touched all academic staff, but it has certainly<br />
encompassed all Faculties and has involved individuals from almost all subject areas. This<br />
institutional approach to curriculum design implemented at a localized, subject level, will help to<br />
sustain the key initiatives of the project, and will help ensure that a wide range of technologies<br />
continue to be discussed at departmental and course team level.<br />
Throughout the project, emphasis was placed on the curriculum rather than on technology, and tools<br />
were discussed in the context of curriculum design and the needs of specific groups of learners and<br />
154
Christine Davies<br />
teachers. The important role of academic staff in directing the use of technology was emphasized, for<br />
teaching staff with little personal experience of technology often assume that they cannot, or should<br />
not, get involved in educational technology. A dialogic, personalized, approach was also appreciated<br />
by academic staff who are accustomed to being treated as experts and sometimes find it<br />
uncomfortable to be in the position of novices. As a result, technologies that had been previously<br />
unknown are being adopted for the benefit of lecturers and learners. This approach does, however,<br />
require a significant input of time, and additionally, resource needs to be allocated for training needs<br />
that may ensue from the introduction of new technologies.<br />
The project has made a significant impact on raising awareness of the merits of technology and has<br />
helped to steward the University in its effective use of technology (Wenger at al, 2011). It has<br />
enhanced the curriculum, changed academic practice, and helped deliver an important and<br />
sustainable strategic change.<br />
References<br />
Apple (2011) iTunesU. Available at: http://www.apple.com/uk/education/itunes-u (Accessed: 21 April 2011)<br />
Butler D, Sellbom M (2002) Barriers to adopting technology for teaching and <strong>learning</strong>. Educause Quarterly 2, p22<br />
Conole, G. (2006), ‘What impact are technologies having and how are they changing practice?’, I. McNay (ed),<br />
Beyond Mass Higher Education: Building on Experience, Open University Press/ McGraw-Hill Education,<br />
81-95.<br />
Cooperider, DL, Whitney, D, Stavros, JM (2008) Essentials of Appreciative Inquiry. Brunswick OH: Crown<br />
Custom Publishing Inc<br />
Garrison,R, and Kanuka, H (2004) Blended <strong>learning</strong>: Uncovering its transformative potential in higher education.<br />
Internet and HE, 7 (2) p 95-105.<br />
JISC (2009) Managing Curriculum Change. Available at:<br />
http://www.jisc.ac.uk/publications/programmerelated/2009/managingcurriculumchange.aspx jisc 09<br />
(Accessed: 29 January 2011)<br />
JISC (2009) The University of Glamorgan Smoothies: the recipe for effective blended <strong>learning</strong>. Available at:<br />
http://tinyurl.com/6z7ogbx ( Access: 12 May 2011)<br />
JISC (2009) Responding to Learners. Available at:<br />
http://www.jisc.ac.uk/publications/programmerelated/2009/respondingtolearners.aspx (Accessed: 2<br />
February 2011)<br />
JISC Infonet (2006) Change Management Infokit. Available at: http://www.jiscinfonet.ac.uk/infokits/changemanagement/index_html<br />
(Accessed: 19 January 2011)<br />
JISC Infonet (2008) Exploring tangible benefits of e-<strong>learning</strong>: does investment yield interest? Available at:<br />
http://www.jiscinfonet.ac.uk/publications/info/tangible-benefits-publication (Accessed: 18 January 2011)<br />
Laurillard, D (2002) Rethinking University Teaching: A Conversational Framework for the Effective Use of<br />
Learning Technologies. 2nd edition. London: Routledge Falmer<br />
Mainka C (2007) Putting staff first in staff development for the effective use of technology in teaching. Br. J. Ed.<br />
Tech. 38 (1), p138<br />
Open University (2011): Openlearn LeaningSpace. Available at: http://openlearn.open.ac.uk (Accessed: 20<br />
February 2011)<br />
Rogers, E M (1962). Diffusion of Innovations. Glencoe: Free Press.<br />
University of Glamorgan (2006). Learning, Teaching and Assessment Strategy, 2007-2012. Available at:<br />
http://celt.glam.ac.uk/sites/default/files/LTA%20Strategy%202007-2012.pdf (Accessed: 12 March 2011)<br />
University of Glamorgan (2011) GWELLA project: Enhancing Learning & Teaching through Technology for<br />
Higher Education in Wales. Available at: http://<strong>learning</strong>.weblog.glam.ac.uk/2011/2/2/gwella-reportenhancing-<strong>learning</strong>-teaching-through-technology-for-higher-education-in-wales<br />
(Accessed: 22 January<br />
2011)<br />
Wenger E, White N and Smith JD (2011) Digital Habitats: Stewarding technology for communities. Available at :<br />
http://technologyforcommunities.com (Accessed 20 May 2011)<br />
155
Web Conferencing for us, by us and About us – the Leeds<br />
Met Elluminate User Group<br />
Mark de Groot, Gill Harrison and Rob Shaw<br />
Leeds Metropolitan University, Leeds, UK<br />
m.degroot@leedsmet.ac.uk<br />
g.harrison@leedsmet.ac.uk<br />
r.shaw@leedsmet.ac.uk<br />
Abstract: How do you implement a strategy for institutional adoption of web conferencing? What are the<br />
potential benefits for students, tutors, support and administrative staff? What strategies can maximise those<br />
benefits? How do you provide the relevant staff development? At Leeds Metropolitan University answers have<br />
been sought by bringing together current users - meeting in web conferences naturally - to shape, to share and to<br />
develop their ideas. This case study first describes the environment in which it makes sense for higher education<br />
institutions to fast track the implementation of appropriate aspects of web conferencing. It next describes the<br />
establishment and remit of the University’s web conferencing group. It analyses web conferencing benefits and<br />
implementation recommendations as presented in the group and in core related activities. Finally it identifies both<br />
the quick wins on offer and the challenges to be met from the institutional, student, and staff points of view. This<br />
paper explores what is possible when the principles that underlie technology supported student centred <strong>learning</strong><br />
are applied to technology supported user centred staff development. It specifically addresses the suggested<br />
conference theme: This<br />
promising and productive experiment in both user driven staff development and, more generally, the<br />
dissemination of ideas and sharing of good practice will be of interest to all responsible for shaping and<br />
supporting their institutional web conferencing policies; providing insights into: The benefits and quick wins for an<br />
institution prepared to adopt web conferencing The challenges and ways in which they can be addressed<br />
Effective staff development strategies in a changing and economically restrained environment<br />
Keywords: web conferencing, staff development, Elluminate, online <strong>learning</strong>, change management<br />
1. Context and background<br />
1.1 Changes in higher education, nationally and internationally<br />
Increased flexibility of provision and efficiency of business processes will be essential in the response<br />
of Higher Education Institutions (HEIs) both to the Browne Review (Browne 2010) and the<br />
Comprehensive Spending Review (HM Treasury 2010). The most recent government white paper<br />
(BIS 2011) makes clear the government’s intention to promote student choice as a key way to drive<br />
forward innovative forms of delivery.<br />
This approach is entirely consistent with an earlier report, Higher Education in a Web 2.0 World, from<br />
the UK’s Joint Information Systems Committee (JISC 2009). In challenging the inertia of the<br />
established system the report provides an important analysis of the need for and the means by which<br />
HEIs can adapt to meet the needs and expectations of students who are increasingly comfortable with<br />
technology. The report states that the process of engagement, especially with those technologies that<br />
support collaborative networking, develops a skill set that matches both to views of 21 st century<br />
<strong>learning</strong> skills and to those of 21 st century employability skills –collaboration, creativity, leadership and<br />
proficiency with technology.<br />
Pressures of student numbers, increased competition and constrained resources are not unique to<br />
the UK. Rapid advances in educational technologies, particularly internet based technologies, are<br />
blurring the boundaries between institutions and between countries. Predicting the future virtualization<br />
of international higher education Sohail Inayatullah suggests that as costs go down, over the longer<br />
term current distinctions between virtual and real will disappear, with an important shift taking place<br />
from merely more technology in the class room to actual, digital pedagogy (Inayatullah 2011).<br />
1.2 Web conferencing in higher education<br />
Web conferencing is increasingly available across the Higher Education sector and beyond (Murphy<br />
and Ciszewska-Carr 2007; de Freitas and Neumann 2009).<br />
156
Mark de Groot et al.<br />
Evidence from the sector however indicates that the variables impacting successful synchronous<br />
<strong>learning</strong> (Park and Bonk, 2007, de Freitas and Neumann, 2009) remain relatively unexplored. In their<br />
discussion of pedagogic models appropriate to synchronous conferencing, especially communities of<br />
enquiry, task design and multimodal theories, De Freitas and Neumann (2009) can be said to<br />
establish a framework, in the dedicated context of web conferencing, for the future ‘digital pedagogy’<br />
envisaged by Inayatullah (2011).<br />
However, despite increased availability of practical and technical guidance in the use of web<br />
conferencing (Chatterton, 2011), the already significant investment made in web conferencing<br />
remains largely in the hands of specialists.<br />
1.3 Approaches to staff development<br />
What this suggests is that in moving beyond awareness raising in exploiting our use of virtual<br />
conferencing spaces we need also to find more efficient ways to develop and to capitalise on the real<br />
knowledge, experience and insights of staff and students who involve themselves with web<br />
conferencing. The core proposal here is that mirroring collaborative climate of Web 2.0 technologies,<br />
web conferencing itself can be the vehicle to provide more appropriate, targeted staff development.<br />
The need for such adjustment is indicated by MacDonald (2006). She argues that effective staff<br />
development must adapt to accommodate the preferences and working habits of both full and part<br />
time tutors. Further, Garrison & Vaughn (2008) are clear about the limitations of “one off” staff<br />
development sessions, however well they may be planned, executed and evaluated. The collegiate<br />
network they identify as fundamental for effective professional development in education needs to be<br />
mirrored by an evolving network of resources and opportunities that is in sympathy with changing<br />
institutional needs, participant constraints and aspirations and pedagogic principles.<br />
Smyth et al (2007) report on the established social constructivist principles that underlie the pedagogy<br />
of their TESEP project. Their learner centred staff development evidences their core belief that the<br />
best way to encourage transformation in teaching practice is to immerse practitioners in the kinds of<br />
<strong>learning</strong> experiences they would ultimately support for their own students.<br />
This paper reviews the running of a collegiate staff development web conferencing network. It<br />
describes the immersive web conferencing activities of this learner centred staff development forum. It<br />
provides a thematic analysis of web conferencing benefits and recommendations as discussed by the<br />
group. It then tracks related spin off events from the group’s activity. It concludes by summarising the<br />
challenges involved.<br />
2. Web conferencing at Leeds Met<br />
Even as Leeds Met institutional strategies (Leeds Met, 2010) have evolved to reflect both the<br />
demands and constraints of a changing national and international environment certain core priorities<br />
have remained constant. These include:<br />
The continuous review and improvement of teaching and assessment practices across Leeds Met<br />
The development of new, cost-effective and flexible ways of working that make optimal use of our<br />
estate and technology<br />
The empowerment of staff, students and visitors to reduce their environmental impact by<br />
encouraging, and celebrating environmental improvements<br />
In this context funding was sought in early 2008 to assess the possible value of making a common<br />
web conferencing system available to all staff and students. A comprehensive evaluation was<br />
conducted, involving some 45 representative stakeholders drawn from students and from staff<br />
responsible for teaching, administration and technical support. This thorough evaluation process<br />
resulted in a three year site licence for an externally hosted web conferencing system then known as<br />
Elluminate.<br />
All involved in this process agreed that the chosen system could best and most reliably handle all<br />
predicted situations for academic and administrative use. Users can set up as many sessions as they<br />
need, whenever they need them and for as long as they need them. Web conferences can be<br />
accessed through the University’s virtual <strong>learning</strong> environment or, independently by clicking on a<br />
URL. Equally, if required, there is controlled access to recordings of all sessions. An Elluminate white<br />
157
Mark de Groot,<br />
paper (Elluminate 2010) describes the early, positive engagement of both staff and students at Leeds<br />
Met in this collaborative process.<br />
Since the early adoption of this system, the experience of staff and students at Leeds Met has<br />
provided increasingly compelling evidence that web conferencing offers benefits in the spheres of<br />
flexible online and distance <strong>learning</strong>. Importantly there are also clear economic and environmental<br />
savings through reductions in travel time and costs for staff attending what are typically shorter and<br />
more focussed meetings.<br />
Figure 1 below illustrates the year on year growth in web conferencing at Leeds Met over this period.<br />
Figure 1: Number of Elluminate sessions 2008 - 2010<br />
As of Jan 2011 there are 163 members of staff who have between them set up / run more than 1500<br />
web conferencing sessions. We currently average about 60 sessions a month.<br />
The graph tracks the use of web conferencing from the processes of awareness raising through to<br />
take up by early adopters. This is healthy progress. Having established the context the remainder of<br />
this paper will describe the dynamics of staff development activity that can help bring web<br />
conferencing into the main stream of University activity. It explores what is possible when the<br />
principles that underlie technology supported student centred <strong>learning</strong> are applied to technology<br />
supported user centred staff development.<br />
3. Web conferencing user group - overview, aims and operation<br />
When new software has been introduced to University it has proven consistently challenging to<br />
provide sufficient familiarization in a timely and appropriate manner for both individuals and course<br />
teams. This is an issue recognized across the sector and is associated with the difficulties in<br />
establishing flexible mechanisms to provide the follow up local network of support and resources<br />
necessary to put new ideas into practice (Garrison and Vaughn 2008).<br />
Setting up a web conferencing user group would, it was hoped, address some of these difficulties by<br />
making it possible to provide a series of staff development sessions that made minimal demands on<br />
people’s time (all sessions lasted 60 minutes or less), made it possible to catch up should a given<br />
session of interest be missed, and required no travel. There was no compulsion, or expectation to<br />
attend all or any particular session; just the understanding that all information about the work of the<br />
group would be available to all. More importantly the intention was that session content while<br />
coordinated by the three nominal leaders would be discussed, agreed and largely delivered by<br />
members of the network. Importantly this would guarantee immediacy and relevance and flexibility of<br />
content. Sessions would only deal with what was of current interest.<br />
By the end of March 2010 some 70 staff signed up to Leeds Met’s ‘Elluminati’ web conferencing user<br />
group. A request to state which activities they expected to be interested in produced these results:<br />
158
Table 1: Activities<br />
Mark de Groot et al.<br />
Activity Number interested<br />
Monthly email 44<br />
Monthly meeting 35<br />
Promotional events 24<br />
Practical support 23<br />
Leeds Met group 16<br />
Centre of Excellence 15<br />
Licence rationale 14<br />
Impact analysis 12<br />
Other 7<br />
The group agreed to meet online once a month and at the first meeting agreed to convert these<br />
expressed interests into an overall remit in three areas:<br />
Good practice – provide a forum to share, experiment with and promote good practice<br />
Resources – develop accessible, straightforward guidance especially for staff and students new<br />
to web conferencing<br />
Licensing – based on an analysis of current and potential benefits across the university, promote<br />
the group’s consensus view on an appropriate way forward with future web conferencing<br />
licensing.<br />
An initial 12 month time limit was proposed and participation in any or all of these three activities was<br />
entirely voluntary. Overall the intention was to find an effective and practical way to support the 160<br />
registered moderators and promote web conferencing generally.<br />
Meetings proved to be lively. Their general format included warm up activities, reports on progress,<br />
discussions of strategy, experimental activities, and guest speakers demonstrating their own good<br />
practice. Additionally a series of 12 master classes was organised. This was delivered, at times<br />
suitable for UK colleagues, from across the Atlantic by Elluminate experts. The short, 30 minute<br />
sessions covered a range of techniques for engaging, effective presentation online. All sessions were<br />
recorded as were all user group meetings. They are now a lasting resource to help develop the digital<br />
pedagogy (Inayatullah 2011) skills of new converts to web conferencing.<br />
A monthly email newsletter summarized what was happening in the group. A public folder was setup<br />
within the University’s Outlook email system to enable open access to records of all the group’s<br />
activities.<br />
4. Web conferencing user group – activities<br />
These are here considered under the same headings as given for the remit of the group.<br />
4.1 Good practice<br />
The analysis of good practice discussed in the group meetings presented in the table 2 makes clear<br />
the three contexts under consideration: Student meetings, Staff Meetings and Virtual <strong>Conferences</strong>. It<br />
lists the subjects of web conferences held in each of these categories and then collates the benefits<br />
and recommendations for each category. The consensus in all three categories is that web<br />
conferencing does not replicate the face to face environment. It is necessary for all involved to be<br />
open to different behaviours…and to have the time and space to develop those behaviours.<br />
4.2 Resources<br />
Asked to suggest what would be helpful to support their use of Elluminate 52% of respondents put<br />
readily available resources as their first priority. Figure 2 illustrates the user group’s response to this<br />
perceived need. The links on the right hand column predict the most frequently needed tasks and lead<br />
to step by step instructions to complete the task selected. The examples of good practice are a<br />
growing collection of 3 min (max) recordings describing best practice as first presented to the user<br />
group. The alphabetical list of videos and paper guides allows the creation of reference resources as<br />
159
Mark de Groot,<br />
requested. The target here is to present users first with straightforward guidance on the tasks they<br />
need to do first.<br />
Table 2: Good practice<br />
Good Practice Context Benefits and Recommendations<br />
Student Meetings<br />
Benefits:<br />
Real time communication adds dynamic element to<br />
Running tutorials for MSc Distance Learners<br />
otherwise purely asynchronous experience<br />
Increases tutor availability to students…especially<br />
Small group activities in breakout rooms with placement / dissertation and distance <strong>learning</strong><br />
students<br />
Student office hours Web Tour demonstrations Possibility of recorded presentations of speakers who<br />
can’t be present<br />
Working with guest speakers<br />
Recorded sessions valued for revision<br />
Staff Meetings<br />
Setting up subject group meetings<br />
Running course review meetings<br />
Running project support meetings<br />
160<br />
Recommendations:<br />
Expect and prepare for students to have initial access<br />
problems<br />
Especially effective for one to one and small group<br />
meetings<br />
For larger groups if feasible prepare students in a pc<br />
lab beforehand (30 mins orientation)<br />
For distance learners set meeting time at end of<br />
working day<br />
Consider use of video…to confirm identity of student<br />
tutor is talking to<br />
Develop tutor skills to encourage interactivity eg:<br />
Using Chat room for Students to paste URL of what<br />
they have found<br />
Web tour to access online images for discussion.<br />
Practice with exercise in breakout rooms<br />
Challenge staff to be prepared to be available in<br />
these ways…change working practices<br />
Doesn’t replicate face to face…be open to different<br />
behaviours.<br />
Benefits:<br />
Better time management<br />
Better attendance (esp. pt tutors<br />
More focussed / reflective discussion<br />
Better participation<br />
More inclusive (e.g. course administrators).<br />
Save money. Save travel time. Saved meeting time<br />
Clearer agreed action points.<br />
Better note taking …recording available<br />
Green & efficient<br />
Recommendations:<br />
Planning and preparation needed<br />
Expect early technical problems<br />
Encourage peer support.<br />
Consider alternating face to face meetings with web<br />
conferences<br />
Encourage interactivity<br />
Encourage use of Video / Chat<br />
Doesn’t replicate face to face…be open to different<br />
behaviours
Virtual <strong>Conferences</strong><br />
Organising an online conference<br />
Supporting guest speakers<br />
Figure 2: Elluminate resources<br />
4.3 Licensing<br />
Mark de Groot et al.<br />
Benefits<br />
Access to recorded sessions following conference<br />
Opportunity for collegiate, reflective discussion pre<br />
and post conference<br />
Savings in conference running costs<br />
Savings in delegate attendance costs<br />
Greener<br />
Opportunity to provide a different, deeper experience<br />
Recommendations:<br />
Support website needed<br />
Manage registrations (consider e.g.<br />
http://eventbrite.com )<br />
Keep sessions short (20 mins)<br />
Prepare video for backup<br />
Speaker practice area needed<br />
Speaker support/guidance needed<br />
Clear timelines needed<br />
Conference practice session needed<br />
Doesn’t replicate face to face…be open to different<br />
behaviours<br />
The initial site licence for Elluminate expired at the end of May 2011. This licence has now been<br />
extended until December 2012 based on the group’s report on the on benefits gained and conditions<br />
under which future potential could be harnessed.<br />
From the beginning of 2012 a sub group will re-evaluate this commitment in the light of the growing<br />
availability of competitive alternatives. What is established however as central to this future decision is<br />
the support and commitment of the core stakeholders as represented by the user group.<br />
5. Web conferencing user group - impact<br />
In this section evidence of the impact of the work done by the user group is provided first by samples<br />
of feedback from students whose tutors are either members of the group or have been supported by<br />
161
Mark de Groot,<br />
members of the group. Further evidence of impact is then provided by ‘spin off’ events supported by<br />
the activities of group members.<br />
5.1 Student feedback<br />
Distance learners have consistently made clear that web conferencing creates for them an active<br />
sense of belonging to their study group, adding a dynamic and personalised element to what would<br />
otherwise necessarily be a purely asynchronous, online experience. Further, they value access to<br />
recordings the flexibility, convenience and discipline inherent in this resource. Evidence here is taken<br />
from two MSc courses offered to distance learners (Construction Law Dispute and Resolution and<br />
Parkinson’s Disease respectively)<br />
It really helps us communicate and get comfortable seeking support from other members of the<br />
group<br />
I’ve found it very difficult to motivate myself to study other modules which haven’t used this<br />
method of teaching<br />
The recordings are brilliant and even when I have attended the live sessions - I have used them<br />
for revision and to go over any parts I missed<br />
This for me offers the best balance of being able to interact whilst still maintaining a lot of the<br />
flexibility of a distance <strong>learning</strong> course, which given my work commitments is crucial.<br />
I look forward to the sessions - they act as discipline to keep pace with the delivery of the course<br />
and without them I might also feel that I was studying alone.<br />
Exceedingly impressed with the system. .. It avoids issues of having to travel to attend a fixed<br />
location (travel time, commuting and parking etc) which considering that everyone has already<br />
had a long day at work… is a very significant benefit for myself and no doubt others also.<br />
Elluminate gave me the sense that I was not on my own<br />
The live meetings gave me support throughout the module<br />
It’s invaluable because…you feel less isolated<br />
It has to be said that technical problems accessing web conferencing are a significant risk most<br />
especially for students working off campus where neither internet connection nor pc setup can be<br />
predicted or standardised. This feedback finds many echoes but also evidences student willingness to<br />
accept initial problems in favour of a perceived value in the resource.<br />
In principle, the Elluminate sessions are a good idea. Unfortunately I was unable to attend many<br />
of the sessions because of a variety of reasons: a) I could not use the software on my Apple Mac<br />
b) I could not use the software on my Apple Ipad c) I could not use the software on my work<br />
laptop or at work as I do not have the software privileges to install it.) The final option was that I<br />
used my own windows laptop and this is what I used successfully. Unfortunately, when I was<br />
staying away with work I then had problems connecting to the internet in a hotel / anywhere else.<br />
Despite all the above, in better personal circumstances, I would be happy to continue using it in<br />
future.<br />
Student enthusiasm is further (and cheerfully) evidenced from answers written on an Elluminate<br />
conference whiteboard by students on a campus based course gathered in a pc lab who had worked<br />
with an expert speaker working from her home in Somerset.<br />
5.2 Related events and activities<br />
Events and activities that have informed or spun off from the activities of the user group are<br />
characterised by innovative forms of staff development. In all cases their fundamental provision is a<br />
collegiate environment that fosters collective consideration of the pedagogic implications of web<br />
conferencing.<br />
5.2.1 Web conferencing with Elluminate – Faculty of Health and Social Sciences<br />
One of the founder members of the web conferencing group, the <strong>learning</strong> technologist from the<br />
Faculty of Health and Social Sciences, ran very successful 4 week on line courses in the use of web<br />
conferencing. The courses entailed a weekly one hour synchronous session with planning and<br />
reflective tasks carried out by asynchronous discussion in between. A number of participants went on<br />
162
Mark de Groot et al.<br />
to use web conferencing within their courses and reported positive outcomes. Many participants<br />
stressed the need for an ongoing network of Elluminate users and their experiences have in turn been<br />
fed in to and shared across the University by their involvement in the wider user group.<br />
Figure 3: Student feedback<br />
5.2.2 Let’s talk – Faculty of Arts, Environment and Technology<br />
A member of the group from the Faculty of Arts, environment and Technology secured external<br />
funding to research different ways in which web conferencing might support the work of his<br />
colleagues in the School of the Built Environment. The resource was adopted on 6 courses within the<br />
school with results reported in July 2011 at an online conference, Let’s Talk. Speakers again reported<br />
positive outcomes for both staff and students in using web conferencing to:<br />
Communicate and provide individual support for placement students and students writing<br />
dissertations.<br />
Conduct meetings and exam boards with overseas partners<br />
Deliver inductions, online tutorials and seminars to distance learners<br />
5.2.3 Show and Tell events<br />
Two face to face ‘Show and Tell’ events have been run by members of the group. These have been<br />
as part of our annual Assessment Learning and Teaching conference and at the user group’s<br />
anniversary party. In addition to those outlined in section 5 above, themes covered at these events<br />
included:<br />
Interviewing prospective international postgraduate students<br />
Feedback and assessment of student presentations<br />
The academic coaching of students<br />
Running taster sessions for first timers.<br />
6. Recommendations from the group<br />
Insights from the experiences of those involved in the activities of this Leeds Met user group will be of<br />
interest to all responsible for shaping or supporting their institutional web conferencing policies. The<br />
high payback areas, or quick wins, are here identified along with challenges and suggested strategies<br />
with which these challenges can be addressed.<br />
163
6.1 The quick wins<br />
Mark de Groot,<br />
Students have consistently been alive to the added value afforded by web conferencing. It can put<br />
them in touch with experts in their field, give them more access to their tutors or simply provide a<br />
convenient, collaborative <strong>learning</strong> and social environment. Distance learners have been particularly<br />
appreciative of the group contact and stronger cohort identity afforded through live conferencing. All<br />
have appreciated the access to revision material provided by recorded sessions.<br />
For tutors the most tangible benefits come with minimum overheads where web conferencing is used<br />
in one to one situations. These include supervision of dissertations and research projects, support for<br />
placement students and tutor contact through scheduled office hours.<br />
Overheads increase as group sizes become larger; nevertheless small groups (5 to 15) of distance<br />
learners have been particularly appreciative of the contact afforded through live conferencing and the<br />
access to revision material provided by recorded sessions.<br />
For administrators and support staff web conferencing has proved to be especially rewarding both<br />
where part time staff are involved in course delivery and where full time staff have needed to travel<br />
between campuses. Benefits have included better attendance, better use of time, more focussed<br />
discussion and agreement on subsequent actions if key points are written and displayed to all during<br />
the meeting.<br />
External projects have also benefited, especially where travel and meeting time has been an explicitly<br />
costed expense.<br />
For the institution the experience of staff and students at Leeds Met provides increasingly cogent<br />
confirmation that web conferencing can contribute to the rethink in the administration, support and<br />
delivery of student <strong>learning</strong> that is necessitated by increases in student fees from September 2012.<br />
There is clear potential to deliver better value to online and distance learners, better support for<br />
individual students, and reduced travel and environmental costs for those travelling to meet in<br />
University buildings.<br />
6.2 The challenges<br />
Technical challenges need to be expected, recognized and addressed. They may vary from simple<br />
(headsets muted) to more complex (Firewall / Java issues) They are a potential barrier for all new<br />
users but especially for those working off campus. Our licence includes access for all to a 24/7<br />
technical helpline. This has helped distance learners with access problems. It is an especially<br />
valuable resource for students working in different time zones. Student tolerance of technical<br />
difficulties has consistently been greater than that of staff. Possibly the stakes are higher for staff<br />
where issues they cannot always control directly can and have meant cancellation of scheduled<br />
sessions. Guidance generally has been to ensure sufficient early preparation, to encourage practice<br />
sessions and to test access in good time before a session starts.<br />
Challenges relating to pedagogy have tended to arise where the assumption has been made that face<br />
to face approaches will transfer directly to online web conferences. The identified need is to provide<br />
for staff sufficient good experiences and examples of engaging, interactive web conferences.<br />
Web conferencing presents a direct challenge to existing patterns of staff development. Rather than<br />
schedule content driven sessions prepared by support staff consider staff development characterised<br />
as: time limited, task focussed, resource light, technology enabled and user driven. Our experience is<br />
that it makes absolute sense to introduce web conferencing itself as the core means of supporting<br />
such staff development.Software upgrades and company buyouts are ever present and not always<br />
welcome. What is certain however is that change is constant. Staff investing time to become familiar<br />
with particular web conferencing software need to understand and be confident of the benefits, and<br />
inevitability, of major upgrades. The consultation and consensus building that is possible within a user<br />
group goes far in addressing this challenge, empowering users as contributors and advisers on key<br />
decisions such as selection of resources and licensing. The challenge to existing work habits is<br />
perhaps the most far reaching challenge of all. Web conferencing provides a natural, inviting<br />
environment for students schooled and familiar with sophisticated communication technologies. This<br />
needs to be reflected in institutional strategies, especially teaching and estates strategies. Leaders<br />
164
Mark de Groot et al.<br />
need to encourage their staff to challenge established methods and procedures if the needs and<br />
expectations of tomorrow’s students are not to be frustrated (JISC 2009).<br />
7. Conclusion<br />
The web conferencing user group has:<br />
Identified the high payback areas<br />
Taken on board the need for careful induction for new users of web conferencing<br />
Established forward looking models of online staff development and support<br />
Agreed recommendations to respond practically to the future development of an enhanced web<br />
conferencing system.<br />
This group has in summary given practitioners at Leeds Met a collaborative space in which their<br />
individual development needs could be explored and addressed, proving to be a promising and<br />
productive experiment in user driven staff development. It has also, most importantly, proved to be an<br />
influential forum that has helped and will continue to help shape web conferencing practice across the<br />
institution. The dissemination of ideas and the sharing of good practice across the group have proven<br />
effective and valued components of a movement towards institutional change.<br />
References<br />
Browne, J (2010) “Securing a sustainable future for higher education in England”, [online],<br />
http://hereview.independent.gov.uk/hereview/report/ (Accessed 25/3/11)<br />
BIS (2011) “Higher Education: Students at the Heart of the System”, [online],<br />
http://www.bis.gov.uk/news/topstories/2011/Jun/he-white-paper-students-at-the-heart-of-the-system<br />
(Accessed 27/7/11)<br />
Chatterton, P (2011). “Designing for Participant Engagement with Elluminate Live”, [online]<br />
http://www.jisc.ac.uk/elluminateguidance (Accessed 11/11/11)<br />
de Freitas, S. and Neumann, T. (2009) “Pedagogic strategies supporting the use of Synchronous Audiographic<br />
Conferencing: A review of the literature”. British Journal of Educational Technology, Vol 40, No 6, pp 980-<br />
998.<br />
Elluminate (2010). “Leeds Metropolitan University Embraces Technology Enhanced Learning”, Implementing<br />
Elluminate Technology in the United Kingdom,. [online],<br />
http://www.elluminate.com/Resources/White_Papers/White_Paper_request/25/?id=312 [Accessed 15 /6/11)<br />
Garrison D. R. & Vaughn N.D. (2008) Blended <strong>learning</strong> in Higher Education. Framework, Principles, and<br />
Guidelines. San Francisco, Jossey-Bass.<br />
HM Treasury (2010) Spending Review. [online], http://www.hm-treasury.gov.uk/spend_index.htm (Accessed<br />
21/4/11)<br />
Inayatullah S (2011) “Open Learning Comes of Age”, Observatory on Borderless Higher Education: Perspectives<br />
on the Future, [online],<br />
http://www.obhe.ac.uk/documents/2011/Reports/Borderless_2011_Perspectives_on_the_Future (Accessed<br />
27/7/11)<br />
JISC (2009) “Higher Education in a Web 2.0 World”, [online],<br />
http://www.jisc.ac.uk/publications/generalpublications/2009/heweb2.aspx#downloads (Accessed 20/1/11)<br />
Leeds Metropolitan University (2010) “Strategic Plan 2010 -1015”, [online]<br />
http://www.leedsmet.ac.uk/strategicplan/Leeds-Metropolitan_Strategic-Plan_2010-2015.pdf (Accessed 7<br />
Sep 2011)<br />
MacDonald J. (2006). Blended Learning and Online Tutoring – A Good Practice Guide. Aldershot, Gower.<br />
Murphy, E. and Ciszewska-Carr, J. 2007. Instructors' experiences of web based synchronous communication<br />
using two way audio and direct messaging. Australasian Journal of Educational Technology. 23(1), pp. 68-<br />
86.<br />
Neumann, T. (2010). “Practice and Pedagogies of Synchronous Online Learning”, ExcelDL – Synchronous<br />
Online Learning: Bridging the Divide. Recording [online],<br />
http://www.exceldl.com/ (Accessed 20 /6/11)<br />
Neumann, T. and Carrington, A. (2007). A mass collaboration approach to e-<strong>learning</strong>: multiple venue production.<br />
Association for Learning Technology Newsletter, April 2007. [online]<br />
http://newsletter.alt.ac.uk/e_article000783770.cfm (Accessed 26 /5/11)<br />
Park, Y. & Bonk, C. (2007) Synchronous Learning Experiences: Distance and Residential Learners’ Perspectives<br />
in a Blended Graduate Course. Journal of Interactive Online Learning Vol 6, No 3. [online] <<br />
http://www.ncolr.org/jiol/issues/pdf/6.3.6.pdf > (Accessed 5/1/11)<br />
Smyth, K., Comrie, A., Foulis, L., Greatorex, D. & McCran, J. (2007). From bruised to enthused: tackling the<br />
challenges of championing online <strong>learning</strong> for personal and institutional change. Paper presented at JISC<br />
Innovating e-Learning 2007: Institutional Transformation and Supporting Lifelong Learning. [online] <<br />
http://tiny.cc/4y5aa > (Accessed 10/12/10)<br />
165
Tools for Evaluating Students’ Work in an Interactive<br />
(Open) Virtual Space: Case Study of an eLearning Course<br />
in an International Network of Universities<br />
Jana Dlouhá, Martin Zahradník, Jiří Dlouhý and Andrew Barton<br />
Charles University Environment Center, Charles University, Prague, Czech<br />
Republic<br />
jana.dlouha@czp.cuni.cz<br />
Abstract: This article presents a brief analysis of changes in educational practices associated with the “third role”<br />
of higher education institutions (HEIs) that are occurring not only at an institutional level, but also within the<br />
<strong>learning</strong> process taking place at lower levels (individual, course, program) and could be effectively combined with<br />
the introduction of eLearning methodologies into teaching that stress the social aspects of <strong>learning</strong>. Teaching in<br />
the open space provides the opportunity to use active forms of teaching / <strong>learning</strong> and creates conditions for<br />
social <strong>learning</strong>. Conceptual and practical shifts would also involve methods of assessment to justify their benefits<br />
and stress certain qualities in higher education (HE). Based on these theoretical considerations, practical<br />
experience with the eLearning course “Multiple Perspectives on Globalization and Sustainable Development”<br />
operated as part of the international Virtual Campus for Sustainable Development (VCSE) eLearning program is<br />
analyzed. The method of teaching was geared toward independent and collaborative student work in a wiki<br />
environment and the development of key competences necessary to understand and be active in the complex<br />
field of sustainability. The authors show how to practically apply the pedagogical principle that educational<br />
objectives, methods, <strong>learning</strong> environments and assessment procedures must be aligned. For assessment, a<br />
combination of evaluation tools was introduced, such as rubrics evaluated by teachers and questionnaires<br />
completed by the course participants which provided feedback on course outcomes in comparison with its<br />
educational goals. The method of assessment focused on “students’ approaches to <strong>learning</strong>” is described, and<br />
the possibilities for promoting and evaluating social <strong>learning</strong> processes that would contribute to the development<br />
of capabilities to communicate across disciplinary and academic boundaries within higher education are<br />
discussed.<br />
Keywords: higher education, eLearning, competences, social <strong>learning</strong>, Wiki, assessment<br />
1. Introduction – change of practices in higher education<br />
With regard to their potential engagement in society and active participation in its transition toward<br />
sustainability, universities have to develop a third role based on interaction with other societal players,<br />
and mutual interdependence with society (“co-evolution” with its systems) (Ferrer-Balas et al., 2009).<br />
In this article we will argue that the envisaged change might be applied at an individual level and thus<br />
brought into the heart of education - through transformation of the knowledge generation process that<br />
will emphasize its social functions, the development of participation and to involve a “system’s<br />
thinking, joint <strong>learning</strong>, open communication…” (Lukman et al., 2009). In practice, an expanded set of<br />
educational outcomes is required: besides knowledge (that should reside in an individual’s head),<br />
competences that are developed and are subsequently demonstrated in meaningful practice are<br />
important. An operational definition of competences is based on a combination of skills, knowledge<br />
and attitudes that are appropriate to particular situations (Dlouhá, 2009a, 2009b); competences<br />
relevant to sustainable development (the theme of the course described further on) involve:<br />
“competency for using, shaping, handling and sharing different sets of information and knowledge”,<br />
“competency for dealing with uncertainties and thinking proactively”, and “process-oriented and<br />
structural knowledge” (Burandt and Barth, 2010).<br />
This perspective is associated with constructivist theories, focusing on the <strong>learning</strong> process as a<br />
distinct field, situational understanding of it (the process is affected by a particular situation), and<br />
research that is no longer focused on student characteristics such as intelligence, but rather on their<br />
individual preferences, perceptions of the <strong>learning</strong> environment and motivations. This focus brought<br />
about not only considerable methodological gains but also possibilities to influence the <strong>learning</strong><br />
process and resulting student achievements through teacher interventions (Entwistle, 2000).<br />
1.1 Learning approaches<br />
Learning processes started to be researched in Britain and Sweden in the 1970s and the studies were<br />
focused on the <strong>learning</strong> environment factors - these factors are perceived differently by students and<br />
result in a combination of student motives and strategies which was defined as student approaches to<br />
166
Jana Dlouhá et al.<br />
<strong>learning</strong>: “surface”, “deep” and “achieving <strong>learning</strong> approach” (Biggs, 1987). Research proved that<br />
student approaches to <strong>learning</strong> considerably affect <strong>learning</strong> processes in terms of their efficiency and<br />
have an impact on the final student’s performance. The <strong>learning</strong> environment factors are explored in<br />
studies that concentrate on the educational requirements, the quality of the teaching and the nature of<br />
the assessment, but also student interests and support, the enthusiasm shown by the instructor and<br />
the opportunity for students to manage their own <strong>learning</strong> process (Richardson, 2009).<br />
New methodologies for assessment of the <strong>learning</strong> environment and its psychical (motivations) and<br />
social determinants were subsequently developed and numerous psychometric measurement<br />
techniques based on questionnaires assessing student <strong>learning</strong> experiences have already been<br />
tested so that they can be applied in practice as evaluation instruments for educational management<br />
purposes. The interest of researchers is mostly determined by the attempt to improve student<br />
performance by influencing in particular the <strong>learning</strong> environment as an external prerequisite for a<br />
good <strong>learning</strong> process and consequent success.<br />
1.2 Assessment process<br />
Assessment is a rather complex task both at an institutional or system level, and at the level of<br />
<strong>learning</strong> processes. Different factors that play a role in these processes should be aligned: not only<br />
educational goals and content, but also tasks and assessment. Evaluators of higher education<br />
teaching agree that the most commonly misaligned factor is assessment and “most assessment<br />
strategies tend to focus on what is easy to measure rather than what is important.” For success in<br />
<strong>learning</strong>, all different aspects of the <strong>learning</strong> process are relevant – for example, social interactions<br />
play a role in which students are engaged when they carry out their tasks, such as collaboration and<br />
teamwork, relationships with their tutors, and also <strong>learning</strong> environment factors. Best teaching<br />
assessment practices in higher education should capture these aspects and therefore focus on<br />
“critical thinking, problem solving, creativity, curiosity, concern for ethical issues” as well as “breadth<br />
and depth of specific knowledge”; they also reflect the “methodologies and standards of evidence<br />
used to create that knowledge”. Attention paid to these factors is a requirement of active,<br />
constructivist pedagogy, but in practice, in most cases assessment is based upon multiple-choice<br />
tests or academic essays (Reeves, 2006).<br />
Higher education has therefore many specific features that make the assessment task even more<br />
complex than at lower educational levels. Rather ambitious evaluation goals include not only<br />
assessment of outcomes, but also reflection of the <strong>learning</strong> process itself by teachers AND students<br />
themselves; assessment techniques comprise peer assessment, self-assessment, authentic<br />
assessment and other sophisticated methods. All of these assessment techniques require that<br />
students evaluate work of the same kind that they themselves are producing, and thus also obtain<br />
experience of realistic evaluation (Sadler, 2005). Besides this, they are aware of assessment criteria<br />
that are transparent and could apply to their own performance.<br />
1.3 Social <strong>learning</strong> perspective<br />
A social <strong>learning</strong> definition will be used further as an explanatory framework for the case study<br />
(Wenger, 2000); it describes this phenomenon as “an interplay between social competence and<br />
personal experience; it is a dynamic, two-way relationship between people and social <strong>learning</strong><br />
systems in which they participate. In this constellation, the necessary competences are negotiated<br />
through the experience of direct participation: engagement in joint enterprise requires that members<br />
of a community are competent to contribute to it, to interact and mutually reflect on this interaction;<br />
and they also share a common repertoire of communal resources such as language, routines, etc.<br />
According to Reed, social <strong>learning</strong> “occurs through social interactions and processes between actors<br />
within a social network, either through direct interaction, e.g., conversation, or through other media,<br />
e.g., mass media, telephone, or Web 2.0 applications” (Reed et al. 2010). A role for the open virtual<br />
space such as Wiki in education is also reported by other authors (Wheeler et al, 2008; Wheeler &<br />
Wheeler, 2009).<br />
2. Case study - MPG&SD course in the VCSE network<br />
The Multiple Perspectives on Globalization and Sustainable Development (MPG&SD) course ran over<br />
the 2009/2010 European winter semester. The course was part of the international VCSE network of<br />
European universities and participated in by students from Germany and the Czech Republic. The<br />
167
Jana Dlouhá et al.<br />
course was distant (fully eLearning) and the planned workload was 150 hours (5 credits). Twelve<br />
students of the 15 who enrolled successfully completed it.<br />
2.1 Educational goals<br />
In the course, students had to develop three literacies – environmental & SD, academic writing, and<br />
ICT literacy. Environmental & SD literacy was related to the theme of the course and students had to<br />
learn about environmental problems and sustainable development policies very generally, and study a<br />
theme of their choice in more depth. Writing literacy was considered to be an important metacognitive<br />
skill necessary to carry out the writing assignment. Students should proceed through all the stages of<br />
the academic writing process; their progress from one stage to the next was monitored. ICT literacy<br />
was a prerequisite and also a byproduct of <strong>learning</strong>. Students’ written assignments as the main<br />
product of the course were displayed in the students’ wiki space, and finally, the tutor compiled a<br />
Globalization Handbook as a common virtual publication available for a future “generation” of students<br />
as a starting point (Dlouhá, 2010). The rules and formal customs of research and academic writing<br />
had to be followed while stressing a creative and critical approach.<br />
A wiki environment as an open space for students’ creative work on the course themes was used:<br />
they wrote their assignments, discussed their content (including a peer review) and shared their views<br />
there (Dlouhá & Macháčková-Henderson, 2008; Dlouhá & Dlouhý, 2009). The role of the <strong>learning</strong><br />
environment was specific; it should stress interdisciplinary aspects of the <strong>learning</strong> process, and<br />
support active <strong>learning</strong> approaches and solutions. These goals were achieved thanks to its more fluid<br />
character which offered different ways of going through content, and provided an interactive character<br />
and a three-dimensional structure (hyperlinks opening successive themes).<br />
2.2 Rules and requirements<br />
No online testing occurred (cheating is difficult to prevent in distant education). Students experienced<br />
academic discourse and the principles of communication in a community of researchers: its<br />
possibilities (freedom of choice of the research theme), rules (research ethics and norms of writing,<br />
respect for the critical procedures of a peer review, formal requirements) and limitations (the necessity<br />
to narrow the theme of interest and to proceed with discipline and efficiency towards justified<br />
conclusions).<br />
The relationship between the different processes could be tracked (peer review versus changes in the<br />
article). Moreover, the <strong>learning</strong> processes could also be assessed – they were documented in the<br />
Wiki environment (where the writing process was realized) in the page history. This option allowed a<br />
comparison of subsequent versions of the article and was used to analyze the influence of a student’s<br />
review on the quantity of the text (supposedly affecting its quality) – see Figure 1.<br />
Figure 1: Comparison of student essays in the wiki environment before and after the peer review<br />
168
Jana Dlouhá et al.<br />
The number of bytes before and after the peer review (documented in the page history) was<br />
compared; the results showed changes especially in the articles that were underdeveloped before<br />
peer review.<br />
The role of communication was important – a negotiated standpoint from participants and a common<br />
decision on a hypothetical problem were required. The evidence of communication was available in<br />
the fora; it could be analyzed and served as a resource for further research. The mutual peer review<br />
process was undertaken in the Wiki space and its results played a role in the quality criterion. The<br />
course outcomes were demonstrated in student portfolios which consisted of: fora discussion<br />
contributions (evidence available in Moodle), student essays and mutual critical reviews of them<br />
(evidence in Wiki).<br />
All actors were actively involved in the <strong>learning</strong> processes and were in permanent interaction with<br />
each other as well as with the <strong>learning</strong> environment – it was not only a resource base, but also a<br />
factor in changing their roles. As subsequent research proved, the electronic environment (the Moodle<br />
system) itself helped to monitor student activity, or control the quantity and timing of assignments<br />
handed in, and thus influenced the performance of authority in the <strong>learning</strong> process. The virtual<br />
electronic environment thus provided some almost invisible powers and thus took over some functions<br />
of the authority that teachers use in face-to-face interactions (Dlouhá, M., 2010).<br />
2.3 The role of, and the tools for assessment<br />
The <strong>learning</strong> goals in the MPG&SD course were focused on the skills and competences needed for<br />
self-regulation of the <strong>learning</strong> process, such as understanding, critical judgment, complex thinking<br />
(awareness of the interdisciplinary context), commitment (the values behind scientific work), and a<br />
constructivist and action-oriented approach (in theory and practice). Assessment goals were process,<br />
not outcome oriented; the assessment options were adjusted to these goals and closely related to the<br />
features and qualities of the articles and other texts produced by students – their essays and all other<br />
contributions in fora were carefully analyzed in each stage of the course. New assessment tools for<br />
evaluation of the specific qualities mentioned above had to be developed – evaluation rubrics were<br />
used for all texts produced in the course. The rubrics assessed to what extent educational goals in<br />
terms of (sustainability oriented) competences demonstrated by the specific properties of the texts<br />
written by students were met (see Table 1).<br />
Table 1: Example of the assessment rubric for a student’s essay – Wiki text (shortened)<br />
Criteria/levels of<br />
fulfillment<br />
Content<br />
Focus<br />
Context<br />
Clarity<br />
Critical approach<br />
10 0<br />
quality of resources & well-founded<br />
argumentation<br />
strong conclusions, values behind<br />
the topic are clear<br />
wide context & core of the problem<br />
identified<br />
logical structure of the text, clear<br />
ideas in sentence, paragraph etc.<br />
balanced text: opposing views<br />
presented<br />
Commitment ethics (writing) & length (text)<br />
Individual input &<br />
risk-taking<br />
Formal features<br />
initiative in researching topic,<br />
originality, independent work with<br />
resources<br />
respecting academic genre, citation<br />
format respected<br />
information is limited; argument is<br />
biased<br />
various themes with no clear<br />
priorities, value orientation not<br />
certain<br />
narrow context, problems not<br />
interrelated<br />
ideas unclear, disconnected;<br />
structure not logical<br />
Competence under<br />
consideration<br />
Knowledge (deep)<br />
Understanding<br />
(research question)<br />
Knowledge (broad)<br />
Information<br />
management<br />
one-sided, demagogic Critical thinking<br />
ethical principles not satisfactorily<br />
respected<br />
no new perspectives; conclusions<br />
not original<br />
mixed genre, citation format and<br />
ethics insufficient<br />
Values<br />
Action competence<br />
Respect for formal<br />
requirements<br />
Assessment tools designed for specific evaluation needs (competences) and online environment<br />
options exploited the evidence of the <strong>learning</strong> outcomes – a portfolio of students’ work was<br />
transparently available in the <strong>learning</strong> space.<br />
169
2.4 Feedback – student perceptions<br />
Jana Dlouhá et al.<br />
Because of the complex character of the skills and competences to be developed within the course,<br />
the principle of student interactive involvement and the rule that the <strong>learning</strong> process should be<br />
reflected by them was applied (due also to the low number of course teachers/tutors and a presumed<br />
lack of evaluation objectivity), while particular attention was paid to the students’ reflection of the<br />
<strong>learning</strong> environment, requirements and outcomes.<br />
Students’ perceptions of the academic quality of courses in distance education are (according to<br />
relevant research) strongly associated with the approaches to studying that they adopt on those<br />
courses (Richardson, 2009). Moreover, by comparing desirable educational goals in terms of<br />
competences with characteristics of the students’ approaches we have identified a deep study<br />
approach, in contrast to a surface approach, as the necessary condition for the development of<br />
desirable skills and competences. The deep study approach can be described as “looking for<br />
meaning in the matter being studied and relating it to other experiences and ideas with a critical<br />
approach. Students adopting a deep approach aim to understand the subject and are intrinsically<br />
interested in, and derive enjoyment from, studying. A surface approach can be thought of as a<br />
reliance on roteLearning and memorization in isolation to other ideas. Surface learners perceive the<br />
task of <strong>learning</strong> as an external imposition and they are externally motivated. They typically treat parts<br />
of the subject as separate entities and fail to integrate topics into a coherent whole.” (Duff, 2004).<br />
It has been argued that students must usually “take a deep <strong>learning</strong> approach in order to develop an<br />
effective understanding of the multidimensional meaning of the complex topic of sustainability or<br />
sustainable development” because complexity of the problem itself requires an active approach and<br />
deep understanding (von Blottnitz, 2006).<br />
To assess the extent to which students applied a deep/surface approach to studying, the<br />
standardized Experiences of Teaching and Learning Questionnaire (ETLQ) was used to explore<br />
student approaches to <strong>learning</strong> and studying (Richardson, 2009). Even though the questionnaire<br />
explores solely the process of <strong>learning</strong>, the nature of this process relates strongly to final educational<br />
outcomes, as has been proved by several studies (Entwistle et al., 2000; Duff, 2003; Mattick et al.,<br />
2004). At the same time, researchers conclude that the studying approach is in turn influenced by the<br />
context of <strong>learning</strong> (Duff, 2004; Parpala et al., 2010). The MPG&SD course is designed with the goal<br />
of supporting the development of students’ competences using web 2.0 tools in an eLearning<br />
environment and we expected that the strong emphasis put on their involvement in communication<br />
and self-directed action will be associated with a deep study approach in the process of <strong>learning</strong>. The<br />
ETLQ questionnaire designed by Entwistle to distinguish different student approaches was applied to<br />
verify this assumption: we used its short 18 item version that considers four dimensions/approaches<br />
to <strong>learning</strong>: the deep approach, monitored studying, organized studying, effort management, and the<br />
surface approach (Mattick et al., 2004). The first three of them relate to desirable ways of studying<br />
and refer to deep involvement in the <strong>learning</strong> process characterized by interest and understanding.<br />
The results of testing are presented in Table 2 - students apparently refer to the prevailing deep<br />
approach and reject the surface <strong>learning</strong> statements. This inquiry (application of a quantitative<br />
questionnaire to a low number of students) was based on the application of a standardized tool whose<br />
validity and reliability has been proved by previous surveys (Mattick et al., 2004; Parpala et al., 2010;<br />
Duff, 2004). A median was used instead of a mean to represent students’ opinions as a more<br />
appropriate measure from such a low number of respondents. The fact that most of the positive<br />
statements resulted in 4 = agree and the prevailing answer to the surface approach was 2 = disagree<br />
supports the hypothesis about encouraging course environment, i.e. context of <strong>learning</strong>.<br />
Color<br />
scale<br />
1 1.5 2 2.5 3 3.5 4 4.5 5<br />
Color scale used in all following tables: 5-point Likert scale (from Strongly disagree = 1 point to<br />
Strongly agree = 5 points)<br />
170
Jana Dlouhá et al.<br />
Table 2: Results obtained in final standardized Experiences of Teaching and Learning Questionnaire<br />
(ETLQ): DA = Deep approach, MS = Monitoring studying, OS = Organized studying and<br />
effort management, SA = Surface approach. 5 item scale, 1 = strongly disagree, 5 =<br />
strongly agree<br />
Indicate to what extent you agree with the<br />
following statement...<br />
Data 2009/2010<br />
(12 students)<br />
median<br />
Demonstration of the <strong>learning</strong> approach<br />
I have usually tried to understand for myself the<br />
meaning of what we had to learn<br />
4 DA: Intention to understand for oneself<br />
In reading for this course, I’ve tried to find out for<br />
myself exactly what the author means<br />
4 DA: Intention to understand for oneself<br />
In making sense of new ideas, I have often<br />
related them to practical or real life contexts<br />
4<br />
DA: Relating ideas (including<br />
constructivist <strong>learning</strong>)<br />
Ideas I’ve come across in my academic reading<br />
often set me off on long chains of thought<br />
3.5<br />
DA: Relating ideas (including<br />
constructivist <strong>learning</strong>)<br />
I’ve looked at evidence carefully to reach my own<br />
conclusion about what I’m studying<br />
4 DA: Use of evidence<br />
It has been important for me to follow the<br />
argument, or to see the reasons behind things<br />
When I’ve been communicating ideas, I’ve<br />
4.5 DA: Use of evidence<br />
thought over how well I’ve got my points across<br />
(so that others understood)<br />
4 MS: Monitoring generic skills<br />
I’ve tried to find better ways of tracking down<br />
relevant information in this subject<br />
3 MS: Monitoring generic skills<br />
I have been revising the work I’ve done to check<br />
my reasoning and see that it makes sense<br />
4 MS: Monitoring understanding<br />
If I’ve not understood things well enough when<br />
studying, I’ve tried a different approach<br />
4 MS: Monitoring understanding<br />
Concentration has not usually been a problem for<br />
me, unless I’ve been really tired<br />
4 OS: Concentration<br />
I have generally put a lot of effort into my studying 5 OS: Effort<br />
On the whole, I’ve been quite systematic and<br />
organized in my studying<br />
4 OS: Organised studying<br />
I’ve organized my study time carefully to make<br />
the best use of it<br />
4 OS: Time management<br />
Much of what I’ve learned seems no more than<br />
lots of unrelated bits and pieces in my mind<br />
2 SA: Fragmented knowledge<br />
I’ve often had trouble in making sense of the<br />
things I have to remember<br />
2.5 SA: Memorizing without understanding<br />
I’ve just been going through the motions of<br />
studying without seeing where I’m going<br />
2 SA: Unreflective studying<br />
I’ve tended to take as guaranteed what we’ve<br />
been taught without questioning it much<br />
2 SA: Unthinking acceptance<br />
This finding was further supported by the results of a questionnaire designed to explore specific<br />
competences developed through the course in a post-course survey using a 5-point Likert scale. The<br />
answers were also compared with students’ expectations about achieving the competences in a<br />
similar pre-course survey and thus provided feedback on the course achievements related to the<br />
<strong>learning</strong> outcomes expected by students entering the virtual environment. For this subjective<br />
reflection on the degree to which students developed a certain competence or have found some<br />
process of <strong>learning</strong> useful see Figures 2 and 3, and for those who are interested in detailed wording<br />
see Tables 3 and 4. Comparison with an objective assessment of students’ achievements undertaken<br />
by the teacher using an evaluation rubric (see above) was not done due to non-comparable<br />
categories used in both cases; nevertheless, visible outcomes in terms of a portfolio of students’ work<br />
(essays, peer reviews, discussions) could not be achieved without using the surveyed competences in<br />
practice.<br />
Students’ perception of <strong>learning</strong> outcomes in terms of competences is in accordance with findings<br />
about the good quality of the <strong>learning</strong> process (manifested as students’ approaches to <strong>learning</strong>).<br />
These findings might support the hypothesis that the method of teaching used in MPG&SD was<br />
appropriate for sustainability-oriented education and the high expectations placed on social <strong>learning</strong><br />
processes in a virtual space were, at least in this limited case, justified. We can state that an open<br />
171
Jana Dlouhá et al.<br />
virtual <strong>learning</strong> environment that promotes independent work, communication and social interaction<br />
fulfilled the teacher’s and students’ expectations to a great extent.<br />
Figure 2: Results obtained in the final questionnaire developed for assessment of competences<br />
Comparison of means with relevant questions in the pre-entry questionnaire (see these questions in<br />
Table 3.)<br />
Table 3: Questions in final questionnaire developed for assessment of competences.<br />
Learning Outcomes Data 2009/2010<br />
Indicate to what extent do you agree<br />
with the following statement:<br />
I would be able to explain the main<br />
topics of the seminar to a third<br />
person.<br />
I increased my ability to be able to<br />
understand and interpret global<br />
issues from multiple perspectives<br />
(disciplinary perspectives e.g.<br />
sociology, economics, philosophy<br />
etc).<br />
I developed confidence in my own<br />
ability to express my own wellinformed<br />
arguments in globalization<br />
and sustainable development<br />
debates.<br />
I increased my ability to plan my time<br />
and study independently.<br />
I developed my ability to write texts<br />
which express my points clearly.<br />
Description of the skill /<br />
competence<br />
Final<br />
12 students<br />
Comprehensive understanding 5<br />
Expectations Preentry<br />
15 students<br />
median median<br />
Multiple perspectives 5 5<br />
Confident argumentation 4.5 4<br />
Planning & studying 4 4<br />
<strong>Academic</strong> writing 4 5<br />
I developed my capacity to evaluate<br />
whether information is biased or<br />
credible.<br />
Evaluation of info 4 4<br />
I developed my computer skills.<br />
I developed my ability to<br />
ICT 4 3<br />
communicate with people from other Cross-national communication 4 4<br />
countries.<br />
I developed my ability to study and<br />
communicate with people from other<br />
disciplinary backgrounds.<br />
I developed my ability to provide<br />
constructive criticism on fellow<br />
students’ work.<br />
I developed my ability to reflect on<br />
the quality of my own work.<br />
Cross-disciplinary<br />
communication<br />
4 4<br />
Constructive criticism 4.5 4<br />
Reflection on own work 5 4<br />
172
Jana Dlouhá et al.<br />
Results compared with previous year of the course and with relevant questions in pre-entry<br />
questionnaire<br />
In the pre-entry questionnaire, modified wording was used: “My expectations of the course can be<br />
characterised by the following sentences …” “I expect to...“<br />
Figure 3: Results received in the final questionnaire developed for assessment of the <strong>learning</strong><br />
process<br />
Comparison of means with relevant questions in pre-entry questionnaire<br />
(see these questions in Table 4.)<br />
Table 4: Questions in the final questionnaire developed for assessment of the <strong>learning</strong> process.<br />
Learning Process Data 2009/2010<br />
Indicate to what extent do you agree<br />
with the following statement:<br />
I developed new ideas and learned<br />
new information through<br />
communication with my fellow<br />
students.<br />
I developed new ideas and learned<br />
new information through discussion<br />
with my tutors.<br />
I developed new ideas and learned<br />
new information through reading<br />
provided materials.<br />
I developed new ideas and learned<br />
new information through researching<br />
my own topics of interest within the<br />
course.<br />
My tutor helped me to find my own<br />
answers about the content of the<br />
course when I needed it.<br />
My tutor provided answers to my<br />
questions about the content of the<br />
course when I need it.<br />
Description of the skill /<br />
competence<br />
Final<br />
12 students<br />
median median<br />
Communication - students 4 4<br />
Discussion - tutors 5 4<br />
Reading - provided materials 5 5<br />
Own research 5 5<br />
Supportive assistance 4 4<br />
Instructive assistance 5 4<br />
Expectations Preentry<br />
15 students<br />
Results compared with previous year of the course and with relevant questions in the pre-entry<br />
questionnaire.<br />
173
3. Discussion<br />
Jana Dlouhá et al.<br />
Qualitative feedback is more appropriate for courses such as MPG&SD. It provided information about<br />
prior knowledge of the course topic, detailed feedback on eLearning tools, course content, discussion<br />
topics, workload, and satisfaction with students’ own performances. Regarding evaluation of trained<br />
skills and competences, we can identify the following items as what students perceived they gained:<br />
language skills (English – all students were non-native speakers), writing (by creating their own wiki<br />
article, developing the proper formal structure of an article, use of citations and references, a true<br />
peer review process, communication: discuss and consider other opinions, and time management.<br />
Looking at students’ open comments on course quality we can again find mostly positive reactions<br />
like: “In total I really liked this course and I am happy that I participated in it. This course I really found<br />
fun to handle. Hope to participate in other course like it.” The last comment was elicited from a<br />
specific statement: “I would recommend this course to my colleagues/fellow students.” Eight students<br />
agreed, while 4 of the 12 provided no response. Critical opinions also appeared. The intellectual<br />
demands of the course were perceived as high. In addition, the whole course was seen as too timeconsuming<br />
and sometimes deadlines were not 100% clear. Nevertheless, the positive reaction to the<br />
course environment resulted in a unanimous appreciation of the tutors’ involvement, which is always<br />
crucial for the creation of a well-functioning eLearning course.<br />
Other researchers combine the results of an analysis of <strong>learning</strong> approaches with other analytical<br />
tools focused on students’ reactions to teaching. This is only possible when a certain number of<br />
students is analyzed. For example, Noel Entwistle, Velda McCune and Jenny Hounsell (2002) report<br />
on research that looks for relationships between approaches to studying and the teaching-<strong>learning</strong><br />
environment carried out within several courses. They find the concepts and categories used to<br />
describe general differences in studying to be a valuable analytical framework that allows<br />
consideration of how individuals study and shows how various aspects of a teaching-<strong>learning</strong><br />
environment may affect student engagement in the course. Thus, the effects of the teaching-<strong>learning</strong><br />
environment, as perceived by the students, in terms of their deep approach to it might be assessed to<br />
a certain extent. In addition, students’ perceptions of a particular course might be compared with their<br />
general approach to studying, so that a pedagogical effect would be indicated. These authors<br />
concluded that the ETLQ questionnaires are working effectively and are able to detect changes in<br />
approaches to studying depending on changes in course design aimed at enhancing teaching<strong>learning</strong><br />
environments to support high-quality <strong>learning</strong>, or compare the responses of students who<br />
had first-hand experience of the innovation with those who had not.<br />
In general, the deep studying approach should result in good quality outcomes and academic<br />
performance (Entwistle et al., 2000; Duff, 2003). We have not overestimated the results of the<br />
evaluation of on our course with a limited number of participants due to which we were not able to<br />
proceed with a deeper statistical analysis; we have simply presented feedback on possible evaluation<br />
tools that could be used for the assessment of social <strong>learning</strong> processes that give educational<br />
outcomes different to those achieved in traditional HE settings.<br />
4. Conclusion – the applied social <strong>learning</strong> concept in an eLearning course<br />
The HE course presented as a case study is a model of the applied concept of social <strong>learning</strong> in<br />
eLearning. In the MPG&SD course, students had to learn from the social environment with the<br />
support of an electronic environment. The manner of involvement should follow the principles of (and<br />
communication in) the academic community in general; the teaching method used and the eLearning<br />
environment selected were mediators of the educational aims. The eLearning environment was<br />
flexible enough to provide the options required, and, moreover, it provided evidence so that the<br />
<strong>learning</strong> processes could be researched in different stages, as the work of the students was<br />
continuously documented. Evidence of participants’ contributions in the forum discussion space<br />
provided the opportunity to observe the social aspects of scientific work and the principles of<br />
academic dialogue – the roles of participants in the dialogue, the distribution of authority, possibilities<br />
of communication across boundaries, the added value of a multicultural environment, etc. The<br />
research dimension was highlighted in an eLearning environment that makes external evaluation<br />
possible.<br />
Some of explorations undertaken and experiences obtained would not be possible without the<br />
multicultural and interdisciplinary <strong>learning</strong> environment that was achieved within the VCSE<br />
174
Jana Dlouhá et al.<br />
international network of cooperation. International collaboration generally promotes intercultural<br />
understanding, fosters the exchange of information on global problems, supports the transfer of<br />
experiences and provides a broader sustainability context (Vann et al., 2006).<br />
The course represented an experiment in teaching, with well documented and transparent outcomes<br />
that were tenable with regards to academic skills – student academic essays compiled in the final<br />
edition of the Globalization Handbook, and their reviews of colleagues’ articles in the discussion part<br />
of the wiki page (and also numerous fora contributions). It was an example of the applied “community<br />
of practice” model in teaching where the <strong>learning</strong> environment played a crucial role, especially with its<br />
social aspects (actors and their roles, communication tools and means), but also the information<br />
management options, etc. The social <strong>learning</strong> methodology – teaching through involvement in the<br />
community, accepting and applying its discourse – brought about an educational experience that<br />
should be explored further.<br />
References<br />
Biggs, J.B. (1987) Student Approaches to Learning and Studying. Research Monograph. Australian Council for<br />
Educational Research Ltd., Radford House, Australia, 14–19.<br />
von Blottnitz, H. (2006) Promoting active <strong>learning</strong> in sustainable development: experiences from a 4th year<br />
chemical engineering course. Journal of Cleaner Production 14(9-11), 916-923.<br />
Burandt, S., Barth, M. (2010) Learning settings to face climate change. Journal of Cleaner Production 18(7), 659–<br />
665.<br />
Dlouhá, J., Macháčková-Henderson, L. (2008) ELearning as an opportunity for virtual mobility and competence<br />
development within European Universities. In: Williams R; Remenyi, D. (ed.) The Proceedings of the 7th<br />
European Conference on eLearning (ECEL). NR Reading: <strong>Academic</strong> Publishing <strong>Limited</strong>. Book 1, pp. 297–<br />
307<br />
Dlouhá, J., Dlouhý, J. (2009) Use of Wiki Tools for Raising the Communicative Aspect of Learning. In: Remenyi<br />
D. (ed.) Proceedings of the 8th European Conference on eLearning. NR Reading: <strong>Academic</strong> <strong>Conferences</strong><br />
Ltd. 165-173<br />
Dlouhá, J. (2009a) General educational qualities and the term “competence”. Envigogika 4(1) Online. Retrieved<br />
on 2009-06-16, from http://envigogika.cuni.cz/index.php/cs/texty/20091/299-obecne-vzdlavaci-kvality-apojem-kompetence<br />
Dlouhá, J. (2009b) Competences in environmental education. Envigogika 4(1) Online. Retrieved on 2009-06-16,<br />
from http://envigogika.cuni.cz/index.php/cs/texty/20091/300-kompetence-v-environmentalnim-vzdlani-<br />
Dlouhá, J. (2010) Globalisation Risks and Opportunities. VCSEWiki. Online. Retrieved on 2010-08-31, from<br />
http://www.czp.cuni.cz/vcsewiki/index.php?title=Globalisation_Risks_and_Opportunities&oldid=1350<br />
Dlouhá, M. (2010) Authority in virtual education. Charles University in Prague, FSV, ISS, Praha.<br />
Duff, A. (2003) Quality of Learning on an MBA Programme: The Impact of Approaches to Learning on <strong>Academic</strong><br />
Performance. Educational Psychology 23, 123-139<br />
Duff, A. (2004) The Revised Approaches to Studying Inventory (RASI) and its Use in Management Education.<br />
Active <strong>learning</strong> in Higher education 56(5), p. 57<br />
Entwistle, N. (2000). Promoting deep <strong>learning</strong> through teaching and assessment: conceptual frameworks and<br />
educational contexts. TLRP conference, Leicester.<br />
Entwistle, N., Tait, H. & McCune, V. (2000) Patterns of Response to an Approaches to Studying Inventory Across<br />
Contrasting Groups and Contexts. European Journal of the Psychology of Education 15, 33-48<br />
Entwistle, N., McCune, V., & Hounsell, J. (2002). Approaches to studying and perceptions of university teaching<strong>learning</strong><br />
environments: Concepts, measures and preliminary findings. Enhancing Teaching and Learning<br />
Environments in Undergraduate Courses Occasional Report, 1.<br />
Ferrer-Balas, D., Buckland, H., de Mingo, M. (2009) Explorations on the University's role in society for<br />
sustainable development through a systems transition approach. Case-study of the Technical University of<br />
Catalonia (UPC). Journal of Cleaner Production 17(12), 1075-1085.<br />
Lukman, R., Krajnc, D., Glavic, P. (2009) Fostering collaboration between universities regarding regional<br />
sustainability initiatives - the University of Maribor. Journal of Cleaner Production 17(12), 1143-1153.<br />
Mattick, K., Dennis, I. & Bligh, J. (2004) Approaches to <strong>learning</strong> and studying in medical students: validation of a<br />
revised inventory and its relation to student characteristics and performance. Medical Education 38(5),<br />
535-543<br />
Parpala, A., Lindblom-Ylänne, S., Komulainen, E., Litmanen, T. & Hirsto, L. (2010) Students’ Approaches to<br />
Learning and their Experiences of the Teaching-<strong>learning</strong> Environment in Different Disciplines. British<br />
Journal of Educational Psychology 80, 269-282<br />
Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and<br />
L. C. Stringer (2010) What is social <strong>learning</strong>? Ecology and Society 15(4): r1. [online] URL:<br />
http://www.ecologyandsociety.org/vol15/iss4/resp1/<br />
Reeves, T.C. (2006) How do you know they are <strong>learning</strong>? The importance of alignment in higher education.<br />
International Journal of Learning Technology, 2(4), 294–309.<br />
Richardson, J.T. (2009) Face-to-face versus online tutoring support in humanities courses in distance education.<br />
Arts and Humanities in Higher Education 8(1), 69.<br />
175
Jana Dlouhá et al.<br />
Sadler, R.D. (2005) Interpretations of criteria-based assessment and grading in higher education. Assessment &<br />
Evaluation in Higher Education 30(2), 175–194.<br />
Vann, J., Pacheco, P., Motloch, J. (2006) Cross-cultural education for sustainability: development of an<br />
introduction to sustainability course. Journal of Cleaner Production 14(9-11), 900-905.<br />
Wenger, E. (2000) Communities of practice and social <strong>learning</strong> systems. Organization 7(2), 225.<br />
Wheeler, S., Yeomans, P., Wheeler, D. (2008) The good, the bad and the wiki: Evaluating student-generated<br />
content for collaborative <strong>learning</strong>. British Journal of Educational Technology, 2008, 39(6), 987–995.<br />
Wheeler, S., Wheeler, D. (2009) Using wikis to promote quality <strong>learning</strong> in teacher training. Learning, Media and<br />
Technology, 2009, Vol. 34, No. 1, March, 1-10<br />
176
Putting Things in Context - Designing Social Media for<br />
Education<br />
Jon Dron, Terry Anderson and George Siemens<br />
TEKRI, Athabasca University, Canada<br />
jond@athabascau.ca<br />
terrya@athabascau.ca<br />
gsiemens@athabascau.ca<br />
Abstract: The rich promise of social software in formal education can be offset by a clash between hierarchical<br />
organisational structures and the bottom-up, distributed nature that characterizes network development and<br />
growth. Learners often experience confusion when using social networking systems within formal education<br />
systems, negating many of the potential benefits of sharing, collaboration, communication and personal<br />
ownership associated with social networking systems. This often leaves learners and their teachers lost or at<br />
least disorientated in social space. Social networking systems are mainly based on explicit individual social<br />
connections, while students and staff in academia constantly shift between overlapping but delineated<br />
hierarchically organised community contexts like classes, committees, research groups, centres and schools as<br />
well as less formal person-to-person networks. Each context presents different needs for communication, shared<br />
resources and connectivity. Most existing educational social software systems blur these contexts into a single,<br />
confused and confusing sub-optimal space that is neither fully social and user controlled nor fully institutionally<br />
controlled. In this paper we describe a set of partial solutions that we are evolving for the Elgg system, providing<br />
multi-faceted profiles for both users and groups to allow control of content, presentation and audience for shared<br />
artefacts, catering for different social, organisational and personal/group contexts. A facet is represented as a<br />
page comprised of draggable widgets. Unlike the static views often associated with e-portfolios, these facets can<br />
be interactive, inviting comment, assessment and other responses in specific contexts. Facets can be moulded to<br />
fit the shifting contexts of academic activities and leisure lives, thus reducing the confusion of network, group, set<br />
and personal spaces that besets current social software use in education, without losing the personal control,<br />
sociability and ownership that makes it valuable in the first place.<br />
Keywords: social media, higher education, technology-enhanced <strong>learning</strong><br />
1. Introduction<br />
We and others (Dron and Anderson, 2009a; Ford et al., 2011; Sebastion et al., 2009) have observed<br />
the enormous potential of social media (including but not limited to blogs, file sharing, link sharing,<br />
social tagging, wikis, social networking, microblogging) to enable methods, pedagogies and forms of<br />
<strong>learning</strong> that earlier technologies have failed to support. A clear set of affordances makes their use<br />
compelling, including:<br />
Content<br />
Sharing of found and created objects<br />
Emergence of patterns, computer augmented and visible<br />
Authenticity as education activity aligns with business and social activity<br />
Connection<br />
Discovery of others with whom to learn<br />
Leveraging networks that go beyond the formal classroom or workplace community<br />
Serendipity as networks and sets interconnect<br />
Sustainance of sociability with a positive association of social network use and traditional forms<br />
of social contact (Hua Wang and Wellman, 2010)<br />
Control<br />
Empowerment to be both a reader and a writer<br />
Adaptability to varying needs due to flexible and mashable (soft) technologies<br />
Communication<br />
Collaborating in teams and groups<br />
Engagement and motivation brought on by persistence, visibility to and interaction with others<br />
177
Jon Dron et al.<br />
Social networking is a feature, not a destination, of most social systems (Anderson, 2007) and, apart<br />
from in a few high-profile sites, it is a necessary but not sufficient condition for them to operate. The<br />
largest educational benefit arguably lies not in the ability to connect with friends and colleagues but in<br />
the ability to own, build and create a stake in a shared social space, to create and content, to interact<br />
with others, in an equal space where all have the same capabilities and in which there is no inherent<br />
hierarchical structure. The benefits relate not only to the potential to be constructors and consumers<br />
of valuable <strong>learning</strong> content that makes <strong>learning</strong> easier, but to improve intrinsic motivation and<br />
engagement through the enabling of pedagogies that are based on learner control and the sense of<br />
connectedness with others that is entailed (Ratelle et al., 2007; Shroff and Vogel, 2009).<br />
For all their potential benefits they present new problems – issues of trust, lack of centre, privacy and<br />
information overload can weaken the efficacy of social software systems, especially for intentional<br />
<strong>learning</strong>. This is not the kind of space in which learners in educational institutions have become<br />
habituated. For many it is a bewildering and potentially frightening or frustrating new context (Kop,<br />
2011; Mackness et al., 2010; Turkle, 2011). We will argue that this is mainly because the primary<br />
social and educational form within formal institutions is the closed group, such as the class, tutorial or<br />
seminar, whereas social systems occupy shifting social spaces that include social networks and sets.<br />
This leads to a mismatch of organisation, activity and technology that can deter, demotivate and<br />
sidetrack learners. We have addressed the nature of social forms in such systems in previous papers<br />
(Dron and Anderson, 2007; Dron and Anderson, 2011, under review) but provide a summary here for<br />
convenience.<br />
1.1 Groups<br />
The group has, historically, been the primary social and organizational form in formal education.<br />
Classes, tutorial groups, seminars, faculties, schools, centres, and the institutions themselves embed<br />
and embody this form deep in their DNA. Groups are formed by people, intentionally and explicitly.<br />
Groups tend to have roles and rules, usually involving leadership, often delegating aspects of<br />
responsibility to members that differentiate them from non-members. Groups in education, notably<br />
courses, tend to have a defined purpose and often operate over a defined period. Groups tend to<br />
have implicit and explicit rules of acceptable and unacceptable behaviours that differ or represent a<br />
subset of rules that exist outside the group. Groups are defined as much by who they exclude as who<br />
they include (Shirky, 2008). It is very rare to be a member of a group without knowing that one is a<br />
member. Groups rely on trust and knowledge of sameness and inclusion to create a safe, but<br />
cloistered <strong>learning</strong> context.<br />
1.2 Sets<br />
An overlapping but distinct kind of social entity common in education is the set. In <strong>learning</strong> situations,<br />
we often find ourselves interacting with an indeterminate number of typically unknown individuals that<br />
are bound by a shared attribute. For example, when we publish books or papers they are seldom<br />
aimed at specific people or a known group but relate to a subject matter and/or audience<br />
demographics: we thus treat potential readers as a set of people with shared characteristics. Sets are<br />
not only about publication. A show of hands in a classroom might be asked for simply to count the set<br />
of those hands in order to make a decision. When we tag a blog post we are explicitly categorising<br />
that post into a set, and we generally have an expectation that potentially unknown others with an<br />
interest in that tag will find it useful. Sets are about categories: humans and some programmed<br />
machines make decisions about what category is of interest in order to define a set. Having made that<br />
decision then interesting emergent features may be observed and analysed, especially when we look<br />
at intersections between them – people with doctorates tend to buy more scholarly books than people<br />
with high school diplomas, for example.<br />
1.3 Networks<br />
A network is formed of connections between nodes. To be part of a network means to be linked to<br />
one or more of the nodes within it. Unlike the group but like the set, we may not be aware of<br />
membership of a network: we join one just by connecting to one of its nodes. Local connections may<br />
be all that we ever see of it – our friends and perhaps their friends, for instance, may link to a network<br />
of friends that extends across much of the planet. Networks often have no fixed size limits but, like<br />
sets, can occur within other social organisational forms, especially within groups. Networks can<br />
connect us directly or indirectly, not only through people but also through objects. For example, a rich<br />
178
Jon Dron et al.<br />
network of connections may be linked through a popular blogging site and the blogs themselves may<br />
be connected through trackbacks and blogrolls to each other, allowing people to traverse a network<br />
structure in order to explore connected ideas, conversations and people. Networks and sets may<br />
overlap and help to form and inform each other. For example, through set-oriented tags displayed as<br />
a tagcloud, we may follow links and discover networked connections. We may treat a network as a set<br />
by counting similar objects within it, and treat a group as an analysable network. We may analyse the<br />
structure of the network itself to discover the patterns and shapes of connections within it – who is<br />
most connected, how strong are the different kinds of connections and so on. Learning in networks is<br />
quite different from <strong>learning</strong> in groups (Downes, 2007).<br />
Despite our efforts above to differentiate these aggregations of the many we do not wish to imply that<br />
these are hierarchical or exclusive categories. It is useful to think of them as being part of emergent<br />
organization that Kontopoulos (Kontopoulos, 1993) describes as a heterarchy. "Hierarchy means<br />
complete inclusion and supersession; heterarchy means partial inclusion and tangledness." (p.64)<br />
2. Why this matters<br />
Different virtual social spaces are implicitly or explicitly designed to support different social forms. A<br />
system designed to support collaboration and centrally managed closed groups may have different<br />
functions and be informed by different design decisions than one built to support bottom-up network<br />
formation, even though they may share many of the same tools. This profoundly affects the nature of<br />
interactions within them. ‘We shape our dwellings and afterwards our dwellings shape our lives’<br />
(Churchill, 1943) or, as McLuhan said it, ‘we shape our tools and then our tools shape us’ (McLuhan,<br />
1964). Using a system that supports a particular social form may often lead to difficulties for those<br />
trying to use it to support a different form, even though the tools and affordances can be used that<br />
way. By way of analogy, a lecture theatre may be used to support small group work, but the design of<br />
the tiered forward-facing seats makes it more difficult to conduct smaller team discussions than to<br />
conduct lectures.<br />
As McLuhan (ibid) also noted the content of any new media is (at first) always the content of the older<br />
media. The design of early <strong>learning</strong> management tools was based upon the existing shapes and<br />
forms found in educational and training institutions and enables the shapes and forms of pedagogies,<br />
processes and communities that had arisen in those physical settings. Unfortunately, many of those<br />
forms had little to do with pedagogy or process and everything to do with physical constraint,<br />
economies of scale and path dependencies. Universities created libraries because books were hard<br />
to come by, lectures to share the scarce resource of learned scholars, classes and timetables to<br />
efficiently allocate those resources, and group-oriented pedagogies because of the constraints of<br />
physical space. Distance universities are, despite freedom from many of those restrictions, equally<br />
driven by constraints: pedagogies of isolation, mainly behaviourist or cognitivist in style, have been<br />
necessary in a technological landscape where communication was slow or prohibitively expensive.<br />
They also inherited the limitations of traditional universities because they had to compete in the same<br />
ecology and thus had to satisfy expectations (and prejudices) about forms of accreditation, roles,<br />
organisational structures and so on that stretched back to medieval forebears (Norton, 1909).<br />
Whether at a distance or face-to-face, the combination of structural constraints that have formed<br />
universities made the group by far the most natural form in traditional institutional <strong>learning</strong>. It was the<br />
only plausible form of organisation that allowed sharing of scarce resources and afforded the benefits<br />
of both individual didactic teaching and social <strong>learning</strong>.<br />
In contrast, educational social software typified by systems such as Elgg, Mahara, and BuddyPress is<br />
predominantly network-oriented. Although all such systems do have support for groups (often<br />
explicitly) they are architecturally predisposed to network interactions. They also usually offer strong<br />
support for indeterminate sets (implicitly through features like tags, ‘like this’ buttons, ‘what’s hot’<br />
indicators, as well as through publication tools). Content and interaction are discovered and shared<br />
through links with individuals and their content or through set-based features like popularity metrics<br />
and tags, less through formal groups. Even when formal groups are available, it is generally<br />
impossible to ignore or hide the set and net structures that are parallel, overlaid or overlapping, and<br />
which compete constantly for attention, diffusing and blurring the boundaries that are innate in group<br />
forms. This allows for more open, connected <strong>learning</strong> that is not bound by the limits of the classroom,<br />
that can better fit changing and shifting needs, that allows richer knowledge construction,<br />
serendipitous knowledge discovery and great opportunities for self-guided, motivated <strong>learning</strong>.<br />
179
Jon Dron et al.<br />
However, it is also the main source of complaint from learners who are sometimes forced into these<br />
social systems through a formal class requirement (Kop, 2011; Dron and Anderson, 2009c; Mackness<br />
et al., 2010). The innate blurring of focus in their set/net orientation means that those who are seeking<br />
a structured, usually teacher directed, path are constantly struggling with the mass of emergent<br />
structure and diffuse connections that co-exist in that environment. Finding relevant content and<br />
people is hard, keeping track of (creating) a beaten path is difficult. Leaders of groups (typically<br />
teachers) have to constantly intervene, advise, guide and manipulate in order to allow learners to<br />
maintain a focus and to avoid the sense of placelessness that characterises network or set-oriented<br />
social spaces.<br />
3. Problems and solutions<br />
3.1 Problems of control<br />
If teachers give control of the social space to learners then their own control is diminished. Without<br />
the formal structures and cues that overcome such inherent egalitarianism in traditional classes<br />
(timetables, lecture halls, etc) nor the structural technologies that replicate such forms in <strong>learning</strong><br />
management systems (course areas, hierarchical roles, formal announcement systems etc) teacher<br />
contributions have no more inherent weight than those of their students. While teachers, classes and<br />
administrators are recognisable entities that carry some weight, the design of social systems does not<br />
reflect and magnify such roles. This means that the use of social systems for formal education can<br />
confuse learners because their structure does not reflect that of an academic context in which they<br />
are comfortable. To tweak this to match the organisational structure would to diminish the value of the<br />
very things for which we use such systems: if we create hierarchies, roles and privileged content (or,<br />
less regulating, an interface that enhances such things) we take control from the learners and<br />
reinforce power structures that may affect social interaction considerably and greatly diminish the<br />
unique <strong>learning</strong> affordances of networked contexts.<br />
3.2 Problems of context shifting<br />
Academia is a world packed with rapidly shifting contexts in which different activities, groups,<br />
networks and sets constantly change. This relates to at least three facets, the social, the personal and<br />
the group.<br />
3.2.1 Social facets<br />
As social beings, we adopt different facades in different circumstances (Goffman, 1959). This<br />
subsumes but is a looser concept than that of ‘identity’ as described by (Wenger, 1998). We may<br />
simply choose to show different things to different people at different times. A profile meant for one’s<br />
friends will often be very different from a profile for one’s employers or classmates, for example. In<br />
academia, we constantly and continually shift between different social and working contexts, usually<br />
many times a day. The academic world is massively discontinuous, bounded not only by the different<br />
needs of different classes but also of social, clubs, committees and so on, as well as social or work<br />
networks and sets that may interest them. Few other contexts exhibit the same formalised and<br />
frequently changing diversity that is found in almost every academic setting.<br />
3.2.2 Personal facets<br />
Some tools, such as Elgg or Wookie, are at least partly presented as personal <strong>learning</strong> environments<br />
(PLEs) which provide a single, configurable <strong>learning</strong> space in which we might aggregate the things we<br />
need when we are <strong>learning</strong>. However, in academia, personal needs change with alarming regularity.<br />
The resources and toolsets that we may need in one class may be entirely different from those in<br />
another, or overlap to different degrees. The groups that interest us when engaged in hobbies are not<br />
the same as those that we might look for when working on a research project. A single personal<br />
<strong>learning</strong> environment is too broad and non-specific in the context of most formal education.<br />
3.2.3 Group facets<br />
A similar problem besets collaborative groups working in social spaces. On the one hand they use<br />
such spaces to support their working practice while, on the other, they often represent the group as its<br />
external face to the world. Unfortunately, the needs of process are typically different from the needs of<br />
product, though there will usually be overlaps – publications, for example, may be both objects to<br />
180
Jon Dron et al.<br />
work on and outputs of the group. Many academic groups (notably classes) also shift context on a<br />
regular basis as they change activities and topics: they are not static objects, but exist in<br />
discontinuous phases.<br />
3.3 Current solutions<br />
There are many ways that the structural conflict between learner control and teacher guidance can be<br />
overcome, but all current solutions have their weaknesses. We discuss these in the following<br />
subsections.<br />
3.3.1 Pick and mix<br />
Until now, the most effective way to present different social facets was through creating identities or<br />
‘sockpuppets’ on different systems – a professional profile on LinkedIn, a profile for friends on<br />
Facebook, an academic profile on Academia.edu, for example. Similarly, the simplest way to deal with<br />
the conflict between social and formal systems is to use both, using the formal system to guide and<br />
the informal systems to give the social interaction, connectivity, serendipity and other benefits.<br />
Unfortunately this can be no less confusing, adds greatly to the cognitive and technical load for<br />
learners and teachers, and often remains a very teacher-controlled process.<br />
3.3.2 Group tools<br />
A similar concern relates to the use of group tools provided by a social system. Because they are<br />
situated in a space that emphasises local connection and collective crowd-mining, learners are a click<br />
away from chaos. For instance, it is common to aggregate personal blogs by group members in the<br />
group environment, but these exist outside the group in the individual’s networked space, where the<br />
surrounding context is divorced from the guided space of the group, with nothing but browser<br />
navigation buttons to lead back to the centre.<br />
3.3.3 Collective tools<br />
Using the system itself to mine the collective activities of its users, typically through analytics of sets<br />
or nets, such as through collaborative recommendations, reputation-mining, automated clustering, tag<br />
clouds and so on, can help to provide some structure. Unfortunately that structure, usually based on a<br />
single dimension of implicit or explicit preference or pre-stated goal, is almost bound to be at odds<br />
with the formal structure of a course and is as likely to lead to confusion as it is to good <strong>learning</strong>. And,<br />
again, once the visitor has followed links determined by that structure, they are in a networked space<br />
where that structure plays no further role.<br />
3.3.4 Hybrids<br />
Content management systems with social elements such as Drupal, Joomla, Plone, LifeRay, etc,<br />
tread a middle ground between the highly formal <strong>learning</strong> management system and the informal social<br />
network. Roles and hierarchies are built into such systems, allowing them to be as controllable as<br />
needed, from an administrator’s perspective. Unfortunately, for the end-user, the benefits of bottomup<br />
network and set structures are heavily constrained by what those administrators, decide is of<br />
value. Although such systems allow an element of social networking and crowd processes to emerge,<br />
the boundaries are regulated. This can be leveraged to provide a more flexible learner-centric<br />
approach than a system that directly embodies traditional teaching models, but it does not provide the<br />
benefits of ownership, control, serendipity and connection of a full-blown socially networked system.<br />
3.3.5 Filtering<br />
Existing social <strong>learning</strong> environments mostly treat an individual as a single entity defined by<br />
structured/semi-structured data in a profile, but most offer some filtering. Systems such as Elgg,<br />
Orkut, Google Plus and Diaspora provide fine-grained access control at an individual post level,<br />
allowing users to easily specify who gets to see what of a user’s content. Other systems such as<br />
BuddyPress, Facebook and LinkedIn work at a level of user rights rather than individual permissions<br />
so, for the most part, make it possible to specify kinds of things that can or cannot be seen by certain<br />
people. Either way, rights or permissions enable control over what is seen, but not how it is seen: the<br />
single profile is filtered, not transformed to cater for different visitors. This does not fit social reality nor<br />
the discontinuous nature of formal educational systems, and it makes it difficult to separate out the<br />
many roles and groups of related activity that a typical learner is engaged in. Mahara’s Views go<br />
181
Jon Dron et al.<br />
further in enabling the selective publication of diverse content from a profile, but these are solely<br />
meant for special-purpose presentations of portfolios and are not in themselves meant to assist the<br />
dynamic process of social <strong>learning</strong>.<br />
4. Context switching<br />
The problem of overlapping social spaces could be reduced considerably if we were to reduce the<br />
overlap, not just in an aggregated group space but also in the network and set spaces to which that<br />
space leads, maintaining continuity and connection throughout. However, to do this without imposing<br />
a pre-ordained hierarchy is difficult to manage without breaking the fragile and egalitarian<br />
connectedness of the network space. Ideally, learners themselves should be able to control the way<br />
that social spaces overlap and shift between their changing contexts as and when desired.<br />
We have built a tool to extend the Elgg social framework that allows learners, both individually and in<br />
groups to shift easily between different personal and social contexts, to aggregate different things in<br />
different ways for each aspect of their <strong>learning</strong> (and, if they wish, social and working) lives. The tool<br />
can present different facets to different, specified, groups of people, from entirely private workspaces<br />
to entirely public facades. Each facet can have an entirely different content, appearance and layout<br />
from that of every other, and they can overlap as much as needed.<br />
4.1 Configurable facets<br />
From an individual’s perspective, facets appear as a list of pages of a personal profile. Individuals can<br />
add as many of these pages as they wish, and adjust permissions, layout and appearance as they<br />
wish. By moving pages up and down the list, they can make different facets appear by default to<br />
different people. A visitor to their profile will see the first facet to which they have access. So, for<br />
example, a learner might create a personal page limited only to friends at the top of the tab set, then<br />
professional portfolio accessible to the general public next down the list. Friends will see the personal<br />
page, anyone else will see the public portfolio.<br />
Figure 1: A configured facet showing option to choose other facets<br />
4.2 Configurable widgets<br />
The content of facets is constructed using configurable widgets. Elgg and the Elgg community provide<br />
a wide range of these and they can be used to display many different kinds of content from within and<br />
beyond the system, including blog posts, group membership, pictures, wiki pages, bookmarks,<br />
external RSS feeds, arbitrary text, podcasts, files and so on. In order to provide greater control for the<br />
182
Jon Dron et al.<br />
individual, we have extended the functionality of many of these widgets to allow fuller control over<br />
what is shown and how it is presented. For instance, rather than simply showing a user’s own posts, a<br />
blog widget can show selected individual posts belonging to friends and shared group members, in<br />
different sort order, with its own title, with varying-sized snippets of the original post, displayed<br />
differently. In effect, this offers a kind of desk-top publishing tool for the social web, drawing dynamic<br />
and/or static content from across the social site and beyond into different pages for different<br />
purposes.<br />
Figure 2: onfigured widget<br />
Figure 3: Widget configuration options<br />
183
4.3 Differentiated friends<br />
Jon Dron et al.<br />
Elgg lets users enable access control that is private, public, logged-in users, people they are<br />
following, and the various groups of which they are members. It also supports the functionality of<br />
collections, which enable an individual to subdivide those they are following (in some systems these<br />
are known as ‘friends’ or ‘contacts’) into subsets that can be used to control access to posts. We have<br />
built a plugin to make it explicit at the time of following what kind of relationship is being specified<br />
using relationship tags, or ‘r-tags’ – friend, colleague, classmate, etc. There are no limits on the tags<br />
that people may use to describe these relationships and use these to grant or deny access. This<br />
makes it very simple for access to be granted to (for instance), friends, teachers, classmates in a<br />
course, employers, family and so on.<br />
Figure 4: Relationship tags<br />
5. The benefits<br />
Context switching provides a centre for learners and groups to organise not only groups but also<br />
networks and sets in the learners’ own manageable spaces. Contextual profiles are only ever one<br />
click away within the system so it is impossible to get lost when following links to personal, networked<br />
or set spaces. Each facet adjusts dynamically to the social content provided by its widgets, updating<br />
with new relevant posts as they are posted, yet the learner retains control over exactly which<br />
elements are displayed. When visiting another learner’s profile, visitors can choose which facets they<br />
see, if they have permission to do so. So, should they visit the profile of someone who is a friend as<br />
well as a member of other classes or communities, they are able to see only those facets the friend<br />
wishes them to see in each specified context, greatly reducing the confusion and potential for<br />
becoming lost in social space. Similarly, owners of groups can create facets for users, networks, sets<br />
activities, roles, presentations to the outside world and much more, with widgets displaying specified<br />
activities of users outside the group and within it, making them far more self-contained and intuitive to<br />
navigate, combining the parts of networked and set space that are most relevant to the group in a<br />
given context. The result is that a community with the context switcher in place can switch between<br />
networked, group and set modes of engagement without any central authority determining how it is<br />
organised.<br />
6. Conclusions<br />
The system we have presented here is just one of many possible solutions to the problem of shifting<br />
and overlapping social and <strong>learning</strong> contexts. As we implement, evaluate and refine the toolset we<br />
expect that the solution will evolve and we expect to see increasing recognition of the importance of<br />
context in other social systems that may lead to approaches that are better or more suited to learner<br />
needs than what we have described here. Our purpose in highlighting the importance of shifting<br />
contexts is to help others to recognise ways that the current generation of social systems for <strong>learning</strong><br />
need to move beyond a naïve, single-faceted view of the people that use them to a richer more<br />
human set of tools that support the complex context(s) in which we find ourselves, in which multiple<br />
184
Jon Dron et al.<br />
groups, networks and sets coexist and intertwine, shaping and shaped by the activities and<br />
organisational forms their members engage with.<br />
References<br />
Anderson C. (2007) Social networking is a feature, not a destination. Available at:<br />
http://thelongtail.com/the_long_tail/2007/09/social-networki.html.<br />
Churchill W. (1943) HC Deb 28 October 1943 c403.<br />
Downes S. (2007) Learning Networks in Practice. In: Ley D (ed) Emerging Technologies for Learning. BECTA,<br />
19-27.<br />
Dron J and Anderson T. (2007) Collectives, Networks and Groups in Social Software for E-Learning. In:<br />
Bastiaens T and Carliner S (eds) E-Learn 2007. Quebec City, Canada: AACE, 2460-2467.<br />
Dron J and Anderson T. (2009a) How the crowd can teach. In: Hatzipanagos S and Warburton S (eds) Handbook<br />
of Research on Social Software and Developing Community Ontologies. Hershey, PA: IGI Global<br />
Information Science, 1-17.<br />
Dron J and Anderson T. (2009b) Lost in social space: Information retrieval issues in Web 1.5. Jounral of Digital<br />
Information 10.<br />
Dron J and Anderson T. (2009c) Lost in social space: Information retrieval issues in Web 1.5. Journal of Digital<br />
Information 10.<br />
Dron J and Anderson T. (2011, under review) Groups, Sets and Nets in Online Learning. Journal of Computer-<br />
Assisted Learning.<br />
Ford N, Bowden M and Beard J. (2011) Learning together: using social media to foster collaboration in higher<br />
education. In: Wankel L (ed) Higher Education Administration with Social Media (Cutting-edge Technologies<br />
in Higher Education, Volume 2). Bingley, UK: Emerald Group Publishing, pp.105-126.<br />
Goffman E. (1959) The Presentation of Self in Everyday Life, New York: DoubleDay.<br />
Hua Wang and Wellman B. (2010) Social Connectivity in America: Changes in Adult Friendship Network Size<br />
From 2002 to 2007. American Behavioral Scientist 53: 1148-1169.<br />
Kontopoulos K. (1993) The logics of social stracture, Cambridge: Cambridge Unievrsity Press.<br />
Kop R. (2011) The challenges to connectivist <strong>learning</strong> on open online networks: Learning experiences during a<br />
massive open online course.<br />
Mackness J, Mak SFJ and Wiliams R. (2010) The ideals and reality of participating in a MOOC. In: Dirckinck-<br />
Homfield L, Hodgson V, Jones C, et al. (eds) 7th International Conference on Networked Learning. Aalborg,<br />
Denmark: University of Lancaster, 266-274.<br />
McLuhan M. (1964) Understanding media: The extensions of man, Toronto: McGraw-Hill.<br />
Norton A. (1909) Readings in the History of Education Medieval Universities, Cambridge:<br />
Harvard University.<br />
Ratelle CF, Guay F, Vallerand RJ, et al. (2007) Autonomous, controlled, and amotivated types of academic<br />
motivation: A person-oriented analysis. Journal of Educational Psychology 99: 734-746.<br />
Sebastion V, Namsu P and Kerk FK. (2009) Is There Social Capital in a Social Network Site?: Facebook Use and<br />
College Students' Life Satisfaction, Trust, and Participation1. Journal of Computer-Mediated<br />
Communication 14: 875-901.<br />
Shirky C. (2008) Here comes everybody: the power of organizing without organizations, New York: Penguin<br />
Press.<br />
Shroff RH and Vogel DR. (2009) Assessing the factors deemed to support individual student intrinsic motivation<br />
in technology supported online and face-to-face discussions. Journal of Information Technology Education<br />
8: 59-85.<br />
Turkle<br />
S. (2011) Alone together, New York: Basic Books.<br />
Wenger<br />
E. (1998) Communities of Practice: Learning, Meaning and Identity, New York: Cambridge University<br />
Press.<br />
185
Experimental Assessment of Virtual Students<br />
Michaela Drozdová 1 , Ondřej Takács 2 and Jana Šarmanová 1<br />
1 Department of Information and Communication Technologies, University of<br />
Ostrava, Ostrava, Czech Republic<br />
2 Department of Computer Science, Faculty of Electrical Engineering and<br />
Computer Science, VSB - Technical university of Ostrava, Ostrava, Czech<br />
Republic<br />
l09430@student.osu.cz<br />
ondrej.takacs@gmail.com<br />
Abstract. In this fast developing society full of changes is <strong>learning</strong> getting more and more important. How can we<br />
make it more effective? One of the approaches uses electronic <strong>learning</strong> with its capability to adapt <strong>learning</strong><br />
process to individual students needs. Students' characteristics important for adaptive <strong>learning</strong> must be known in<br />
order to achieve this. In this paper the most frequently used method – the questionnaire is used to identify the<br />
students' characteristics. The problem is that even small number of student's properties leads to vast number of<br />
its combinations. To make the design of adaptive <strong>learning</strong> easier, we try to narrow this combinations by setting<br />
few virtual students with given values of respective properties that represent most common combinations of<br />
student properties. In this paper we describe the methods and results of virtual student assessment using<br />
quantitative analysis of filled <strong>learning</strong> styles questionnaire. We created new questionnaire that measures sensual<br />
perception, motivation and <strong>learning</strong> styles. Sensual perception describes which form of information suits students<br />
best: visual, auditive, verbal or kinesthetic. Social aspects concerns with type of company that student prefers<br />
when <strong>learning</strong>. The systematic approach describes sequence of <strong>learning</strong>, which can be either in logical sequence<br />
of steps or almost randomly, without connections, in great steps. We divide way of <strong>learning</strong> on deriving and<br />
experimenting. In accordance to sequence of <strong>learning</strong> we can divide students on detail oriented, which focus on<br />
small pieces of information and compose them to the global picture, and holistic, which focus on big pieces of<br />
abstract information from which it works through to details. The approach of <strong>learning</strong> can be divided into: deep,<br />
where student's main goal is to fully understand the curriculum, strategic, where are prioritized results and<br />
effectiveness and surface approach, where students only try to meet basic requirements. Degree of student's<br />
ability to independently control his <strong>learning</strong> is given by his self regulation of <strong>learning</strong>. This questionnaire was filled<br />
by 500 students from different fields of study. We analyzed these results using clustering, decision tree and<br />
principal component analysis. Cluster analysis tries to discover groups of mutually similar objects which differ<br />
from other groups. We set virtual students using this method. Decision tree analysis focuses on one property and<br />
tries to discover other properties that affect its values. Several interesting dependencies between properties were<br />
discovered using this method, for example students have high motivation if they do not possess surface <strong>learning</strong><br />
style.<br />
Keywords: <strong>learning</strong> styles, eLearning, data mining, virtual students<br />
1. Introduction<br />
Learning process can be influenced by a number of aspects. For our field of interest, aspects<br />
characterizing <strong>learning</strong> style, preferences, motivation and the way particular students learn using<br />
eLearning materials have been chosen. Our aim is to prepare a <strong>learning</strong> environment in eLearning<br />
domain respecting students’ differences and adapting to them accordingly.<br />
In a traditional teaching process, the educator teaches all of the students in the same way despite the<br />
fact that the educator realizes that each student is different. Each student can be talented in different<br />
subjects, has a different level of previous knowledge, possesses different <strong>learning</strong> styles, different<br />
memory, motivation for <strong>learning</strong>, different family background, different habits concerning when and<br />
how to learn, etc. Unfortunately, a teacher in the class lacks the capacity to respect and consider such<br />
preferences as well as the instant dispositions of all individuals in the class (Šarmanová, 2010).<br />
This can be different in eLearning. In this <strong>learning</strong> form the student studies through the Internet and<br />
represents an individual person to a tutor, who communicates with the student individually, gives<br />
advice and guides the student through the <strong>learning</strong> process. The tutor is thus able to adapt to the<br />
preferences of the student. Ideal assistants for such adaptation are modern information and<br />
communication technologies, vast databases, multimedia equipment, internet, etc. We asked<br />
ourselves a question, if it is possible to simulate the process of going through a course, which would<br />
suit each student with his individual properties. Our intention is to find methodology and algorithms to<br />
search for the optimum procedure. This procedure shall respect the differences among students on<br />
the basis of detected <strong>learning</strong> style and also consider the altering level of knowledge and skills of<br />
186
Michaela Drozdová et al.<br />
each individual student in the course. On the basis of students’ personal characteristics identification,<br />
they will be provided with <strong>learning</strong> material in the form which shall be most convenient to them.<br />
The model of <strong>learning</strong> being created is based on a new paradigm – the personalization of <strong>learning</strong><br />
environment. In an ideal case, this environment consists of three parts: a student part, a <strong>learning</strong><br />
materials database part and a part representing the system of adaptive algorithm.<br />
Our goal is to create adaptive eLearning environment – an environment in which the student learns<br />
through directed self-study form. Suitable instruments for this form of teaching are so called Learning<br />
Management Systems - LMS. These are systems incorporating teaching supports, features to support<br />
<strong>learning</strong> management and, finally, an information system registering students, their activities and<br />
results (Kostolányová, 2010).<br />
For an adaptive system being able to respect the diversity of users, it cannot be anonymous (Takács,<br />
2010). The collection of data about a student will be implemented in several phases. The most<br />
important part is the student self-assessment, i.e. testing before entering the course. The second type<br />
of students testing during the study course (testing will be classified as dynamic changing<br />
characteristics, crucial for adjustments of the proposed course route).<br />
In such an extensive project we first concentrated on students and their characteristics, as they are<br />
the key subjects. Especially their <strong>learning</strong> preferences which will influence the creation of the adaptive<br />
course route the most.<br />
2. Student part of adaptive system<br />
To enable the <strong>learning</strong> management system to react on different students, we have to choose,<br />
describe and suitably store the student’s qualities and other attributes, which influence the process of<br />
his/her <strong>learning</strong>.<br />
We can gain one group of qualities straight from students with the help of a suitable questionnaire and<br />
the second group of qualities we gain by long time monitoring of their study activities. The second<br />
group can serve as feedback not only during the current <strong>learning</strong>, but also for the alteration of the<br />
student’s qualities, possibly for monitoring of his/her development.<br />
It can be generally said that each person is a truly an individual with various points of view. There is<br />
an abundant literature available dealing with <strong>learning</strong> styles and also describing their classification. By<br />
analyzing and studying the already published classifications by various authors (Gregorc, 1979),<br />
(Kolb, 1984). (Mareš, 1998) we have come to the following characteristics that may be taken into<br />
account in the area of eLearning. We have divided these characteristics into several groups:<br />
The group of characteristics called the sensual perception describes the form of information most<br />
convenient for the student – visual type, auditive type, kinesthetic type and verbal type. For this group<br />
of characteristics we used the VARK questionnaire (Fleming, 2009) based on (Fleming, 1992). The<br />
acronym VARK stands for Visual, Aural, Read/write, and Kinesthetic sensory modalities that are used<br />
for <strong>learning</strong> information.<br />
Another group of characteristics called the social aspects deals with the company most convenient to<br />
the student within the <strong>learning</strong> process – whether the student prefers <strong>learning</strong> with his classmates,<br />
with a teacher or by himself. For this group of characteristics the Czech version of LSI (Learning Style<br />
Inventory) questionnaire was used (Mareš, 1993).<br />
The group of characteristics called the affective characteristics deals with student’s feelings and<br />
attitude which influence the <strong>learning</strong> process. The most important characteristic of this category is<br />
motivation. For this group a part of the Czech version of LSI questionnaire was used (Mareš, 1993).<br />
The most extensive group of characteristics is called the tactics of <strong>learning</strong>. These describe the<br />
“technology“ of the way the student learns. The method of the student’s <strong>learning</strong> is described by the<br />
order of <strong>learning</strong>, which can be done either subsequently in mutually related blocks (pole rule) or in<br />
almost random manner, without connections, with large skipping (pole latitude). To describe this<br />
characteristic a part of the ILS (Index of Learning Styles) questionnaire was used (Felder, 2009).<br />
According to the way of <strong>learning</strong> we divide the tactics to theoretical deduction and experimentation.<br />
To describe these characteristics a part of the ILS questionnaire was used (Felder, 2009). According<br />
187
Michaela Drozdová et al.<br />
to the order of <strong>learning</strong> there can be either detailistic or holistic tactics. For this characteristic a part of<br />
the TSI (Thinking Style Inventory) questionnaire was used (Sternberg, 1999).<br />
The conception of student’s <strong>learning</strong> can be divided into three levels: deep, strategic and surface.<br />
This characteristic is tested by ASSIST (Approaches and Study Skills Inventory for Students)<br />
questionnaire (Tait, 1996).<br />
The <strong>learning</strong> auto-regulation defines, to which extent the student is able to manage his <strong>learning</strong><br />
independently. This implies his need of external supervision of the processes of <strong>learning</strong> where, on<br />
one side, there are those who appreciate punctual instructions and on the other side those who prefer<br />
to manage the <strong>learning</strong> processes themselves. For this characteristic a part of the LSI questionnaire<br />
was used (Mareš, 1993).<br />
2.1 Testing and data analysis of individual <strong>learning</strong> styles<br />
To measure aforesaid characteristics, it is best to use questionnaires. In the pilot phase we used the<br />
above mentioned questionnaires, but their combination was not applicable due to their length. This<br />
led to inaccurate filling of the questionnaires, shown by the results of their analysis (Takács, 2009).<br />
These reasons led to compiling a new shortened Learning Profile Questionnaire - LPQ (Novotný,<br />
2010). The usual duration of filling the LPQ is only ten minutes while measuring all properties needed.<br />
This questionnaire is inspired by following questionnaires: LSI, ILS, TSI and ASSIST. Only three to<br />
five questions measures one property of the student. This questionnaire was not verified on students<br />
yet.<br />
Most questions in the LPQ questionnaire are made of statements together with scales of agreement<br />
to this statement. These scales are usually from 1 (I don’t agree) to 5 (I agree). Some questions have<br />
four variants, one for each type of perception.<br />
This questionnaire was converted into electronic form. Before filling the questionnaire, its purpose was<br />
introduced to the students. A group of university students from different fields of study were asked to<br />
fill this questionnaire, as well as a group of high school students. A total of 508 students filled this<br />
questionnaire, 190 from those being grammar school students, 196 pedagogical university students,<br />
68 high school students and 62 informatics university students. The duration of filling the<br />
questionnaires varied between two minutes and half an hour. To exclude suspiciously fast filling<br />
students we sat aside 45 questionnaires filled in less than five minutes.<br />
From filled questionnaires we saved data about students’ answers and questionnaire results that were<br />
computed according to a scoring key of this questionnaire. Values of all properties ranged from 0 to<br />
100, zero meaning that the student does not possess this characteristic and one-hundred meaning<br />
that he does.<br />
For the data analysis association rules method, which looks for dependencies between attributes and<br />
conveys them in form of rules, was used. Each rule consists of a condition and a conclusion and is<br />
assigned with reliability and support. Reliability provides information about how many percent of<br />
records found with this condition being true meet also the conclusion. Support provides information<br />
about for how many percent of records the condition is true.<br />
Principal component analysis was used to identify possible dependent attributes measuring the same<br />
or very similar student’s characteristics. This method searches for principal components – new<br />
attributes which values are determined by linear combination of former attributes. We can distinguish<br />
composition of each component by the size of its coefficients. Every principal component is also<br />
described by standard deviation that indicates significance of the component.<br />
Decision tree analysis was also performed. This analysis finds numbers of interesting rules for one<br />
goal attribute. Rules are then represented in a form of a tree. Leaves gives value of the goal attribute,<br />
nodes are conditions of rules and branches are values of conditions. Finally, a cluster analysis was<br />
performed, which tries to find groups of similar objects that are different from other group‘s objects in<br />
data. For the computation of the distance was used Euclidian metric and then two grouping methods<br />
were used: single link and Ward-Wishard Ward’s minimum variance method (Ward, 1963) aims at<br />
finding compact, spherical clusters. The single linkage method adopts a ‘friends of friends’ clustering<br />
strategy.<br />
188
3. Results of <strong>learning</strong> style analysis<br />
Michaela Drozdová et al.<br />
In this part, results obtained during the analysis are presented.<br />
3.1 Association rules<br />
Association rules found many dependencies between properties. We were looking for rules with<br />
minimal support of 10% and minimal reliability of 70% and found total of 147 rules. The majority of<br />
these rules have very low level of depth <strong>learning</strong> in conclusion. Conclusions also contained visual<br />
type, strategic <strong>learning</strong> and motivation. Some of these rules are shown in Table 1. Of most interest<br />
are the rules with high support. For example almost half of non-visual type of students mostly do not<br />
possess depth <strong>learning</strong> style (see rule number 1, where support 43% means that this rule is applied to<br />
43% of students and reliability 71% means that 71% of students with non visual <strong>learning</strong> also do not<br />
possess depth <strong>learning</strong>). Unfortunately 70% of all students do not possess depth <strong>learning</strong> style, so<br />
many of these rules are influenced by this fact. Because of this we cannot make any valid conclusions<br />
based on this analysis. But we can use its results to determine goal attributes for the decision tree<br />
analysis: depth and strategic <strong>learning</strong>, visual perception and motivation.<br />
Table 1: Selected rules with 70% minimal reliability and 10% minimal support, ordered by support<br />
Num Condition Conclusion Support Reliability<br />
1 visual = very low depth = very low 43% 71%<br />
2 strategic = very low depth = very low 38% 76%<br />
3 social (alone - group) = middle depth = very low 38% 72%<br />
4 verbal = low depth = very low 31% 73%<br />
5 theoretical – experimental = high depth = very low 31% 76%<br />
… … … … …<br />
32 motivation = high, theoretical – experimental = high depth = very low 17% 73%<br />
33 social (alone - group) = middle, verbal = low depth = very low 17% 75%<br />
34 auditive = middle visual = very low 16% 80%<br />
… … … … …<br />
depth = very low, systematic (sequential - random) =<br />
69<br />
very low<br />
strategic = very<br />
low<br />
13% 72%<br />
visual = very low, motivation = high, strategic = very<br />
70<br />
low<br />
depth = very low 13% 73%<br />
… … … … …<br />
depth = very low, visual = very low, field =<br />
87<br />
University, pedagogical<br />
motivation = high 12% 70%<br />
88 strategic = very low, visual = middle depth = very low 12% 79%<br />
… … … … …<br />
depth = very low, surface = very low, autoregulation =<br />
107 visual = very low<br />
low<br />
11% 71%<br />
… … … … …<br />
147 autoregulation = low, social (alone - group) = low depth = very low 10% 87%<br />
3.2 Decision trees analysis<br />
This method was used for four goal attributes based on the association rule analysis results: depth<br />
and strategic <strong>learning</strong>, visual perception and motivation. Further on we describe results of these<br />
analyses.<br />
By analyzing depth <strong>learning</strong> (see figure 1) we found out that group <strong>learning</strong> (left branch of node<br />
“social: alone/group”), holistic (left branch of node “detailistic/holistic”) students mostly have depth<br />
<strong>learning</strong> style (yellow color in leaf labeled “high”). Other results derived from this tree cannot<br />
unfortunately be used because 70% of students in this data sample do not have depth <strong>learning</strong> style.<br />
189
Michaela Drozdová et al.<br />
Figure 1: Decision tree for depth <strong>learning</strong><br />
The analysis of strategic <strong>learning</strong> style revealed (see figure 2) that most experimenting students, not<br />
<strong>learning</strong> alone and randomly do not possess the strategic <strong>learning</strong> style. Other rules provided by this<br />
tree cannot unfortunately be used because of their low reliability.<br />
Figure 2: Decision tree for strategic <strong>learning</strong> style<br />
190
Michaela Drozdová et al.<br />
By analyzing motivation (see figure 3) we discovered that students have high motivation if they:<br />
Do not possess surface <strong>learning</strong> style,<br />
Possess surface <strong>learning</strong> but do not have high auto-regulation,<br />
Possess neither surface <strong>learning</strong> style, nor high auto-regulation nor strategic <strong>learning</strong> style.<br />
Figure 3: Decision tree for motivation<br />
Through the analysis of visual perception (see figure 4) we found out that students don’t possess<br />
visual perception if they:<br />
Are auditive types,<br />
Are not auditive and are kinesthetic types,<br />
Are neither auditive nor kinesthetic, but are verbal types.<br />
3.3 Cluster analysis<br />
Agglomerative clustering with single-link method didn’t find any significant clusters. So, the Ward-<br />
Wishard method was used and the results are presented here. The final dendogram (see figure 5)<br />
was cut on the height 75 which created 5 clusters within the data. In bottom part of this figure are<br />
more than 500 vertical lines, where each line represents one student. Vertical lines connected by<br />
horizontal lines forms clusters. The longer the vertical line is, the more dissimilar are the objects in the<br />
cluster. The thick horizontal line on the height 75 represents cut and chooses five clusters.<br />
191
Figure 4: Decision tree for visual perception<br />
Michaela Drozdová et al.<br />
Figure 5: Dendrogram of agglomerative clustering analysis<br />
Figure 6 presents detail description of the five clusters created. Each cluster is represented by one<br />
color in the boxplot chart. The most interesting results are at first four properties. Red cluster is typical<br />
by average level at all properties except kinesthetic, auditive and verbal property. Yellow cluster is<br />
described by very high values of auditive property. Green cluster is characterized by low value of<br />
kinesthetic property, high value of auditive and verbal properties. Blue cluster is described by higher<br />
value of kinesthetic, low value of auditive and above average value of verbal property. Black cluster is<br />
typical by its high value of visual and auditive property and low value of kinesthetic, social and verbal<br />
property. The main finding is that the clusters of students depend mainly on type of perception of<br />
students.<br />
192
Figure 6: Box plots of five clusters<br />
3.4 Principal component analysis<br />
Michaela Drozdová et al.<br />
Principal component analysis found 13 components. The last component has much lower standard<br />
deviation compared to the other components (see table 2), so this component can be removed from<br />
the analysis. Most of the principal components have high coefficient in the covariance matrix (see<br />
table 3) at only one property, so this property is independent. Significant components with more than<br />
one high coefficient have high values mostly at sensual properties. This means that sensual<br />
properties depend on each other, but that was already known. All other properties are almost entirely<br />
independent.<br />
Table 2: Standard deviation table of principal components<br />
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 PC12 PC13<br />
3,46 2,45 2,26 2,09 1,51 1,25 1,08 0,75 0,34 0,31 0,29 0,26 0,00<br />
Table 3: Covariance matrix of principal components<br />
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 PC12 PC13<br />
visual 0,0 0,1 -0,1 0,0 -0,8 0,0 0,1 0,0 0,0 0,0 0,0 0,0 0,5<br />
kinesthetic 0,0 0,7 0,2 -0,1 0,3 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,5<br />
auditive 0,0 -0,6 0,6 -0,1 0,1 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,5<br />
verbal 0,0 -0,2 -0,7 0,3 0,3 0,0 -0,1 0,0 0,0 0,0 0,0 0,0 0,5<br />
social -1,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0<br />
motivation 0,0 -0,1 -0,3 -0,9 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0<br />
order 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 -0,7 -0,7 0,2 0,0 0,0<br />
theor/exper 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,1 -0,4 -0,9 0,2 0,0<br />
detail/holistic 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 -0,7 0,6 -0,2 0,4 0,0<br />
surface 0,0 0,0 0,0 0,0 -0,1 1,0 -0,1 0,1 0,0 0,0 0,0 0,1 0,0<br />
strategic 0,0 0,0 0,0 0,0 0,1 0,1 0,9 -0,3 0,0 0,0 0,0 0,0 0,0<br />
deep 0,0 0,0 0,0 0,0 0,1 -0,1 0,3 0,9 0,0 0,0 0,0 0,0 0,0<br />
auto-regulation 0,0 0,0 0,0 0,0 0,0 0,1 0,0 0,0 -0,3 0,2 -0,3 -0,9 0,0<br />
4. Conclusion<br />
The newly created LPQ questionnaire has many advantages compared to the group of questionnaires<br />
used in our previous research. Its main benefit is lower filling time that led to more honest filling.<br />
Analysis of filled questionnaires found only minor dependencies between chosen <strong>learning</strong> styles, so<br />
there is no need to change the set of measured student characteristic. One potential problem is that<br />
193
Michaela Drozdová et al.<br />
very few students possess the depth <strong>learning</strong> style. This can be explained by wrongly designed<br />
questions in the questionnaire or by the composition of the set of the tested students.<br />
Results of cluster analysis which discovered five dominant clusters divided according to type of<br />
perception tell us that this property is very important for definition of virtual students: set of properties<br />
that describes group of students with similar properties. Finding of these virtual students will be the<br />
goal of the future work. The definition of virtual students can be very useful in eLearning. It is much<br />
easier to design the course for only five types of students instead of trying to design it for every<br />
individual student and all possible combinations of his properties.<br />
Analysis of principal components revealed that evaluated properties except type of perception are<br />
independent. For practical use it is more appropriate to distinguish the four types of perception,<br />
because there is clear way of using this information in eLearning.<br />
LPQ questionnaire proved to be suitable for using directly in adaptive <strong>learning</strong> process. It will be<br />
presented to untested students at the beginning of the lecture. We plan to do more analysis of results<br />
of this questionnaire on students that will participate in adaptive <strong>learning</strong>.<br />
Acknowledgments<br />
This publication is supported by the ESF project OP VK CZ.1.07/2.3.00/09.0019 title Adaptive<br />
individualization of study through eLearning.<br />
References<br />
Felder, M. R., Soloman, A. B. (2009) Index of Learning Styles [Online], Available: http://www.ncsu.edu/felderpublic/ILSpage.html<br />
[2009-05-10].<br />
Fleming, N. D. and Mills, C. (1992) “Not Another Inventory, Rather a Catalyst for Reflection“, To Improve the<br />
Academy, vol. 11, pp. 137-155.<br />
Fleming N. D. (2009) VARK -- A Guide to Learning Styles [Online], Available: http://www.varklearn.com/english/index.asp<br />
[2009-05-10].<br />
Gregorc A. (1979) “Learning/Teaching Styles: Their Nature and Effects”, Christian Education Journal, vol. 4, pp.<br />
62<br />
Kolb D. A. (1984) Experiental <strong>learning</strong>: Experience as the source of <strong>learning</strong> and development, Engelwood Cliffs,<br />
NJ, Prentice Hall<br />
Kostolányová, K., Šarmanová, J., Takács, O. (2010) “Learning Characteristics”. Proceedings of the 9th European<br />
Conference on eLearning, Porto, pp. 107—107.<br />
Mareš, J., Skalská, H. (1993) “Česká verze dotazníku LSI, zjišťující styly učení u žáků základních<br />
škol”, Pedagogický výzkum a transformace české školy, pp. 54.<br />
Mareš, J. (1998) Styly učení žáků a studentů, Praha, Portal<br />
Novotný, J. S. (2010) “Individualization of teaching through eLearning: Development of Students? Learning<br />
Profile Questionnaire”, Theoretical and Practical Aspects of Distance Learning, Cieszyn.<br />
Sternberg, J. R. (1999) Thinking styles, Cambridge: University Press.<br />
Šarmanová, J., Kostolányová, K. and Takács, O. (2010) “Intelligent Tutoring in ELearning System“, 9th European<br />
Conference on eLearning, Porto, pp. 794-796.<br />
Tait, H. & Entwistle, N. (1996) “Identifying students at risk through ineffective study strategies”, Higher<br />
Education, vol. 31, pp. 99-118.<br />
Takács, O., Šarmanová, J. and Kostolányová, K. (2009) “Results of analysis of <strong>learning</strong> styles”, Proceedings of<br />
ICTE2009, Ostrava, pp. 205-210.<br />
Takács, O., Kostolányová, K., Šarmanová, J. (2010) “The use of adaptive individualized eLearning at<br />
teaching”, IMSCI 2010, Florida, pp. 147—152.<br />
Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American<br />
Statistical Association, 58, 236.<br />
194
Priming for Modules: A Case Study Evaluation of ‘Pre-<br />
Workshop’ Online Resources for an Executive MBA Course<br />
Glenn Duckworth<br />
Bristol Business School/e-Learning Development Unit, The University of the<br />
West of England, Bristol, UK<br />
glenn.duckworth@uwe.ac.uk<br />
Abstract: This case study looks at a series of 'pre-workshop' resources and tasks developed for the Executive<br />
MBA (EMBA) at the University of the West of England (UWE). Students on the EMBA course at UWE are<br />
generally in full time employment and therefore do not have the opportunity to follow a traditional semester - or<br />
year - long module structure. For each module on the EMBA programme, the taught classroom based component<br />
runs as a five day intensive block. It is therefore very important for the students to get to grips with concepts and<br />
terminology for each module as quickly as possible so that the time spent in class can be used for high level<br />
discussions and debates as well as the more traditional lecture formats. This is common of a variety of executive<br />
education and CPD courses across the sector. A set of online resources were developed for a range of modules<br />
on the EMBA course and delivered online. Their aim was to ‘prime’ the students for the modules in advance of<br />
the classroom based components. As well as providing readings, the resources provided a range of online<br />
interactive exercises to introduce the topics covered on the module and to help the students assess their level of<br />
<strong>learning</strong>. Student evaluations were conducted on three modules to assess the usefulness of these pre-workshop<br />
resources. These took the form of surveys with likert-type questions and more open ended questions, as well as<br />
informal group discussions. Views of the academic staff teaching on the modules were also sought. For all three<br />
modules evaluated, the pre-workshop resources were seen as being a very valuable resource. There were fears<br />
that students would not complete the exercises and readings before the classroom based sessions began, but<br />
the majority of them did so. They reported that the exercises helped them to structure their <strong>learning</strong> and that they<br />
felt more confident and prepared for the first day of classroom based teaching. Lecturers also felt that the<br />
resources and exercises were useful – they felt that they were able to engage with students at a higher level from<br />
day one of the classroom based sessions. This case study will look at the evaluations in more detail and go on to<br />
assess whether a ‘pre-workshop’ component may be of use on a variety of other executive and CPD intensive<br />
courses.<br />
Keywords: priming, pre-workshop activities, online <strong>learning</strong>, interactive exercises<br />
1. Introduction<br />
This paper is concerned with intensive courses - also known as block, compressed or time-shortened<br />
courses – and in particular a programme developed for the part-time Executive MBA (EMBA) at the<br />
University of the West of England (UK). Sets of online <strong>learning</strong> resources were developed to<br />
complement the more traditional classroom based <strong>learning</strong> and teaching. They were provided in<br />
advance of the classroom based sessions and became known as ‘pre-workshop activities’. They were<br />
developed with the intention of ‘priming’ students to the subject matter before beginning the intensive<br />
block of classroom based teaching. Post-workshop activities were also provided – generally with a<br />
focus on the assignment submission.<br />
The paper will take the form of a case study examining the views of students and lecturers as to how<br />
useful these online priming activities were seen as being. It will draw together evaluations of the preworkshop<br />
activities used on three modules on the EMBA programme. It will also examine the views of<br />
module leaders relating to the usefulness of these activities.<br />
There has been debate about the efficacy of intensive courses for many years, although research on<br />
the topic has been somewhat patchy. As Kasworm (2001) put it, “While there is recent significant<br />
growth of accelerated degree programs, there is little empirical research regarding the quality and<br />
impact of accelerated degrees on adult <strong>learning</strong>” (cited in Davies 2006). According to Davies (2006)<br />
“a considerable amount of the literature … appears to exist in academic areas where skill acquisition<br />
is paramount, rather than discursive, conceptual <strong>learning</strong>. This point may be critical in assessing the<br />
value of intensive teaching in various subjects.”<br />
Where research into this topic does exist, the findings have been somewhat mixed. Many studies<br />
have found that compressed courses compare favourably with traditional semester - or year - long<br />
courses. Daniel (2000) in a review of relevant literature suggested that intensive courses offer as<br />
effective a <strong>learning</strong> experience as do traditional courses – and in fact they may sometimes lead to<br />
superior outcomes (although she did go on to question methodologies used in many such studies).<br />
195
Glenn Duckworth<br />
Other studies have questioned the usefulness of such accelerated courses with regards to levels of<br />
higher order thinking and analysis skills, finding that students performed worse in assessments<br />
requiring these higher order skills (e.g. Petrowsky 1996).<br />
Although much research points to positive outcomes, there may be a number of threats to the<br />
effectiveness of intensive courses. A common belief is that “some material and problems require an<br />
extended period of time to ‘digest’ or ‘simmer.’ There is evidence in the psychological literature that<br />
setting a problem aside for a while (a process labeled “incubation”) does help the learner and problem<br />
solver.” (Tatum 2010). Tatum goes on to tentatively state that “It would appear that for some courses<br />
in some disciplines, an accelerated approach could be a disadvantage for the students because they<br />
are not given sufficient time to ruminate over the course content or the homework problems.”<br />
Another potential threat relates to stress and time pressures. A study by Petrowsky (1996) found that<br />
students taking a two week long intensive course found it particularly stressful.<br />
Davies (2006) when discussing studies by Conrad (1996) and Wlodkowski (2003) stated that certain<br />
key attributes were necessary to achieve successful <strong>learning</strong> outcomes in an intensive course and<br />
these determined whether the experience was ‘painful and tedious’ or a good <strong>learning</strong> experience.<br />
These characteristics include high levels of classroom interaction, student input, class discussion and<br />
active <strong>learning</strong>. Scott (2003) investigated the student experience with a comparison of traditionally<br />
scheduled and intensive formats using a range of qualitative methods. She also found that intensive<br />
courses could provide useful benefits as long as certain key characteristics were present. In terms of<br />
teaching methods, she found that active <strong>learning</strong>, high levels of classroom interaction/discussion and<br />
experiential/applied <strong>learning</strong> were valuable characteristics.<br />
The pre-workshop activities which are discussed in this case study may go some way to address<br />
some of the concerns. The provision of a longer time period in which students may engage in the<br />
subject area may aid incubation. It was also anticipated that the resources may reduce stress and<br />
increase confidence due to some exposure to the topics. And the provision of structured introductory<br />
tasks and readings would help students to get to grips with some of the key concepts and ideas<br />
before entering the classroom; thus freeing up more time for higher level discussions, debates and<br />
active <strong>learning</strong>.<br />
While the EMBA has always put an emphasis on such high level teaching methods, it was anticipated<br />
that the pre-workshop components would enhance teaching and <strong>learning</strong> by priming students to the<br />
subject matter; thus allowing students and lecturers to ‘hit the ground running’ and allowing such high<br />
level teaching and <strong>learning</strong> to start from an early stage of the intensive block of classroom based<br />
sessions.<br />
2. The students<br />
All of the students involved were taking the Executive MBA course at the University of the West of<br />
England between January 2010 and March 2011. All had at least three years work experience with<br />
management responsibilities. They were all mature students and most of them were in full time<br />
employment.<br />
Evaluations were conducted with three separate groups of students. For each of these groups<br />
approximately twenty students were enrolled on the module.<br />
3. Pre-workshop activities<br />
Pre-workshop activities and resources were developed for a range of modules on the EMBA<br />
programme. The modules discussed in this case study are called ‘Management in a Complex World’<br />
(mainly Economics), ‘Managing Decisions’ (Finance/Accounting) and ‘Leadership and Change’.<br />
The resources were generally delivered via the Blackboard virtual <strong>learning</strong> environment (VLE). They<br />
were made available to students one month before their first scheduled classroom based session.<br />
A short video was included for each module in which the module leader would introduce the subject<br />
and the pre-workshop component. The activities themselves comprised a varied set of tasks which<br />
accompanied key readings and aimed to structure the introductory reading, providing an insight into<br />
196
Glenn Duckworth<br />
some of the key concepts involved with the subject matter. There follow a few examples of tasks used<br />
for pre-workshop activities:<br />
Glossary Exercises: A version of this was deployed on all three of the modules with content tailored to<br />
each module. It is an interactive exercise, developed in-house and delivered online. The first part<br />
involves matching key concepts to definitions. The second part requires the students to recall the<br />
definition and attempt to enhance the short definition provided in part one. As well as providing<br />
computer based marking for part one, it encourages self assessment in part two. It also aims to<br />
encourage students to read key sections of relevant texts.<br />
Finance Resources: This was used on the ‘Managing Decisions’ finance module and is made up of<br />
online mini lectures and exercises designed to teach the basics of financial reporting. Students are<br />
encouraged to work through the exercises building up three key financial reports for a fictional<br />
business (statement of cash flows, statement of comprehensive income and statement of financial<br />
position). The original version of the resources comprised mostly interactive exercises with directions<br />
to read key chapters in text books. The resources were modified for the next group of students taking<br />
the module. This version included online mini lectures and more background information which<br />
introduced the key concepts, as well as some tweaking of the exercises based on student feedback.<br />
Online communication, blogs and wikis: Students were encouraged to communicate with each other<br />
online using tools such as discussion boards. A social networking web site was also set up for them<br />
to use. Students were encouraged to post profiles to help familiarise themselves with each other and<br />
to aid social cohesion within the group. Blogs and wikis were deployed on certain modules. A typical<br />
example might be where students were asked to choose a key topic and write a short piece on that<br />
subject.<br />
4. Evaluations<br />
Evaluations were centered around student views on the usefulness of the pre-workshop activities and<br />
resources.<br />
For the three modules identified students were asked to fill in a questionnaire for each module,<br />
comprising likert-type questions and more open ended questions. Informal group discussions were<br />
also conducted to elicit their thoughts.<br />
The student evaluations were concerned with the pre-workshop resources and activities in general,<br />
but also went into some detail about specific components – most notably the glossary exercise and<br />
the finance resources/exercises.<br />
Views of the academic staff teaching on the modules were also sought. Module leaders were<br />
interviewed to assess their views on the usefulness of the pre-workshop activities.<br />
5. Findings<br />
This section will examine the findings from the questionnaires, group discussions and interviews with<br />
lecturers. It will begin by looking at the pre-workshop components as a whole and will then go on to<br />
look at two aspects in more detail – the glossary exercises and the finance resources/exercises. For<br />
each sub-section a set of quotes from students will be included to provide some insight into student<br />
views. Sections from the interview transcripts of the lecturers will also be included.<br />
6. The pre-workshop activities in general<br />
The majority of students on all three modules did engage with the online resources and tasks. A few<br />
students did comment on the potential for these tasks to cause ‘overloading’ but on balance they were<br />
seen as useful preparation for the modules.<br />
A recurring theme was the notion that the activities forced them to prepare. Students on the EMBA<br />
had always been encouraged to read key texts before beginning the module, but with the introduction<br />
of these structured online tasks students were able to assess their understanding of the reading and<br />
learn more actively. From informal group discussions it became clear that they thought the tasks had<br />
helped them to structure their reading and attempt to understand the issues and concepts rather than<br />
197
Glenn Duckworth<br />
merely skim read the texts. The notion of forcing them to prepare was seen as being an extremely<br />
positive factor for these groups of students.<br />
They also stated that the activities helped to ease them into the academic subject areas and helped to<br />
build confidence and reduce stress. This was particularly apparent with the Management in a<br />
Complex World and Managing Decisions modules. Management in a Complex World was the first<br />
module that the cohort enrolled upon in their MBA studies, and a large percentage of them had been<br />
away from higher education for many years. They indicated that the pre-workshop activities helped<br />
them to re-acclimatise to the world of academia and helped to build confidence. Managing Decisions,<br />
a module dealing with finance and accounting has always been treated with trepidation by students<br />
who are not confident with numbers and accounting. The students reported that the resources and<br />
exercises helped to build confidence and reduce their worries about embarking on the module.<br />
For one of the early modules there were some technical/administrative issues relating to enrolment<br />
which led to some students not being able to access Blackboard in advance of the classroom based<br />
sessions. This was overcome by e-mailing the tasks to these students. Subsequently, the majority of<br />
them did engage in the activities. The technical/administrative issues have now been resolved and all<br />
students have access to modules a month in advance.<br />
A sample of student comments:<br />
“Really forces you to do the pre-reading for the module.”<br />
“It forces you to prepare. Therefore the module achieves more.”<br />
“Overall it was good as it forced me to read.”<br />
“Enabled me to get into the zone, focusing on key <strong>learning</strong>.”<br />
“Very useful - especially for getting you to do pre-reading. Needs to be in Blackboard<br />
plenty of time before module date.”<br />
“Very handy as it gives/offers pre-module insight.”<br />
“I completed the tasks sent by e-mail and they were very helpful.”<br />
The lecturers also found the pre-workshop activities to be very beneficial:<br />
“I found them incredibly useful… we’ve had three cohorts going through using this<br />
system and each time it seems to work extremely well. The students thoroughly enjoy<br />
being exposed to the subject before they get here. In a glossary sense they learn the<br />
terminology which normally takes me weeks to get across to them. And it’s whetted their<br />
appetite to follow through some of these things rather more because there’s a<br />
combination of <strong>learning</strong> the language of Economics and also being able to start thinking<br />
about the theories and the concepts that we explore early on. And they’re doing that<br />
even before they arrive on the campus. I’m finding in the early seminars and discussion<br />
groups a much greater depth of knowledge than I’ve previously seen in students like that.<br />
So I have found it incredibly useful.” – Module leader for Management in a Complex<br />
World.<br />
The lecturers believed that the activities enabled students to prepare for the classroom based<br />
sessions and therefore more could be achieved in class:<br />
“…I was able to teach from a point of view where I knew they had already been<br />
introduced to basic terms. Very often with this sort of subject you have some students<br />
that actually say ‘I know nothing’ about this and almost sit there with a dazed expression<br />
for quite some time until they … get eased into it. Whereas I think with this they had<br />
already been eased into it.” – Module leader for Managing Decisions.<br />
“They usually tell you something towards the end of the module, not at the beginning. At<br />
the beginning I think they find it an additional burden… at the end they keep coming up<br />
and saying it was so useful being able to understand what you’re talking about without<br />
you having to keep explaining the terminology, the theories or the concepts. So it’s a<br />
really good foundation for me to build on more quickly than I normally would with a<br />
module.” - Module leader for Management in a Complex World.<br />
“I was quite taken aback by the fact that they knew more about the topics I was<br />
discussing with them than previous cohorts… They already had some information at a<br />
198
Glenn Duckworth<br />
very basic level which is crucial for making that first quite big step into the theory… With<br />
Economics being the biggest part (of the module) this helps them enormously. It’s the<br />
first thing they do, it’s sixty percent of the module… it’s a much easier entry than it would<br />
have been without it.” - Module leader for Management in a Complex World.<br />
The lecturers also believed that the exposure to the subject via the pre-workshop activities helped<br />
with student confidence and may have relieved some of the stress and anxiety associated with<br />
beginning the intensive courses:<br />
“I think confidence is crucial. Taking on the MBA is a big challenge for anybody. They are<br />
working as well… so they’ve got continual pressure from that direction. Anything which<br />
eases the stress or burden will build that confidence. And I’ve found a remarkable<br />
difference between the way they used to be before we started this process and the way<br />
they are now. It’s a really valuable initial first step. And for them I think they’ve got a …<br />
safety curtain. They know that if they understand these things then they are well placed<br />
to follow the course in for them, what is a very demanding post-graduate area.” - Module<br />
leader for Management in a Complex World.<br />
“It maybe alleviated some of their concerns about an accounting module. Before, I had to<br />
try and do that in the classroom… so basic accounting terms were introduced to them in<br />
a way that they probably thought ‘this isn’t that complicated really’.“ – Module Leader for<br />
Managing Decisions.<br />
7. The glossary exercise<br />
The glossary exercise was seen by students and staff as being a very useful resource. Students were asked<br />
to indicate whether they found the task to be useful in their <strong>learning</strong>. With the phrase “I thought the task was<br />
useful”, 95% of respondents indicated that they agreed or strongly agreed (detailed figures are included in<br />
table 1).<br />
The majority of the students on the three modules found the tasks interesting and engaging and most of them<br />
also indicated that the glossary exercise helped them to grasp key concepts.<br />
Table 1: Student views on the glossary exercise (all figures rounded to 1 decimal place)<br />
Phrase Strongly Disagree Neutral Agree Strongly<br />
disagree<br />
agree<br />
“The task was useful” 0% 5.3% 0% 44.7% 50%<br />
“Completing the task helped me grasp key concepts” 0% 3.4% 17.2% 41.3% 37.9%<br />
“The task was interesting and engaging” 2.3% 7.1% 11.9% 42.9% 35.7%<br />
They stated that the glossary exercise gave them a method to assess their understanding of the<br />
reading materials and “forced us to do the reading” – which was seen as a good thing by all who took<br />
part in the exercise.<br />
A sample of student comments:<br />
“Very brilliant introduction to the module and sets the tone for what the module entails.”<br />
“Helps to reflect on one's understanding.”<br />
“Some of the descriptions were so similar that you had to understand not just to have<br />
speed read.”<br />
“It had drag and drop and write your own – very challenging.”<br />
“Helped embed terminology.”<br />
“Very useful as a warm up.”<br />
“Good and welcome initiation to any module. Sets the tone and gives a brief insight into<br />
what the module will entail.”<br />
Students were asked whether they would like the glossary exercise to be incorporated into other preworkshop<br />
components of future modules (with content tailored for that module). Every respondent<br />
said that they would like to use it again, and quite surprisingly, more than half of them (60%) said that<br />
they would like it to be incorporated into every module.<br />
199
8. Finance resources<br />
Glenn Duckworth<br />
The finance resources have been deployed on two iterations of the Managing Decisions module. For<br />
the first iteration an early version of the interactive exercises was used.<br />
The students found the exercises to be generally useful, but did highlight some issues. Many of them<br />
found the exercises to be very difficult and demanding. They suggested some alterations to the<br />
exercises.<br />
“Great resource - thanks. A little tweaking to improve but key ingredients in place.”<br />
“Good and useful.”<br />
“Thanks - a great resource.”<br />
“ Time consuming. Some useful prompts would be good.”<br />
“Got stuck and couldn't move on to next stage. Would be helpful to get answers/hints/tips<br />
after certain number of attempts.”<br />
“Needed additional hints/help as unable to finish and didn't know where I was going<br />
wrong.”<br />
“Get answers, or hints after a certain number of attempts in order to move on to next<br />
step.”<br />
Having discussed the exercises with the students, the module leader also believed that the exercises<br />
were too difficult as first exposure to the subject matter:<br />
“The original group is where we only did the exercises without any introduction and I<br />
think those students really struggled with that. The feedback was that it took a long time<br />
for them to do and they got to a point where they got stuck and couldn’t go forward.”<br />
Based on feedback, the exercises were amended for the next iteration of the module. Extra prompts<br />
were added and hints were incorporated. These appeared when the student provided an incorrect<br />
answer on multiple occasions. Extra resources were provided in the form of online lectures and extra<br />
background information which covered the main points.<br />
The second version of the resources received very positive feedback. They were seen as being very<br />
useful, interesting and engaging, and provided a useful introduction to the module.<br />
Recurring themes from open ended questions were that students liked the fact that the lectures and<br />
exercises were provided online and would like more resources like this.<br />
All respondents indicated that they felt better prepared for the classroom based sessions and that<br />
they found the resources useful.<br />
The quotes below relate to aspects that they particularly liked about the resources:<br />
“Work through of statements was a particularly useful and applicable exercise.”<br />
“Relevance to programme.”<br />
“That they were online and I could access any time.”<br />
“Well structured, good framework.”<br />
“Opportunity to retry.”<br />
“The ability to stop & start.”<br />
“Provided much needed ‘face’ contact.”<br />
“Very useful. Excellent context setter and good insight into what to expect.”<br />
“Make more of them!”<br />
“Generally a very good use of Blackboard.”<br />
The module leader also believed that the second iteration of the finance resources/exercises worked<br />
well and that their incorporation into the pre-workshop component was very useful:<br />
200
Glenn Duckworth<br />
“… I think the second set of exercises that we did where we actually introduced basic<br />
terms - I think that was a lot better for the students. I think the way that the exercises<br />
introduced the module to the students … was to say this is a module which can be very<br />
nerve wracking…. They may not be very au fait with numbers and things like that. And<br />
rather than reading a text book and not really understanding things, I think the<br />
introduction of the basic concepts was very good for them and that really helped them …<br />
(they) got the general basic understanding, were able to do the exercises and this really<br />
started the group off from a very different position. And perhaps that eased them into<br />
reading the textbook.”<br />
9. Other aspects of the pre-workshop components<br />
As well as the glossary exercise and the finance resources, a range of resources and activities were<br />
incorporated.<br />
The introductory videos were generally seen as being useful, as long as they provided useful<br />
information and that information was provided in written form as well.<br />
“Very useful.”<br />
“Some info was only on video. Would have been useful if given written version also to<br />
prevent having to re-run video to pick up elements later.”<br />
“Could use brief repetition of required tasks at close of video.”<br />
“It made it more personable.”<br />
“Could replay to understand.”<br />
“Good. Not too long. Just about right in terms of time.”<br />
Online communication was encouraged through various means. The use of the social<br />
networking/profiles site in advance of the first module was generally seen as useful for group<br />
cohesion. Some students did indicate that it was somewhat time consuming to set up but most<br />
indicated that they found it to be a useful exercise and that they enjoyed <strong>learning</strong> about their fellow<br />
MBA peers from the profiles.<br />
“Great idea.”<br />
“Helped each other to remember names”<br />
“Took a good couple of hours to set up.”<br />
“Useful to find out about other students - although noticed that not all posted info.”<br />
The use of other communication tools such as discussion boards was somewhat varied. For example,<br />
on the Managing Decisions finance module, the first group of students made use of discussion boards<br />
to discuss issues they were having with completion of the exercises. The second group, however, did<br />
not post any messages at all. When asked about the lack of activity on the discussion board, some<br />
students stated that they did not feel the need to use it (“No need - no questions to ask”) or<br />
commented on the lack of posts from others in the cohort (“No reason to - nobody else was using it”).<br />
Blogs and wikis were used with some success. Most of the students engaged in these activities and<br />
they did produce some online discussion.<br />
10. Conclusions<br />
The process of priming via online pre-workshop activities was seen as being valuable by both<br />
students and lecturers.<br />
There was some concern that students would not engage with the online activities due to time and<br />
work pressures. But this has not been the case, and the majority of students did indeed make use of<br />
the resources and activities before the classroom teaching began. Many of the exercises helped<br />
students to structure their reading and forced them to do the appropriate reading rather than merely<br />
skim read the suggested texts. They also provided a method for students to evaluate their<br />
understanding of the key concepts before even entering the classroom.<br />
It seems that the activities did help students and staff to ‘hit the ground running’ and they could begin<br />
engaging in high level discussion of concepts from day one of the classroom based sessions. This<br />
201
Glenn Duckworth<br />
was seen as being very important on intensive courses. Tutors also perceived higher levels of<br />
knowledge and confidence and less stress than previous cohorts of students who had not been<br />
provided with the same resources and activities.<br />
Whether this type of initiative would be useful to other student groups is up for debate. As Daniel<br />
(2000) points out, students who opt to take intensive courses tend to be older and are more motivated<br />
than traditional students. The composition of cohorts on the EMBA at the University of the West of<br />
England does seem to fit this description. It may be that other intensive courses - particularly those<br />
with cohorts with these characteristics - would benefit from a similar pre-workshop initiative. This may<br />
include other areas of post-graduate studies or continuing professional development (CPD) courses.<br />
The longer term benefits of the initiative are also not yet known. This case study has drawn together<br />
findings from three discreet evaluations for three modules. All of the evaluations were conducted<br />
during the classroom based component, not long after the students had worked through the preworkshop<br />
activities. Planned future research will examine the perceptions of a cohort at the end of<br />
their studies – looking at the their views of the initiative as a whole.<br />
To conclude, from the viewpoint of one of the lecturers…<br />
“It’s a very enjoyable experience. It’s completely unique for me… I’ve never been part of<br />
a process like this. I’ve thoroughly enjoyed doing it. I’ve seen it work and I recommend it<br />
to anybody to use.”<br />
References<br />
Conrad, P. A. (1996) Attributes of high-quality intensive course <strong>learning</strong> experiences: Students' voices and<br />
experiences. College Student Journal, 30, pp 69-77.<br />
Daniel, E. L. (2000) “A review of time-shortened courses across disciplines”, College Student Journal, 34, pp<br />
298-308.<br />
Davies, W.M. (2006) “Intensive teaching formats: A review”, Issues In Educational Research, Vol. 16, pp 1-20.<br />
Kasworm, C. (2001) Adult learner experiences of an accelerated degree program. Paper presented at the Adult<br />
Education Research Conference, Michigan State University.<br />
Petrowsky, M. C. (1996) The two-week summer macroeconomics course: Success or failure?, Glendale<br />
Community College.<br />
Scott, P.A. (2003) Attributes of High-Quality Intensive Courses, New Directions for Adult and Continuing<br />
Education, 97, pp 29-38.<br />
Tatum, B.C. (2010) Accelerated Education: Learning on the Fast Track, Journal of Research in Innovative<br />
Teaching, Volume 3, Issue 1, pp 34-50.<br />
Wlodkowski, R. J. (2003) Accelerated <strong>learning</strong> in colleges and universities, New Directions for Adult and<br />
Continuing Education, 97, pp 5-15.<br />
202
Computer-Mediated Reading and its Impact on Learners’<br />
Reading Comprehension Skills<br />
Francisco Perlas Dumanig, Maya Khemlani David and Rodney Jubilado<br />
University of Malaya, Kuala Lumpur, Malaysia<br />
fdumanig@yahoo.com<br />
mayadavid@yahoo.com<br />
rcjubilado@yahoo.com<br />
Abstract: Teaching reading is not an easy skill as most students today easily get bored of reading written texts in<br />
books and other traditional reading materials. With the emergence of technology and its fast development,<br />
educators have integrated the use of computers in teaching reading. Due to technological breakthrough, a study<br />
on computer-mediated reading and its impact on the reading comprehension skills of learners of English as a<br />
Second Language (ESL) is discussed. This study uses an experimental research design to determine the impact<br />
of computer-mediated reading on students’ reading comprehension skills. A total of 140 freshmen engineering<br />
students at Mapua Institute of Technology, Philippines took part in the study. To carry out the study, the<br />
participants were divided into an experimental and a control group. The findings of the study reveal that the use<br />
of computer-mediated reading results in a significant improvement in students’ reading comprehension skills. The<br />
findings are supported by statistical computations using the independent t-test in determining the significant<br />
difference in the pre and posttest results of the two groups. After exposing them to the treatment significant<br />
<strong>learning</strong> is acquired by those in the experimental group as compared to those in the control group. The study<br />
shows that computer-mediated reading creates a significant impact on students’ reading comprehension skills<br />
specifically in following directions, noting details, sequencing events, getting the main idea, making inferences<br />
and making generalizations. On account of the significant findings of the study, it is recommended that teachers<br />
should use computers in the reading classroom.<br />
Keywords: computer-mediated reading, reading comprehension skills, English as a Second Language (ESL),<br />
eLearning, language teaching<br />
1. Introduction<br />
Reading plays a very crucial role in man’s life as reading influences one’s way of thinking, which<br />
consequently affects one’s total development as a person. Although man’s development is influenced<br />
by many factors, <strong>learning</strong> is obtained through reading. Consequently, individuals must find ways how<br />
to develop and improve their reading comprehension skills. Integrating new technologies when<br />
teaching reading can help both the teacher and learner. Technology helps to restructure and improve<br />
the classrooms with authentic activities that will enhance collaboration. It is also an effective tool for<br />
interpretative skills, information management and open inquiry. Consequently, it provides an excellent<br />
avenue for student motivation, exploration and instruction in a multi sensory diverse world. However, it<br />
must be emphasized that using technology for the teaching of reading must be complemented with<br />
teachers who have the appropriate pedagogical skills as technology is only a tool. The challenge rests<br />
with the educators to effectively integrate it in appropriate places through out the curriculum (Wright,<br />
Wilson, Gordon, & Stallworth 2002; Barron & Orwig, 1995).<br />
Before integrating technology in teaching, it is essential for educators to examine first the students’<br />
needs. For instance in applying computer-mediated instruction in developing the reading<br />
comprehension skills of students, teachers should be aware primarily of their students’ level of<br />
reading comprehension skills. This will help teachers to use strategies and think of ideas as to how<br />
technology can be integrated in their teaching. Studies show that sending English teachers to<br />
seminars, workshops and to pursue post graduate degrees help to enhance their competence in<br />
teaching the subject (Dumanig, 2005). Attending such training courses expose teachers to new<br />
strategies particularly in computer-mediated teaching. Anisawal (1993) found that difficulties in<br />
different skills of intermediate students were caused by physical factors and teachers’ lack of<br />
knowledge of the computer. She suggested that sending teachers for training and integrating<br />
technology in teaching would help to improve students’ reading comprehension skills. Teachers must<br />
be wise in understanding their role and use of technology when providing a technologically advanced<br />
<strong>learning</strong> environment.<br />
The integration of technology in teaching brings benefits to students. Traynor (2003) argues that<br />
<strong>learning</strong> by using electronic communication technology helps students develop and enhance their<br />
knowledge and discover new things they might have not thought of without the aid of technology.<br />
203
Francisco Perlas Dumanig et al.<br />
Students can come to understand technology while technological tools help them tap into real<br />
experiences, fantasies and visions. Min Jung Jee (2011) explains that the use of technology<br />
enhances instruction specifically in the field of language instruction. However, the exploration of new<br />
technologies for teaching and <strong>learning</strong> requires time as well as a change in attitude among teachers<br />
and students so that they are able to properly use and apply these tools in teaching. Currently, some<br />
teachers who are not computer literate find it difficult to use computers in their respective classes.<br />
Although computers and multimedia instructional technology are now available in many campuses,<br />
the percentage of teachers using them remains low (Cuban, 2003). It is therefore necessary to equip<br />
teachers with skills in using CAI to facilitate classroom <strong>learning</strong> efficiently and effectively.<br />
Studies on the use of Computer Assisted Instruction (CAI) have identified effective <strong>learning</strong> outcomes<br />
among college students and adult learners (Rouse, 2007). It is believed that CAI appeals to varied<br />
<strong>learning</strong> modalities and consequently meets the diverse needs of every student. With CAI students<br />
can learn at a comfortable pace and interact directly and continually with computers that provide<br />
immediate feedback and positive <strong>learning</strong> effects (Mustafa, Ashhan, and Turgay, 2011). As a result,<br />
many educational institutions today are investing much in technology to enhance students’ <strong>learning</strong>.<br />
In addition, CAI helps students learn many specific skills at various levels, such as; elementary,<br />
secondary, college and adult education. CAI has positive effects on the achievement and<br />
development of students in different skills (Liao, 2007). In fact, the use of any computer software<br />
helps to enhance language teaching and <strong>learning</strong> (Sawhill, 2008). Computers play a valuable role in<br />
focusing students’ attention, presenting content and facilitating students’ understanding in a lecture<br />
(Rap, 2006). Computer programs designed to present information on a screen to students using an<br />
electronic blackboard allows an instructor to present similar kind of information as when he uses<br />
handouts or overhead transparencies. This shows that using CAI does not really deviate from the<br />
traditional approach of teaching, rather it modernizes traditional method using modern technology,<br />
and this makes <strong>learning</strong> easier The use of technology in teaching could be adapted to the type of<br />
activity to be learned .<br />
Cayanan (1998) argues that using computer assisted instruction is significant since it produces a<br />
positive effect on students’ <strong>learning</strong> capacity. It helps students to progress in accordance with their<br />
level of understanding for they learn at their own pace thus making their <strong>learning</strong> gradual yet effective.<br />
With the favorable effects of CAI to students’ <strong>learning</strong> capabilities, Parker (1996) suggests that there<br />
should be improvements to be made in tertiary education regarding the availability and use of<br />
technology in instruction and teacher preparation. It is believed that the use of computer-assisted<br />
instruction will offer a big change and development in improving the quality of education in many<br />
schools around the world.<br />
Due to the fast development of computers the use of computer mediated instruction motivated the<br />
researchers to conduct a study on computer-mediated reading and its impact on learners’ reading<br />
comprehension skills. This study primarily examines the effects of computer mediated reading on<br />
students’ reading comprehension skills such as following simple directions, noting details, sequencing<br />
events, getting the main idea, making inferences and making generalizations.<br />
2. Methodology<br />
This study used a descriptive and experimental research design to determine the effects of computer<br />
assisted instructional devices in teaching reading. These methods were useful in determining the<br />
students’ reading comprehension through the use of computer assisted instructional devices.<br />
The subjects of this study were all freshmen engineering students from Mapua Institute of Technology<br />
(MIT), Manila, Philippines. Four English classes of freshmen engineering students were chosen. The<br />
first two classes were classified as the control group and the other two classes were classified as the<br />
experimental group. The classification of both groups was based on age, gender, computer literacy<br />
and English language proficiency.<br />
The subjects were given two sets of questionnaires, the first set focused on the students’ socio<br />
demographic data to ascertain the students’ age, gender, computer literacy and English proficiency.<br />
The second set of questionnaires focuses on students’ reading comprehension skills. The<br />
questionnaire consists of questions on following simple directions, noting details, sequencing events,<br />
getting the main idea, making inferences and making generalizations. Both sets of questionnaires<br />
204
Francisco Perlas Dumanig,<br />
were distributed to the subjects in the control group and experimental group twice; both in the pre and<br />
posttests. The subjects in the control and experimental groups were given the pretest before they<br />
were exposed to the treatment. After the pretest the subjects in the control group were exposed to the<br />
traditional approach while those in the experimental group were exposed to the use of computer in<br />
teaching reading. The study was carried out in four sessions over four weeks. Each session lasted 1<br />
and a half hours. At the first meeting, the pretest was administered while during the second and third<br />
meetings the actual teaching of reading was conducted and the post test was conducted during the<br />
last meeting.<br />
To validate the findings, an independent t-test was used to determine the significant difference<br />
between the performance of the control group and the experimental group in the pre and post tests. In<br />
addition, a correlated t-test was also used to determine the significant difference between the pre and<br />
posttest scores of both groups.<br />
3. Results<br />
The results of the study are presented in tables to clearly illustrate the reading comprehension skills of<br />
the control and experimental groups in the pre and posttests.<br />
3.1 Pretest reading comprehension level of the control and experimental groups<br />
Table 1 presents a summary of the pretest result of the reading comprehension skills of subjects in<br />
both the control and experimental groups.<br />
Table 1: Pretest level of reading comprehension of the control group and experimental group<br />
Reading Skills No. of Experimental Group<br />
Control Group<br />
Items<br />
Mean Description<br />
Mean Description<br />
Following Simple Directions 5 4.57 Excellent 4.61 Excellent<br />
Noting Details 5 4.29 Good 4.34 Good<br />
Sequencing Events 7 5.77 Very Good 5.36 Meritorious<br />
Getting the Main Idea 5 3.37 Satisfactory 4.33 Very Good<br />
Making Inferences 10 5.59 Satisfactory 4.83 Passing<br />
Making Generalization 3 2.20 Very Satisfactory 2.33 Very<br />
Satisfactory<br />
Table 1 shows the pretest mean of the experimental group’s reading comprehension skill is 4.57 rated<br />
as “excellent” in following directions, 4.29 rated as “good” in noting details, 5.77 rated as “very good”<br />
in sequencing events, 3.37 rated as “satisfactory” in getting the main idea, 5.59 rated as “satisfactory”<br />
in making inferences and 2.20 rated as “very satisfactory” in making generalizations. On the other<br />
hand, the mean of the control group’s reading comprehension skill is 4.61 rated as “excellent” in<br />
following directions, 4.34 rated as “good” in noting details, 5.36 rated as “meritorious” in sequencing<br />
events, 4.33 rated as “good” in getting the main idea, 4.83 rated as “passing” in making inferences<br />
and 2.33 rated as “very satisfactory” in making generalizations.<br />
The results of both groups in different reading comprehension skills show that the subjects have<br />
almost the same level of comprehension skills in the pretest result. The differences in reading<br />
comprehension skills are seen to be minimal and do not create any significance. To measure whether<br />
a significant difference exists in the pretest results, a test of difference is shown In 3.2.<br />
3.2 Test of difference between the pretest scores of the experimental group and<br />
control group<br />
The pretest results of the control and experimental groups are vital in this research. Although, the two<br />
groups of subjects were equated in age, gender, computer literacy and English language proficiency,<br />
the null hypothesis of no significant difference between the score of the experimental group and<br />
control group must be established before the treatment is administered to the experimental group.<br />
Table 2 shows the comparison of the data from the two groups.<br />
The computed t-value is 0.40 which is less than the tabular value of 1.96, therefore, at five percent<br />
level the null hypothesis is accepted. This implies that no significant difference exists between the<br />
pretest mean scores of both groups. Having no significant difference in the scores of the two groups<br />
205
Francisco Perlas Dumanig et al.<br />
in the pretest will help in analyzing if there is a significant difference after exposing the subjects to the<br />
treatment.<br />
Table 2: Comparison of the pretest scores of the experimental group and the control group<br />
Subject Mean S.D. Computed Tabulated Decision Interpretation<br />
t value<br />
t value<br />
Experimental<br />
Group<br />
Control<br />
Group<br />
25.72<br />
25.96<br />
3.51<br />
3.55<br />
0.40<br />
1.96<br />
Accept Ho<br />
Not<br />
Significant<br />
3.3 Level of performance of the experimental and control groups in the posttest<br />
At the end of the experimental period, a posttest was given to the experimental and control groups.<br />
The results are shown in Table 3.<br />
Table 3: Posttest scores of the experimental and control groups<br />
Experimental<br />
Group<br />
Control Group<br />
Score Description f % Score Description f %<br />
32-33 Superior 18 25.71 34-35 Excellent 1 1.43<br />
30-31 Very Good 22 31.43 32-33 Superior 8 11.4<br />
27-29 Good 19 27.14 30-31 Very Good 10 14.2<br />
25-26 Meritorious 9 12.86 27-29 Good 22 31.4<br />
23-24 Very Satisfactory 2 2.86 25-26 Meritorious 17 24.2<br />
23-24 Very satisfactory 10 14.2<br />
21-22 Satisfactory 2 2.86<br />
Total 70 100.00 Total 70 100.00<br />
Mean = 29.49 (Good) Mean= 27.53 (Good)<br />
Standard Deviation = 2.53 Standard Deviation= 3.06<br />
Table 3 shows the percentage scores in the posttest of the experimental and control groups. Twenty<br />
five point seventy one percent (25.71%) of the experimental group is rated as “superior”, in reading<br />
comprehension 31.4 % rated as “very good”, 27.14 % rated as “good”, 12.86 % rated as “meritorious”<br />
and 2.86 % rated as “very satisfactory.” Furthermore, the findings show a standard deviation of 2.53<br />
and a mean of 29.49 rated as “good.” The data tend to show that majority of the subjects are rated<br />
“very good” for they obtain 31.43% and only 2.86 % rated as “very satisfactory”. It is clear then that<br />
the use of computer-mediated instruction is significantly effective in teaching reading particularly in<br />
developing the specific reading comprehension skills. This confirms the findings of (Traynor 2003) that<br />
the process of accumulating knowledge about teaching will be greatly enhanced by technology.<br />
On the other hand, the posttest scores of the control group reveal that 1.43% of the subjects rated as<br />
“excellent”, 11.43% rated as “superior”, 14.29% rated as “very good”, 31.43% rated as “good”,<br />
24.29% rated as “meritorious”, 14.29% rated as “very satisfactory” and 2.86% rated as “satisfactory”,<br />
thus obtaining a mean of 27.53 rated as “good” and a standard deviation of 3.06. Furthermore, the<br />
table shows that the majority of the subjects are rated as “good” which is equivalent to 31.43%. On<br />
the contrary, only 1.43% of the total subjects rated as “excellent.” This means that the improvement in<br />
the reading comprehension skills in the control group is minimal as compared to the experimental<br />
group.<br />
3.4 Posttest reading comprehension level of the control and experimental groups<br />
Table 4 shows the post test result of the experimental group having a mean of 4.93 rated as<br />
“excellent” in following directions, 4.63 rated as “excellent” in noting details, 6.50 rated as “very good”<br />
in sequencing events, 3.94 rated as “good” in getting the main idea, 6.90 rated as “very satisfactory”<br />
in making inferences, 2.79 rated as “excellent” in making generalizations. On the other hand, the<br />
control group obtains a mean of 4.77 rated as “excellent” in following directions, 4.43 rated as “good”<br />
in noting details, 6.44 rated as “very good” in sequencing events, 4.43 rated as “very good” in getting<br />
206
Francisco Perlas Dumanig,<br />
the main idea, 4.84 rated as “passing” in making inferences and 2.54 rated as “excellent” in making<br />
generalizations.<br />
Table 4: The posttest level of reading comprehension of the control and experimental groups<br />
Reading Skills No. of Experimental group<br />
Control Group<br />
Following Simple Directions<br />
Items<br />
Mean Description<br />
Mean Description<br />
5<br />
4.93 Excellent<br />
4.77 Excellent<br />
Noting Details 5 4.63 Excellent 4.43 Good<br />
Sequencing Events<br />
Getting the Main Idea<br />
7 6.50 Very Good 6.44 Very Good<br />
5<br />
3.94 Good<br />
4.43 Very Good<br />
Making Inferences<br />
Making Generalization<br />
10 6.90 Very Satisfactory 4.84 Passing<br />
3<br />
2.79 Excellent<br />
2.54 Excellent<br />
Comparatively, in noting details, the subjects in the experimental group perform better by obtaining a<br />
mean of 4.63 rated as “excellent” whereas the subjects in the control group obtain only a mean of<br />
4.43 rated as “good”. Likewise, in making inferences The experimental group obtains a mean of 6.90<br />
is rated as “very satisfactory” and a control mean of 4.84 rated as “passing”. In getting the main idea,<br />
the control group obtains a mean of 4.43 rated as “very good” whereas the experimental group<br />
obtains 3.94 rated as “good”. The findings reveal that the subjects in the control group perform better<br />
in the pretest but less improvement is shown in the posttest. However, the subjects in the<br />
experimental group show a significant improvement in the posttest for they obtain a mean of 4.33<br />
rated as “very good”. The findings run consistently with the findings of (Mustafa, Ashhan, and Turgay,<br />
2011) that the effectiveness of computer devices, also known as educational technology significantly<br />
enhance student’s <strong>learning</strong> as compared to conventional instruction.<br />
3.5 Test of difference in the pre and posttest scores of the experimental and control<br />
groups<br />
The experimental group was exposed to the use of computer-assisted instructional devices while the<br />
control group was taught using the traditional method of teaching. The results of the pre and posttests<br />
of both groups were compared to determine if the conventional method of teaching or computermediated<br />
instruction is effective. The data are shown in Table 5.<br />
Table 5: Test difference in the pre and posttest scores of the experimental and control groups<br />
Measurement<br />
Experimental<br />
Group<br />
Pretest Scores<br />
Posttest Scores<br />
Control Group<br />
Pretest Scores<br />
Posttest Scores<br />
Mean<br />
15.72<br />
29.49<br />
25.96<br />
27.53<br />
zd<br />
2.59<br />
92<br />
Computed<br />
t-value<br />
13.95<br />
4.87<br />
Tabular<br />
t-value<br />
2.00<br />
2.00<br />
Decision<br />
Reject Ho<br />
Reject Ho<br />
Interpretation<br />
Significant<br />
Significant<br />
In the experimental group, the computed t-value of 13.95 is higher than the tabular value. Therefore,<br />
the null hypothesis is rejected. This means that a significant difference exists between the pretest<br />
scores and the posttest scores of the experimental group. If interpreted further, significant <strong>learning</strong> is<br />
acquired by the members of the experimental group after they were exposed to the treatment. This<br />
agrees with the findings of (Rouse, 2007) that using technology in teaching helps in improving the<br />
reading difficulties of students and enhances their reading comprehension skills.<br />
On the other hand, the control group shows the computed t-value of 4.87, which is higher than the<br />
tabular value of 2.00, therefore, the null hypothesis is rejected. This means that there is a difference<br />
between the pre and posttest scores of the control group. This means that significant <strong>learning</strong> is also<br />
acquired by the members of the control group after exposure to the traditional approach in teaching<br />
reading. However, the difference is less as compared to the experimental group.<br />
207
Francisco Perlas Dumanig et al.<br />
3.6 Test of difference in the posttest results of the experimental and the control<br />
groups<br />
Table 6 shows the comparison of the posttest results of the experimental and control groups. The<br />
level of significance is set at 5 percent and the hypothesis is directional towards the performance of<br />
the experimental group.<br />
Table 6: Comparison of posttests scores of the experimental group and control groups<br />
Subject<br />
Experimental<br />
Group<br />
Control Group<br />
Mean<br />
29.49<br />
27.53<br />
SD<br />
2.53<br />
3.06<br />
Computed<br />
t-value<br />
4.17<br />
Tabular tvalue<br />
(.05)<br />
1.67<br />
Decision<br />
Reject Ho<br />
Interpretation<br />
Significant<br />
From the table, it is seen that the computed value exceeds the tabular value. Therefore the null<br />
hypothesis is rejected hence, the experimental group performs better than the control group in the<br />
posttest. This means that the use of computer assisted instructional devices in teaching reading is<br />
seen to be effective as compared to the traditional approach of teaching reading. CAI improves the<br />
reading comprehension skills of students. We would however like to emphasise that this does not<br />
mean discarding the traditional approach in teaching reading. The traditional or conventional<br />
approach in teaching reading is still effective but the use of computer is seen to be more effective<br />
since students showed improvement after they were exposed to the treatment. The use of computer<br />
mediated instruction still shows an impact on students’ reading comprehension skills after being<br />
exposed to the treatment. Such findings concur with the findings of (Mustafa, Ashhan, and Turgay,<br />
2011) that the use of computer-mediated instruction in teaching produces a positive effect with<br />
respect to the respondents’ <strong>learning</strong> capacity.<br />
4. Discussion<br />
This study is primarily conducted to determine the effects of computer-mediated reading and its<br />
impact on learners’ reading comprehension skills by considering the different reading comprehension<br />
skills of the freshmen engineering students at the Mapua Institute of Technology in the Philippines.<br />
Two hypotheses were tested and the result shows that there is no significant difference in the pretest<br />
result of the experimental and control groups but a significant difference exists in the posttest of the<br />
experimental and control groups. However, the difference in the posttest of the control group is not as<br />
much as that as compared to the experimental group. This shows that using the computer-mediated<br />
instruction is beneficial to improving students’ reading comprehension skills.<br />
The use of technology is seen to be effective since it enhances the students’ reading comprehension<br />
skills such as following simple directions, noting details, sequencing events, getting the main idea,<br />
making inferences and making generalization. Such skills are seen to be a problem to most readers of<br />
English as a Second Language but when using the computer-mediated instruction such barriers<br />
towards <strong>learning</strong> have been minimized. This is evident in the results shown in Tables 1-6. All this<br />
information Francis did this come from qualitative interviews? Otherwise how can you vouch for<br />
this??The use of interactive functions as students read the text may have contributed in the retention<br />
of the message. The memory is triggered by the text that is manipulated by the readers themselves.<br />
In addition, the feeling of being comfortable and free to manipulate the reading text allows the<br />
students to minimize reading anxiety since they are in control of the reading text. Consequently, they<br />
easily absorb and understand what they read.<br />
It must be noted in this study that both computer-mediated instruction and the traditional approach in<br />
teaching reading are both effective in teaching reading. However, the findings shown in Tables 5 and<br />
6 reveal that the use of computer-mediated instruction obtain a much higher mean as compared to<br />
the traditional approach in teaching reading. This only shows that integrating technology in teaching<br />
provides more opportunities for students to develop their reading comprehension skills.<br />
5. Conclusion<br />
Based on the findings of the study, it can be concluded that the use of computer-mediated instruction<br />
in teaching reading helps considerably in improving students’ reading comprehension skills such as,<br />
following directions, noting details, sequencing events, getting the main idea, making inferences and<br />
generalizations. The use of computer-mediated instruction is seen to be useful in teaching reading<br />
208
Francisco Perlas Dumanig,<br />
particularly for the new generation of students who are more exposed to the computer when getting<br />
information. Perhaps boredom disappears when retrieving information using the computer.<br />
Consequently, the use of computer mediated instruction in teaching reading provides positive <strong>learning</strong><br />
effects to students and creates a significant improvement on their reading comprehension skills.<br />
Therefore, reading teachers should use computers as a strategy in enhancing their reading<br />
comprehension skills. Perhaps, minimizing the use of the traditional approach and emphasizing the<br />
integration of technology in teaching reading may produce more competent and skillful readers.<br />
References<br />
Anisawal, L. (1993) “Reading Difficulties of Intermediate Pupils of Bontoc and Salanga Districts”, Unpublished<br />
Master’s Thesis, Baguio City University, Philippines.<br />
Barron, A.E. and Orwig, G.W. (1995) New Technologies for Education 2 nd ed. Englewood, Colorado, Libraries<br />
Unlimited Inc.<br />
Cayanan, A. T. (1998) “The Efficacy of Using Computer-Assisted Instruction (CAI) as a Teaching Aid for College<br />
Students of Philippine Women’s University”,Unpublished Master’s Thesis, Mapua Institute of Technology,<br />
Manila.<br />
Cuban, L. (2003) “Oversold and Underused: Computers in the Classroom”, Language Learning and Technology<br />
Vol. 7, No. 3, pp 42-45.<br />
Dumanig, F. (2005) “The Effects of Computer Assisted Instructional Devices in Teaching Reading”, In KMUTT<br />
and AIAER, ICLORD 2005 Bangkok, pp 352-364.<br />
Liao, Y. (2007) “Effects of Computer-Assisted Instruction on Students’ Achievement in Taiwan: A Meta-analysis’,<br />
Computers & Education Vol. 48, No. 2, pp 216-233.<br />
Min Jung Jee (2011) “Web 2.0 Technology Meets Mobile Assisted Language Learning”, The IALLT Journal Vol.<br />
41, No 1, pp 161-175.<br />
Mustafa, B., Ashhan, K., and Turgay, A. (2011) “The Effect of Computer Assisted Instruction with Simulation with<br />
Science and Physics Activities on the Success of Students: Electric Current”, Eurasian Journal of Physics<br />
and Chemistry Education Jan (Special Issue), pp 34-42.<br />
Rapp, D. (2006) “The Value of Attention Aware Systems in Educational Settings”, Computers in Human Behavior<br />
Vol. 22, pp 603–614.<br />
Rouse, D. (2007) “Computer-Assisted Instruction: An Effective Instructional Method’, Teaching and Learning in<br />
Nursing Vol. 2, No. 4, pp 138-143.<br />
Sawhill, S. (2008) “The Changing Role of the Language Teacher/Technologist”, The IALLT Journal Vol. 40, No.<br />
1, pp 1-17.<br />
Stigler, J. W., and Hiebert, J. (1999) “The Teaching Gap”, New York, Free Press.<br />
Traynor, P. (2003) “Effects of Computer-Assisted Instruction on Different Learners”, Journal of Instructional<br />
Psychology, FindArticles.com. 20 July 2011.<br />
http://findarticles.com/p/articles/mi_m0FCG/is_2_30/ai_105478983/<br />
Wright, V. H., Wilson, E. K., Gordon, W., & Stallworth, J. B. (2002) “Master Technology Teacher: A Partnership<br />
between Pre-service and Inservice Teachers and Teacher Educators”, Contemporary Issues in Technology<br />
and Teacher Education [Online serial], 2(3).<br />
Available:http://www.citejournal.org/vol2/iss3/currentpractice/article1.cfm<br />
209
Do you see What I see? - Understanding the Challenges of<br />
Colour-Blindness in Online Learning<br />
Colin Egan, Amanda Jefferies, Edmund Dipple and David Smith<br />
School of Computer Science, University of Hertfordshire, Hatfield UK<br />
c.egan@herts.ac.uk<br />
a.l.jefferies@herts.ac.uk<br />
Abstract: In this paper we introduce the 'Hertfordshire Colour-blind Emulator' (HCBE) software application.<br />
Currently, our aim is to raise awareness of the challenges that the colour-blind encounter on their <strong>learning</strong><br />
journey. HCBE emulates four major types of colour-blindness: protanopia, deuteranopia, tritanopia and<br />
monochromacy. HCBE accepts an image file and outputs that image in the way that a colour-blind learner would<br />
see the original inputted image. For any inputted images there are four options for outputted images, one for<br />
each colour-blind type. Colour-blindness is often considered to be a mild disability where, on the whole, a colourblind<br />
learner has developed his/her own “coping mechanisms” to avoid, but not to eliminate, problems in their<br />
<strong>learning</strong>. The problems faced by some colour-blind users of online games have recently been highlighted in the<br />
research, this paper identifies some of the issues for students in their <strong>learning</strong>. The Equality Act 2010 places the<br />
responsibility of making reasonable adjustments to aid the disabled on the educational practitioner by providing<br />
legal rights for disabled people, irrespective of the severity of the disability. We show that colour-blind learners do<br />
have problems interpreting information when that information is presented as images. In particular with increasing<br />
reliance on using VLEs (Virtual Learning Environments) as a repository for study materials then it is possible that<br />
the challenges which colour-blind users suffer will be exacerbated. Estimates of the frequency of colour-blindness<br />
show that there are approximately 8% males and about 0.4% females who are colour-blind. It is also our<br />
experience that few practitioners are aware of the problems that colour-blindness can cause and even fewer<br />
practitioners that make any reasonable adjustment as required by the UK’s Equality Act 2010.<br />
Keywords: colour-blind, eLearning, online <strong>learning</strong>, disability, accessibility, Equality Act 2010<br />
1. Introduction<br />
Approximately 8% of males and 0.4% of females in the adult population are typically affected by<br />
colour-blindness (BUPA, 2011). In Further and Higher Education where class sizes of 200 (and<br />
larger) are not unusual, this means that in a class size of about 200 learners there could be up to 16<br />
colour-blind male learners and maybe 1 female learner. As educational practitioners we are typically<br />
using VLEs (Virtual Learning Environments) more and more in the daily delivery of our teaching<br />
materials. Students who suffer from the various forms of colour-blindness may experience problems<br />
when using computers, for example the use of a VLE may cause a colour-blind learner difficulties with<br />
reading text and viewing or interpreting images (including graphs). The focus of this study is to<br />
improve the <strong>learning</strong> experiences of colour-blind learners.<br />
Colour-blindness is normally hereditary and includes a range of conditions where those conditions are<br />
characterised by an inability to see certain colours in the normal spectrum, currently there is no known<br />
cure. There are four major types of colour-blindness: protanopia, deuteranopia, tritanopia and<br />
monochromacy. Most people are aware of the red-green issues for colour-blind users, but this is only<br />
a more common form of the spectrum of effects. It is generally considered to be a mild disability.<br />
However, in the UK, the Equality Act (2010) (which replaces the Disability Discrimination Acts (1995)<br />
and (2005) provides legal rights for all disabled people irrespective of the severity of the disability. As<br />
educational practitioners it is now our responsibility to comply with the Equality Act 2010 and to<br />
provide reasonable adjustments to our educational materials so that colour-blind learners can be<br />
provided with the same opportunity to learn as those who not colour-blind.<br />
In this paper, we introduce the use of a colour-blind software emulator, the Hertfordshire Colour-blind<br />
Emulator (HCBE) which displays coloured images in the same way that colour-blind users would see<br />
them. The aim of HCBE is to raise awareness of some of the problems that the colour-blind encounter<br />
on their <strong>learning</strong> journey and to identify how we as practitioners can help such learners by improving<br />
their educational journey. The software allows the passing of an image into the HCBE application and<br />
then the HCBE outputs coloured images which indicate how a colour-blind learner would see the<br />
same image. We then ask the practitioner a simple question “Do the new images convey the same<br />
message to a colour-blind learner as it would to non colour-blind learners?” Where the answer is yes,<br />
then the practitioner does not need to make any changes as the requirements of the learner and the<br />
Equality Act have both been satisfied. In the alternate case, then the image should be amended to<br />
210
Colin Egan et al.<br />
make reasonable adjustment to the needs of the colour-blind learner and to comply with the Equality<br />
Act. The solution may in fact be to redesign the image with a simple separation of colours or to effect<br />
a simple change of colour schema. Recent research into online gaming has already highlighted the<br />
problems that colour-blind gamers have in differentiating quickly enough between for example friends<br />
and foes in some of the more popular wargames (BBC, 2011)<br />
2. Colour-blindness<br />
Colour-blindness has been defined as “a range of conditions that are characterised by an inability to<br />
see certain colours” (BUPA, 2011). However, colour-blindness can be manifested in many different<br />
ways but there is four most prominent types, which we consider in our study, protanopia,<br />
deuteranopia, tritanopia and monochromacy. To understand how colour-blindness affects how a<br />
person perceives colour, it is important to understand the biology of how the human eye normally<br />
perceives colour. A human retina is comprised of rod and cone cells. Rod cells become active in low<br />
light conditions, and determine the brightness of the image that the brain receives. It is the cone cells<br />
that are affected by colour blindness. The “normal” eye contains 3 variants of cone cells: L, M and S.<br />
The L-cone detects long wavelength light (yellow-red), the M-cone detects medium wavelengths<br />
(green) and the S-cone detect short wavelength light (Blue) (de Paula,2006)). The human brain will<br />
determine the colour of an image based on the ratio between the signals from L, M and S. Colourblindness<br />
occurs when one or more of these cone cells types are either damaged or absent (colourblindness.com,<br />
2011).<br />
The outcome of personal discussions with a number of colour-blind learners has revealed that many<br />
learners have developed their own “coping mechanisms” over their <strong>learning</strong> journey from the early<br />
days of their schooling. “Coping mechanism” may help to reduce the impact of their disability, but it<br />
must be emphasised that such they only reduce and do not eliminate the problems of teaching to the<br />
colour-blind learners. It also highlights that, on the whole, practitioners are not currently making<br />
reasonable adjustments with their teaching material to cater for colour-blind learners.<br />
2.1 Protanopia<br />
Protanopia is manifested as red-green colour-blindness, which is caused by a complete lack of retinal<br />
photoreceptor cells. It has been estimated that approximately 1% of males are affected by this<br />
condition ( Cassin, 1990). Figures 1 a) and b), show a pie-chart that compares normal vision with<br />
protanopian colour-blindness.<br />
Figure 1b) shows a distinct difference to Figure 1a), where for example in red in Figure 1a) appears<br />
as black in Figure 1b), orange in Figure 1a) appears as a green hue in Figure 1b) and so on. If this<br />
pie-chart had been presented as educational material then it is clear that the protanopian colour-blind<br />
person would have difficulty interpreting the data. Furthermore, if the orange and green in Figure 1a)<br />
had been aligned adjacently then the protanopic colour-blind learner would not have been able to<br />
make any differentiation between the two colours. This can also be seen with the blue, and violet of<br />
Figure 1a) and also the indigo and red of Figure 1a). On close examination it can be seen that the red<br />
and green are indistinguishable, and in this example the effect seems to spill over to encompassing<br />
yellow as well.<br />
Figure 1a: Normal vision Figure 1b: Protanopia<br />
211
2.2 Deuteranopia<br />
Colin Egan et al.<br />
Deuteranopia is similar to protoanopia, in that red and green are almost indistinguishable. However,<br />
the cause of the condition is slightly different, in that it is the green receptors in the eye that are<br />
missing [6]. For all intents and purposes the effect is the same as protanopia, although the contrast<br />
between blue and purple seems to be affected to a greater degree by deuteranopia. Figure 2a) is the<br />
same pie-chart as Figure 1a), but Figure 2b) is the same pie-chart but it represents deuteranopia.<br />
Similarly to Figures 1a) and b) Figure 2a) shows a distinct difference from Figure 2b). Following the<br />
same example, red in Figure 2a) appears as black in Figure 2b), the orange in Figure 2a) appears as<br />
a green hue in Figure 2b) and so on. As with Figure 1, if this pie-chart had been presented as<br />
educational material then it is clear that the deuteranopian colour-blind person would have difficulty<br />
interpreting the data.<br />
Figure 2a: Normal vision Figure 2b: Deuteranopia<br />
Furthermore if the orange and green in Figure 2a) had been aligned adjacently then the deuteranopic<br />
colour-blind learner would not have been able to make any differentiation between the two colours.<br />
This can also be seen with the blue, and violet of Figure 2a) and also the indigo and red of Figure 2a).<br />
Again, on close examination it can be seen that that for the colour-blind learners red and green are<br />
indistinguishable.<br />
2.3 Tritanopia<br />
Tritanopia colour-blindness is manifested with a blue-yellow variation due to the lack of blue<br />
photoreceptors. It is not present as frequently as protanopia and deuteranopia [6]. As seen above,<br />
Figure 3a) is the same pie-chart as Figures 1a) and Figure 2a), but Figure 3b) shows the same<br />
coloured pie-chart as those suffering from tritanopia would see it.<br />
Figure 3a: Normal vision Figure 3b: Tritanopia<br />
212
Colin Egan et al.<br />
There are clear differences between normal vision, protanopia vision, deuteranopia vision and<br />
tritanopia vision. Those suffering from tritanopia can distinguish red as shown in Figure 3a) and 3b),<br />
whereas those suffering from protanopia or deuteranopia cannot. However, there are distinct<br />
differences between orange, yellow, green, blue, indigo and violet. On close observation of Figure 3a)<br />
and Figure 3b) it can be seen that blue and green are hard to distinguish, as are purple and<br />
red/orange. Again, as with protanopia and deuteranopia, if this pie-chart had been presented to a<br />
tritanopia colour-blind person there would have been great difficulty in interpreting the presented data.<br />
2.4 Monochromacy<br />
Monochromacy is the complete loss of colour, where colours are visible as shades of grey.<br />
Monochromacy is frequently termed as “total colour-blindness” (Color-blindness.com,2010).<br />
Monochromacy is caused by either defected cones or absence of cones. As discussed above, Figure<br />
4a) is the same pie-chart as Figure 1a), Figure 2a) and Figure 3a), but Figure 4b) shows the same<br />
pie-chart for those suffering from monochromacy.<br />
Figure 4a: Normal vision Figure 4b: Monochromacy<br />
Comparing normal vision with monochromacy vision clearly shows distinct differences to all of the<br />
other varients of colour-blindness. In this case red, orange and green are indistinguishable as are<br />
blue and violet. Only indigo appears distinguishable, but as black. Such a pie-chart would cause a<br />
monchromastic learner a great deal of difficulty interpreting the data.<br />
3. Emulating colour-blindness and HCBE<br />
In this section we discuss how the academic with no colour loss can experience the same view as the<br />
colour-blind user. The simplest method of simulating colour-blindness is by offsetting colours. For<br />
example, in the case of deuteranopia (red-green colour-blindness), in a standard RGB<br />
(red/green/blue) colour value, the red part of the colour could be reduced to 0. However, this method<br />
should not be taken as an accurate portrayal of what a colour-blind person actually sees, since the<br />
cones in the human eye do not map directly to the 3 components of an RGB value. As such, this<br />
method is not a proper representation of colour-blindness.<br />
Alternatively, direct colour mapping can be used where a palette of colours can be referenced directly<br />
against an identically sized palette. However, careful mappings are required to ensure that there are<br />
no errors. A different palette is required for each type of colour-blindness.<br />
A better method is to use linear algorithms where the resultant colour change is computed. Ideally the<br />
result of using direct colour mapping and the usage of linear algorithms should be identical. The<br />
colour values perceived by the eye are mapped out by the LMS (long/medium/short) photoreceptor<br />
activations in the visible spectrum, but these photoreceptor cells do not directly correspond to<br />
standard RGB values. Capilla et al (2004) produced a conversion matrix that converts an RGB value<br />
into an LMS value. In HCBE, we use a matrix based on Capilla's idea where one of the cone values is<br />
modified, depending on which type of colour-blindness is being emulated. Afterwards, HCBE converts<br />
the LMS value back into an RGB value, as shown in Figure 5. The resultant RGB is then outputted.<br />
213
Colin Egan et al.<br />
Figure 5: HCBE's matrix algorithm for RGB conversion to LMS and then LMS conversion back to<br />
RGB<br />
In order to simulate damaged cones or lack of cones, the exact values of all three cones must be<br />
obtained. This can be achieved by using a transformation matrix, such as that proposed by Viénot et<br />
al, (1999) as shown in Figure 6. Viénot's transformation matrix can also be shown as a linear<br />
equation.<br />
Figure 6: HCBE's (Viénot's) transformation matrix algorithm for conversion of RGB to LMS<br />
After modification of either the L, M or S photoreceptor cell value, which depends on the type of colour<br />
blindness, the LMS colour value must then be converted back into RGB to be recognised on the<br />
output display. This is achieved by using the inverse matrix as shown in Figure 7.<br />
To emulate Protoanopia, the L photoreceptor cell values require modification. To emulate<br />
Deuteranopia the M photoreceptor cell value requires modification and to emulate Tritanopia S<br />
photoreceptor cell value requires modification.<br />
214
Colin Egan et al.<br />
Figure 7: HCBE's reverse transformation matrix algorithm for conversion of LMS to RGB<br />
4. Teaching with HCBE<br />
In this section we present an example of the use of a circuit diagram and how this may be seen by<br />
different types of colour-blind users. Figures 8a) 8b) 8c) 8d) and 8e) show a typical circuit block<br />
diagram example used in the teaching of Computer Engineering. The diagram has been generated in<br />
LogicWorks (2010), which is a commonly used interactive tool for the teaching and <strong>learning</strong> of digital<br />
logic. For this study, it is not necessary to have any knowledge or understanding of Computer<br />
Engineering. In this circuit diagram (Quick, 2010) the input signal is obtained from the left hand side<br />
with 1 being an asserted signal and 0 being a de-asserted signal. In this example, both input signals<br />
are asserted as 1 in the boxes. The signals are then transmitted along the (normal vision) red lines<br />
and they then become inputs into various digital logic components (actual details of these<br />
components are not necessary for this study). The output signals are generated and reflected in the<br />
boxes on the right hand side. In this example, all of the output signals are de-asserted and are shown<br />
in the boxes as 0.<br />
By comparison of all the circuit diagrams in 8b), 8c) and 8d), it is evident that colour-blind learners<br />
would have difficulty in interpreting this circuit diagram. For example, comparing Figure 8a) with<br />
Figure 8b) a learner suffering from protanopia, would not be able to see the input signals as they<br />
appear black and grey. Also, there would be difficulty in seeing at least one of the output signals.<br />
Also, comparing Figure 8a) with Figure 8b) a learner suffering from deuteranopia would have similar<br />
problems. Now by comparing Figure 8a) with Figure 8c) a learner suffering from tritanopia would be<br />
able to see the input signals but would have difficulty with the output signals. Finally, comparing<br />
Figure 8a) with Figure 8d) a learner suffering from monochromacy would be unable to detect both<br />
input and output signals. Furthermore, the LogicWorks background grid-lines can also make the<br />
circuit diagram difficult to be viewed by a colour-blind learner.<br />
5. Some example solutions<br />
The Equality Act 2010 clearly indicates that the responsibility of making reasonable adjustments lies<br />
with the practitioner, and yet in reality few (if any) adjustments are typically being made to help colourblind<br />
learners. Once awareness has been raised there are some quick and simple solutions to<br />
enhance the <strong>learning</strong> experience of the colour-blind. From the pie-charts shown as Figures 1 – 4, a<br />
simple separation of individual colours, which may appear merged for a colour-blind learner, may be<br />
sufficient to potentially solve the problem. Alternatively, the use of shading such as stripes and/or<br />
dashes may be a potential solution. In other cases such as the circuit diagram shown as Figure 8, the<br />
use of thicker/thinner lines, dashed lines or embolding lines may potentially solve the problem. We<br />
consider that with thought and creativity the practitioner should be able to improve the <strong>learning</strong><br />
experience of a colour-blind learner without impacting on the experiences of the non colour-blind user.<br />
215
Figure 8a: Normal vision LogicWorks circuit<br />
diagram<br />
Figure 8c: Deuteranopia LogicWorks circuit<br />
diagram<br />
Figure 8e: Monochromacy LogicWorks circuit<br />
diagram<br />
Colin Egan et al.<br />
216<br />
Figure 8b: Protanopia LogicWorks circuit<br />
diagram<br />
Figure 8d: Tritanopia LogicWorks circuit diagram
Colin Egan et al.<br />
6. Discussion and plans for further work<br />
From this study we conclude that colour-blind learners do have considerable difficulties in interpreting<br />
what those without colour-blindness see as simple and/or detailed information and this has been<br />
borne out by inter alia recent research into colour-blind gamers. From our discussions with colourblind<br />
users they have raised particular areas where their <strong>learning</strong> has suffered, with one older user<br />
recalling being dismissed from her school chemistry class because she said that a ‘solution was the<br />
wrong colour.’ Frequent anecdotes from other users relayed to the research team have recalled<br />
problems with coloured wiring, as depicted in the example from the teaching of circuit diagrams given<br />
above.<br />
Currently, many colour-blind learners develop their own method(s) to overcome their disability. Even<br />
though colour-blindness is considered to be a mild disability, the Equality Act 2010 clearly places the<br />
responsibility of reasonable adjustments onto the practitioner. In our experience, currently reasonable<br />
adjustments are rarely made to aid the colour-blind learner. In this paper, we have raised awareness<br />
of the detrimental impact that teaching materials can have on the colour-blind learners. We have also<br />
emphasised that the responsibility of making reasonable adjustments lies with the practitioner and not<br />
with the learner. Furthermore, we have shown that reasonable adjustments can be made with some<br />
consideration and deliberation of the teaching material.<br />
HCBE is in the early stages of development and implementation. Currently HCBE only return images<br />
in the way that a colour-blind learner would see the original inputted image. We, therefore, propose to<br />
enhance HCBE to provide example potential solutions to aid the practitioner to make the necessary<br />
reasonable adjustments of their teaching material for the colour-blind learners. Consequently, the<br />
practitioner should be able to enhance the <strong>learning</strong> experience of the colour-blind learners and<br />
comply with the Equality Act. However, HCBE creating potential solutions will be a non-trivial task.<br />
This is because the solutions HCBE will provide will be generic potential solutions to specific and<br />
detailed teaching material. As HCBE is a software tool, it will have no knowledge or any<br />
understanding of the teaching material. Hence, the ultimate responsibility continues to remain with the<br />
practitioner and HCBE's solutions will serve to identify the problem and support the decision-making<br />
of future suggestions for improving practice.<br />
Acknowledgements<br />
The authors acknowledge with gratitude the support of the University of Hertfordshire’s Proof of<br />
Concept fund for their provision of a generous funding grant to support the HCBE research. We are<br />
also grateful to all those users who have shared their experiences freely to help us understand more<br />
about the difficulties colour-blind users may have with online <strong>learning</strong> materials.<br />
References<br />
BBC, (2011) http://www.bbc.co.uk/news/technology-13054691 (accessed June 16th 2011)<br />
BUPA. (2008). Colour blindness. [Online] (Updated March 2008) Available at:<br />
http://hcd2.bupa.co.uk/fact_sheets/html/colour_blindness.html [Accessed 30 May 2011].<br />
Capilla, P., Luque M J and D´ıez-Ajenjo M J., Looking for the dichromatic version of a colour vision model Jnl.of<br />
Optics A: Pure and Apploed Optics. 6, pp. 906 – 919, 2004.<br />
Cassin, B. and Solomon, S.,( 1990.) Dictionary of Eye Terminology. Gainsville, Florida: Triad Publishing<br />
Company,<br />
Colour-Blindness.com. (2010). How Color Blindness Works. [Online] (Updated 22 nd February 2010) Available at:<br />
http://www.colour-blindness.com/general/how-it-works-science/ [Accessed 30 May 2011].<br />
De Paula J B., Converting RGB Images to LMS Cone Activations TR-06-49 The University of Texas, Austin,<br />
USA. October 2006.<br />
Disability Discrimination Act (1995). Available at: http://www.legislation.gov.uk/ukpga/1995/50/contents<br />
[Accessed 30 May 2011].<br />
Disability Discrimination Act (2005). Available at http://www.legislation.gov.uk/ukpga/2005/13/contents [Accessed<br />
30 May 2011].<br />
Equality Act (2010. Available at: http://www.equalities.gov.uk/equality_act_2010.aspx [Accessed 30 May 2011].<br />
LogicWorks, Available at:http://www.capilano.com/ [Accessed 30 May 2011]<br />
Quick P., (2010) Computer Systems Architecture,Teaching Material for Level 4 BSc Computer Science module<br />
4COM0051. School of Computer Science, University of Hertfordshire. UK. Semester B <strong>Academic</strong> Year<br />
2010/11<br />
Viénot F, Brettel H, Mollon J., (1999) Digital Video Colourmaps for Checking the Legibility of Displays by<br />
Dichromats. Color and Research Application, Vol., Issue 4, pp. 243 – 252, August 1999.<br />
217
Researching in the Open: How a Networked Learning Instance<br />
can Challenge Ethical Decision-Making<br />
Antonella Esposito<br />
Universitat Oberta de Catalunya, Milan, Italy<br />
aesposito47@gmail.com<br />
Abstract: This paper focuses on ethics issues implied in a prospective virtual ethnography study aiming to gain<br />
insights on participants’ experience in an emergent context of networked <strong>learning</strong>, namely a MOOC – Massive<br />
Online Open Course. A MOOC is a popular type of online open course, that provides free content and expertise<br />
to anyone in the world who wishes to enroll. This kind of informal lifelong <strong>learning</strong> initiative is enabled by a network-based<br />
pedagogy and is enacted in a distributed technology-mediated <strong>learning</strong> environment. The purpose of<br />
this article is to explore competing views on ethical decision-making when researching in such a globalized,<br />
online and open <strong>learning</strong> setting. Considering the challenges of this new e<strong>learning</strong> inquiry context, issues as the<br />
underlying research ethics models, the roles of researcher and participants and the integrity of the research<br />
process are discussed in their interplay with the evolving ethos of the ethnographical methodology being adopted<br />
to investigate participants’ views. Elements drawn from the design of a qualitative study are here utilized to identify<br />
an empirical instance that shapes and is being shaped by research ethics decisions. The study aims to answer<br />
the following question: what are the affordances (opportunities and challenges) of online open courses as<br />
they emerge from the participants’ perspectives? This paper considers the potential operationalization of the<br />
above research question and discusses both theoretical and methodological issues arising from applying research<br />
ethics to this specific case of Internet inquiry. In this sense, ethical approaches in online research contexts<br />
as well as main ethical decisions are discussed and justified, envisioning a submission to an institutional<br />
ethics review board before undertaking the ethnographical study. Topics such as privacy concerns in a public<br />
online setting, choice between overt and covert research, researcher as observer or participant, narrow or loosely<br />
defined application of the informed consent and anonymity are outlined, presenting a range of different options.<br />
This article intends to show that ethical decisions are an iterative procedure and an integral part of the research<br />
design process. Moreover, it endorses the opportunity to produce localized and contextualized ethical decisionmaking.<br />
To this end, it takes into account the guidance available (research ethics literature; narratives of ethics<br />
procedures applied to empirical cases); the ethics debates within the ethnographical tradition and the nature of<br />
the setting being researched (the specific format of the networked <strong>learning</strong> instance being examined). The discussion<br />
here proposed orientates ethical decision-making towards an overt and participant research approach,<br />
an informed consent intended as a ‘public notice’ and a consideration of participants both as authors in the online<br />
setting and as human subjects embedding unexpected privacy sensitiveness. However such decisions are intended<br />
as many starting points to build a research ethics protocol intended to a degree as a work in progress, in<br />
a problem-solving approach guided by the practical wisdom of participants emerging over time.<br />
Keywords: internet research ethics, massive online open courses, virtual ethnography, situated ethics<br />
1. Introduction<br />
The emerging networked <strong>learning</strong> environments add unexpected ethical challenges to researchers:<br />
this prompts the exploration of a new vocabulary of research ethics (Beaulieu and Estalella 2009),<br />
and demands a greater effort for “unravelling the intricate tapestry of ethics and method in research<br />
design and process” (Markham 2007: 3). It also seems to suggest an ethics decision-making approach<br />
that strives to combine general rules of the codes of conduct provided by the institutional review<br />
boards with a continuining effort to gain phronesis or ‘practical wisdom’, “which demands understanding<br />
of specific situations and reference to prior experience” (Tracy and Carmichael 2010).<br />
This paper proposes a reflection on an empirical case of networked <strong>learning</strong> inquiry, as outlined in a<br />
small exploratory research to be preliminarily undertaken within my doctoral thesys focusing on open<br />
education issues in postgraduate online research training. This article deals with ethics issues implied<br />
in this preliminary virtual ethnography study, aiming to gain insights on participants’ experience in an<br />
emergent context of online open education, namely a Massive Online Open Course (MOOC). The<br />
general goal of this paper is to cope with the main issues to be included in an ethical review to be<br />
submitted to an institutional ethics review board, before undertaking a study on this topic. However, in<br />
the current evolving landscape of technology–mediated ethnography (Markham 2003; Beaulieu and<br />
Estalella 2009; Estalella 2007), there is an increasing awareness that the mandatory rules imposed by<br />
the institutional review boards are necessary but not sufficient conditions to illuminate hidden aspects<br />
and suggest practical solutions to researchers in online inquiry settings. So, issues such as different<br />
approaches to ethical issues in an online research context, privacy concerns in a public online setting,<br />
the choice between overt and covert research, the application of the informed consent and issues of<br />
218
Antonella Esposito<br />
anonymity are outlined as many springboards to build a ‘doable’ research ethics protocol. However,<br />
decision-making on ethical issues is intended here to be a continuining reflexive interrogation of one’s<br />
method of inquiry to reveal “hidden ethical practices from inside” (Markham 2007: 3). This engagement<br />
in the interrogation of one’s method is complemented by a recursive work aiming to a “production<br />
of localized, contextualized ethical decision-making” (Whiteman 2009: 65), that needs to be<br />
adapted over time. Given that, the initial elaboration of the research ethics strategy takes into account<br />
the guidance available (research ethics literature; narratives of ethics procedures applied to empirical<br />
cases); the ethics debates grounded in the ethnographical tradition and the nature of the setting being<br />
researched (the specific format of networked <strong>learning</strong> instance being examined).<br />
2. The research question<br />
I assume as the specific setting of the prospective pilot study a MOOC's edition (Cormier and Siemens<br />
2010), carried out in the area of educational technology and addressing lifelong learners all<br />
over the world. This kind of informal <strong>learning</strong> experience is enabled by a network-based pedagogy<br />
and enacted in a public, distributed technology-mediated <strong>learning</strong> environment (Moodle forums and<br />
any social media). Usually a few hundreds of individuals play a role as active learners, against thousands<br />
of ‘lurkers’.<br />
The research question I would like to focus on is the following one: what are the affordances (opportunities<br />
and challenges) of online open courses as they emerge from the participants’ perspectives?<br />
The goals of the proposed study entail: 1) to understand participants’ experience in an emergent context<br />
of online education; 2) to draw recommendations for future course design of online open courses.<br />
A virtual ethnography perspective is being applied as methodology: observations of communication<br />
occurring in forums and social media among active participants and an online, open-ended and<br />
anonymous questionnaire are considered as main data gathering methods.<br />
3. Approaching ethical implications of the online setting<br />
Definition and application of ethical issues to specific research contexts is never straightforward –<br />
both in offline and online contexts - because they are made by a complex blend of social norms, values<br />
and legal issues, are dependent on national and local traditions, and refer to different ethics philosophical<br />
frameworks. Ess (2004) notices that in the EU deontological frameworks are usually applied,<br />
while in the US utilitarian approaches are more common, and elsewhere virtue-laden perspectives<br />
are considered. However, in 2002 the AoIR - Association of Internet Researchers defined specific<br />
recommendations that focus on ethical pluralism and flexiblity of guidelines to be adopted by individual<br />
researchers and institutional review boards.<br />
The case being investigated is positioned in the realm of Internet-based research, in which ethics<br />
mainly refer to a human subject research model (Bruckman 2002b) and focuses – as well as in offline<br />
contexts - on issues such as public versus private ownership, informed consent, anonymity and confidentiality<br />
(Eysenbach and Till 2001; Mann and Stewart 2000). To date, these issues keep on causing<br />
ethical discord and uncertainty in e<strong>learning</strong> research (Kanuka and Anderson 2007) as well as in a<br />
number of disciplines (Bruckman 2002b). Other views value a model of Internet research that intends<br />
“Internet as production of cultural texts” and online subjects as authors (Basset and O’Riordan 2002),<br />
envisioning an ethics approach which considers issues such as “appropriation, reproduction and removal<br />
of Internet texts from their original context” (ib.).<br />
Moreover, within the ethnographical tradition, Beaulieu and Estalella (2009) have recently led attention<br />
to consider contiguity of settings (blurring distinctions between research fieldwork, the place<br />
where analysis is being carried out and where findings are disseminated and published) and traceability<br />
of data (“the property of inscriptions to be located through search engines and other mechanisms”,<br />
ib.) as many typical tensions and features of the technology-mediated ethnography settings, that urge<br />
a rethinking of the same principles of research ethics, such as anonymization, exposure, authorship<br />
and ownership.<br />
The ethical approach being adopted takes into account the ethical stances of virtual ethnography<br />
(Markham 1998, 2003; Hine 2000), which relies on human subject research model, but also explores<br />
new kinds of text-based settings, such as a MOOC mostly is. So, to a degree this ethical approach<br />
also values participants as authors; and above all considers ethical expectations by participants in<br />
the specific istance being investigated (Eysenbach and Till 2001; Ess and AoIR 2002), as drawn by<br />
219
Antonella Esposito<br />
previous editions of these pilot courses (i.e. PLENK 2010; Siemens 2010; Cormier and Siemens<br />
2010). The aim is to balance the need of preserving research integrity with the provision of advantages<br />
to participants and the effort to minimize related dangers.<br />
4. Public versus private ownership<br />
The public nature of the open <strong>learning</strong> environment established in a MOOC (Fini 2009; Mak et al.<br />
2010) seems to facilitate the collection of large amounts of observational data, with minor privacy<br />
concerns by researchers. Indeed, unlike formal e-<strong>learning</strong> environments, a MOOC is likely to be assimilated<br />
to an open web space, since it takes place in multiple, non reserved areas (also Moodle forums<br />
usually have full visibility to non-enrolled readers) and it provides loosely defined constraints of<br />
'enrollment' and 'attendance'. In fact, learners can participate and withdraw at any time and at whatever<br />
level of frequency and engagement, and they can mark their own online presence and interact<br />
with their facilitators and peers while undertaking ordinary social/academic/professional activities using<br />
their preferred social media. Given that, drawing from recent social research studies with popular<br />
technologies, it is worth taking into account that behaviours linked to voluntary choice and use of social<br />
media “suggest a mindful aspiration for publicity” (Vieweg 2010) by participants. Thus boundaries<br />
between public and private ownership tend to blur in the open <strong>learning</strong> environment being considered.<br />
Indeed, the problem of ownership of messages' transcripts in Net-based spaces is still controversial<br />
(Kanuka and Anderson, 2007): my general view follows Mann and Stewart's position that when one<br />
posts a message "there is an implied license to read, or even archive, the information it contains"<br />
(2000: 46). It is worth noting that the informed consent adopted in a recent MOOC (PLENK 2010) declared<br />
a default research use of all posted messages tagged with the course's name: otherwise, authorization<br />
was asked. Indeed any acknowledgement of authorship of more complex artefacts being<br />
produced during a MOOC is hardly framed within IPR issues. In fact, the setting of such online open<br />
courses is grounded in a culture of sharing that is at the heart of the knowledge production model<br />
suggested by these experimental <strong>learning</strong> projects. This cultural mood is fed with forms of acknowledgement<br />
of the individual original production – such as the mutual acknowledgement among participants<br />
- that are different from the mere protection of copyright and have more to do with the construction<br />
of one's own digital identity.<br />
Nonetheless, it is also considered that enrolled learners in a MOOC are potentially all over the world<br />
and therefore they are likely to have different cultural and personal sensitiveness about privacy issues<br />
(Vieweg 2010). Moreover, analysis of some specific threads of discussion (e.g. self-evaluation,<br />
learner experiences, etc) might reveal for instance feelings of discomfort by some learners – often<br />
accredited professionals - in a complex networked environment such as a MOOC: these learners<br />
might feel violated if they saw their posts de-contextualized and highlighted in a publication. In this<br />
case the researcher might be at danger to enter learners' private sphere: to mitigate any distress the<br />
researcher should contact authors’ posts to let them the possibility to choose to be anonymized or<br />
credited. Given that, as a general recommendation a debriefing opportunity – usually planned in the<br />
final phase of the study – could be provided as a continuing dialogue between researcher and informants<br />
to be carried out in a devoted forum, in order to monitor if any harm is being perceived and to<br />
provide timely solutions.<br />
Here also issues related to researcher’s sense of ownership and authorship should be considered,<br />
recalling the notion of contiguity of settings discussed by Beaulieu and Estalella: “While fieldwork is<br />
never easy, we felt at time exposed, surveilled and even, on occasions, that actors in the field or colleagues<br />
from ‘home’ were foreclosing on our research” (2009: 8). This challenging perspective questions<br />
traditional conventions to manage distinctions between the role as a fieldworker and that as an<br />
academic, between participants’ and researcher’s voice.<br />
5. Overt vs covert research approach<br />
The ethical attitude of an overt research approach is being endorsed to preserve individual informants,<br />
seen as ‘participants’, and social ecology of the community (Cohen et al. 2007: 156-175). The<br />
negotiation of access to the fieldwork, an “acclimatization process” (Chen et al. 2004: 172), the long<br />
permanence in the field, the acquisition of competence of informants and debriefing procedures are<br />
being used as many cautions to mitigate the disrupting character of the researcher’s intervention. The<br />
disclosure of the researcher’s presence is just among the benchmarks of effective ethnographies<br />
(Splinder and Splinder 1992: 65) and has methodological and ethical relevance in virtual ethnography,<br />
as a by-product of a negotiation of access and self-presence (Hine 2008: 264). However, others hold<br />
that a researcher’s behaviour as a “lurker” is acceptable (Paccagnella 1997; Beaulieu 2004:146), just<br />
220
Antonella Esposito<br />
because online ethnographical observation can be considered as less intrusive than in offline contexts.<br />
On the one hand some official guidelines (Ess and AoIR 2002) seem to authorize a covert role by researchers,<br />
whether participants have chosen to post publicly. On the other hand, Bakardjieva and<br />
Feenberg (2001) warn researchers with respect to ‘technically’ public research settings and refer to a<br />
“non alienation principle”, whereby everyone is welcome to join and use online communities but not to<br />
‘harvest’ or sell information therein.<br />
That said, I think that a covert approach could be even counter-productive for data collection in the<br />
setting being researched, in which participants are encouraged to share critical and creative contributions<br />
within the community. The same disclosure of researcher’s work might constitute a metareflection<br />
in its own right, useful both to community of learners to critically reflect on the collective<br />
knowledge building effort and to course authors to enhance understanding of learners’ perceptions of<br />
the networked <strong>learning</strong> experience. Therefore, the adoption by the researcher of a role as ‘observer<br />
as participant’ (Cohen et al. 2003:179) is likely to be tolerated by the community of learners, by negotiating<br />
access and self-presence (Hine, 2008: 264) and thus appears to be more functional to the<br />
need to gain insights on challenges and opportunities of an online open course. Moreover, such a<br />
choice seems to be inescapable in the endorsed virtual ethnography perspective:<br />
“lurking online to collect data without participating in culture may not just be less desirable,<br />
but perhaps not possibile if the goal is to explore sense making practices” (Markham,<br />
2003: 5).<br />
6. Informed consent<br />
Informed consent is certainly the key issue to be addressed when building an ethical framework. The<br />
choice of an overt and active approach makes the informed consent an instrument for researcher to<br />
demonstrate credibility and trasparency of conduct. However, there are contrasting views about its<br />
mandatory character and its operationalization.<br />
In my view, the informed consent to be submitted to MOOC’s participants of the proposed study could<br />
assume the form of a mere ‘public notice’ (Ess and AoIR, 2002: 7), before the observational data<br />
collection starts. Therefore, a ‘reverse tecnique’ is being proposed, whereby participants must inform<br />
the researcher if they don’t wish to be investigated as posts’ authors.<br />
Indeed, some scholars (Bakardjieva and Feenberg, 2001; Bruckman, 2002a) state that obtaining<br />
consent (through signed form) from each participant is mandatory, even if it is logistically difficult or<br />
potentially disruptive of the online environment. On the other hand, Fahy and Spencer (2004: 33-34)<br />
maintain that an ethics institutional board can waive informed consent where minimal risks in research<br />
are present, if subjects can be provided with additional information after participation, or there are<br />
serious hurdles in getting it. Indeed, to obtain consent through a signed form from individual<br />
participants can be fairly easy in a voluntary online questionnaire survey, for instance by including a<br />
‘check box’ in the online form. However, this might be problematic for observations, given the<br />
‘rhizomatic’ environment of a MOOC, as characterized by high numbers of enrolled people,<br />
discontinuity in learners’ participation and uncertainty to retrieve reliable contact information in<br />
participants’ profiles. Given that, some hold that in a public arena to ask individual participants for<br />
signing and sending back a form seems to be inefficient and time consuming, whilst posting a general<br />
message to inform the community appears to be intrusive (Eysenbach and Till, 2001). Following<br />
Moreno et al. (2008), I set out firstly to obtain permission by the course’s authors, who can act as<br />
many gatekeepers to the research setting, and secondly to widely inform the community of learners<br />
as a whole about the study’s details. In fact, given the non sensitive nature of the topic and the<br />
peculiar research-focused attitude of the setting, I think that to set up an informative web space within<br />
the course’s forum area could be an acceptable solution for course participants to become aware of<br />
research aim and design, gathering and data protection methods, types of dissemination outputs and<br />
planned levels of commitment. In this line, Fahy and Spencer maintain that, under conditions given<br />
above, the “rights of the majority to participate in research are protected over the objections of those<br />
who may not wish to do so” (2004: 33). Nonetheless, in the proposed study participants would be<br />
involved in an iterative debriefing process (occurring in the same forum area), would be allowed to<br />
withdraw consent at any time in questionnaires (through a devoted ‘exit’ button) and would be able to<br />
communicate to researcher if they have any objections to be directly quoted in research accounts or<br />
if they prefer to withdraw.<br />
221
Antonella Esposito<br />
However, beyond the formal fulfilments, a further step could be considered, reflecting on the<br />
participatory nature of the <strong>learning</strong> community being investigated and on the varied cultural and<br />
professional backgrounds of research participants. The same informed consent could be developed to<br />
a degree as a work in progress, in a problem-solving approach guided by the practical wisdom of<br />
participants emerging over time: “the different disciplinary perspectives and varied experience<br />
represented offer not only creative solutions to dilemmas but are also a source of critique of the<br />
ethical framework itself, that continues to evolve as a result” (Tracy and Carmichael 2010: 254).<br />
7. Anonymity<br />
One of the risks of naturalistic research is the over-exposure of individuals and groups, which can be<br />
just avoided preserving their anonymity or at least, as I set out to do, allowing informants to choose<br />
anonymity or disclosure of their personal data.<br />
As regards to observation, taking cue from the adoption of a ‘public notice’ to inform research<br />
participants, I opt for a ‘no disguise’ approach (Bruckman, 2002b), that is a use of pseudonyms or real<br />
names of the posts’ authors. In fact, I consider the low level of risk of the research to be undertaken<br />
but also that “Anonymity may not always be preferred as default, especially in a participatory culture,<br />
where people want to be attributed to the stories they publicly share” (Liu, 2010: 2). Indeed,<br />
participants in a MOOC are invited to comment each other and disseminate blog entries and to<br />
experience a role as producers of “remixed” content and various digital artefacts. So, to a degree,<br />
anonymizing material such as videos, diagrams or blog posts could be even perceived as a harm by<br />
active participants in such a context. This is also in line with the idea of the Internet users as “amateur<br />
artists” (Bruckman, 2002b) to whom it seems appropriate to give credit for their work if they desire it.<br />
So, if on the one hand disclosure of participants as authors can appear as a concession (justified by<br />
the setting’s features) to an Internet research model focusing on textuality, on the other hand the use<br />
of an anonymous online questionnaire can help to give voice to the numerous lurkers – who otherwise<br />
would be unreachable and unheeded - without disrupting their privacy.<br />
Moreover, once again taking cue from debates on technology-mediated ethnography, a further issue<br />
is considered, that sheds a new light on the traditional concept of anonymity as a ‘protection’ bulwark<br />
of subjects: “Being traceable could actually mean greater, and more diverse accountability” (Beaulieu<br />
and Estalella 2009). Just because the traces of researcher’s activities can be found online and are<br />
potentially disclosed to all research participants and stakeholders, a “more subtle and modulated<br />
approach to human subject protection” can be envisioned as an object for new discussions and<br />
formulation of future solutions.<br />
8. Conclusions<br />
This paper stated and justified main ethical decisions to be undertaken in a potential<br />
operationalization of a research question focusing on opportunities and challenges of online open<br />
courses, as emerging from participants’ perspectives. The discussion here proposed orientates the<br />
ethical decision-making towards an overt and participant research approach, an informed consent<br />
intended as a ‘public notice’ and a consideration of participants both as authors in the online setting<br />
and as human subjects embedding unexpected privacy sensitiveness. Such choices are highlighted<br />
as many issues to be submitted to an institutional ethics review board for further negotiation and<br />
approval. However, following Markham’s (2007) recommendations for a ‘reflexive ethics’ that<br />
recursively intertwines ethical and methodological decisions, such decisions are intended as many<br />
starting points to build a research ethics protocol intended as a work in progress. In fact, an open<br />
networked <strong>learning</strong> environment encourages a participatory research approach and therefore fosters<br />
creative suggestions and shared solutions from participants, in an evolving landscape of ethical<br />
opportunities and challenges. This entails for the researcher to devise and assume new kinds of<br />
responsibility and accountability, to research participants and to the same role as a researcher.<br />
Acknowledgements<br />
An early version of this paper was written as a final essay of a module within the MRes in Educational<br />
and Social Research, Institute of Education, University of London. I would like to acknowledge Dr.<br />
William Gibson and Dr. Soh-young Chung for their invaluable support and encouragement.<br />
222
References<br />
Antonella Esposito<br />
Bakardjieva M. and Feenberg A. (2001) “Involving the virtual subject”, Ethics and Information Technology, vol. 2,<br />
n.4, pp. 233-240.<br />
Bassett, E. H. and O’Riordan, K. (2002) “Ethics of Internet Research: Contesting the Human Subjects Research<br />
Model”, Ethics and Information Technology, vol. 4, pp.233-247,<br />
http://www.nyu.edu/projects/nissenbaum/ethics_bas_full.html (accessed on 17/01/2011).<br />
Beaulieu, A. (2004) “Mediating ethnography: objectivity and the making of ethnographies of the internet”, Social<br />
Epistemology, Apr-Sep2004, Vol. 18 Issue 2/3, pp. 139-163.<br />
Beaulieu, A. and Estalella, A. (2009) "Rethinking research ethics for mediated settings" Cologne, Germany. The<br />
5th International Conference on e-Social Science, session 'Ethics and e-Research’, 2009, June 24-26,<br />
http://www.estalella.eu/eventos-academicos/ethics-and-research-practices-in-mediated-ethnography (accessed<br />
on 20/01/2011).<br />
Bruckman, A. (2002a) Ethical guidelines for research online [online], from http://www.cc.gatech.edu/~asb/ethics/<br />
(accessed on 15/01/2011).<br />
Bruckman, A. (2002b) “Studying the Amateur Artist: A Perspective on Disguising Data Collected in Human Subjects<br />
Research on the Internet”, Ethics and Information Technology, vol. 4, n.3, pp. 217-231,<br />
http://www.nyu.edu/projects/nissenbaum/ethics_bru_full.html (accessed on 12/01/2011).<br />
Chen, S. L., Hall, G. J. and Johns, M. D. (2004) “Research paparazzi in cyberspace: The voices of the researched”,<br />
Chen, S. L., Hall, G. J. and Johns, M. D. (2004) Online social research: Methods, issues and<br />
ethics, New York, Peter Lang, pp. 157-175.<br />
Cohen, L., Manion, L. and Morrison, K. (2007) “The ethics of educational and social research”, Cohen et al.<br />
(2007), Research Methods in Education, Routledge, pp. 51-77.<br />
Cormier, D. and Siemens, G. (2010) “Through the Open Door: Open Courses as Research, Learning, and Engagement”<br />
[online], EDUCAUSE Review, vol. 45, no. 4 (July/August 2010): 30-39,<br />
http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume45/ThroughtheOpenD<br />
oorOpenCoursesa/209320 (accessed on 10/01/2011).<br />
Ess, C. and AoIR Ethics Working Committee (2002) Ethical decision-making and internet research: Recommendations<br />
from the AoIR ethics working committee [online] http://www.aoir.org/reports/ethics.pdf (accessed on<br />
13/01/2011).<br />
Estalella, A. (2007) “Field Ethics: Towards Situated Ethics for Ethnographic Research on the Internet”, Forum<br />
Qualitative Social Research, Vol. 8, No. 3. (2007), http://www.qualitativeresearch.net/index.php/fqs/article/view/277<br />
(accessed on 31/05/2011).<br />
Eysenbach, G. and Till, J. E. (2001) “Ethical issues in qualitative research on internet communities”, in BMJ, 10<br />
November 2001, http://www.bmj.com/content/323/7321/1103.full (accessed on 12/01/2011).<br />
Fahy, P., & Spencer, B. (2004) “Research experience and agreement with selected ethics principles from Canada’s<br />
Tri-Council Policy Statement: Ethical conduct for research involving humans”, Journal of Distance<br />
Education, vol. 19, n.2, pp. 28-58.<br />
Fini, A. (2009) “The technological dimension of a massive open online course: The case of CCK08 course tools”,<br />
in IRRODL, vol. 10, n. 5 (2009), http://www.irrodl.org/index.php/irrodl/article/viewArticle/643/1402 (accessed<br />
on 09/01/2011).<br />
Hine, C. (2000) Virtual Ethnography, London, Sage.<br />
Hine, C. (2008) “Overview: Virtual ethnography: modes, varieties, affordances”, N. G. Fielding, R.M. Lee and G.<br />
Blank (eds.) Handbook of Online Research Methods, London, Sage.<br />
Kanuka, H., & Anderson, T. (2007) “Ethical issues in qualitative e-<strong>learning</strong> research”, in International Journal of<br />
Qualitative Methods, vol. 6, n.2, http://www.ualberta.ca/~iiqm/backissues/6_2/kanuka.htm (accessed on<br />
10/01/2011).<br />
Kiegelmann, M. (1996) The Subject Writes Back, Reflections on Ethics in Qualitative Research, conference paper<br />
at American Educational Research Association, New York, Harvard University.<br />
Liu, S. B. (2010) “The Emerging Ethics of Studying Social Media Use with a Heritage Twist”, presented at workshop<br />
Revisiting Research Ethics in the Facebook Era: Challenges in Emerging CSCW Research, The ACM<br />
Conference, Savannah, Georgia, US, http://www.cc.gatech.edu/~yardi/ethicscscw2010_files/AcceptedPapers.htm<br />
(accessed on 10/01/2011).<br />
Mann, C. and Stewart, F. (2000) Internet Communication and Qualitative Research, London, Sage.<br />
Mak, F. J., Williams, R. and Mackness, J. (2010) “Blogs and forums as communication and <strong>learning</strong> tools in a<br />
MOOC”, in (eds) , Proceedings of the 7 th International Conference on Networked <strong>learning</strong>, pp. 275-284,<br />
http://www.lancs.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/Mak.html (accessed on 18/01/2011).<br />
Markham, A.N. (1998) Life online. Researching real experiences in virtual spaces, Walnut Creek, CA, Altamira<br />
Press.<br />
Markham, A. N. (2003) “Representation in online ethnographies: A matter of context sensitivity” [online], Chen,<br />
S., Hall, J. and Johns, M. (eds) (2003) Online social research: Theory, Methods and Ethics, Peter Lang<br />
Publishers, http://collections.lib.uwm.edu/cipr/image/104.pdf (accessed on 17/01/2011).<br />
Markham, A. N. (2007) “Method as ethic, ethic as method. A case for reflexivity in qualitative ICT research”,<br />
Journal of Information Ethics, vol. 15, n. 2, pp.37-55, http://markham.internetinquiry.org/writing/jie.pdf (accessed<br />
on 20/05/2011).<br />
223
Antonella Esposito<br />
Moreno, M. A., Fost, N. C. and Christakis, D. A. (2008) “Research Ethics in the MySpace Era”, Pediatrics, Vol.<br />
121 No. 1 January 2008, pp. 157-161, http://pediatrics.aappublications.org/cgi/content/abstract/121/1/157<br />
(accessed on 15/01/2011).<br />
Paccagnella, L. (1997) “Getting the seats of your pants dirty: strategies for ethnographic resarch on virtual communities”<br />
[online] Journal of Computer Mediated Communication vol. 3, n.1,<br />
http://jcmc.indiana.edu/vol3/issue1/paccagnella.html#bio (accessed on 15/01/2011).<br />
PLENK 2010 – Personal Learning Environments and Knowledge, ‘How this course works’,<br />
http://connect.downes.ca/how.htm (accessed on 15/01/2011).<br />
Siemens, G. (2010) “Reflections on open courses”, blog post August 19 th 2010, in Connectivism. Networked and<br />
social <strong>learning</strong>, http://www.connectivism.ca/?p=267 (accessed on 10/01/2011).<br />
Tracy, F. and Carmichael, P. (2010) “Research ethics and participatory research in an interdisciplinary technology-enhanced<br />
<strong>learning</strong> project”, International Journal in Research & Method in Education, vol. 33, n. 3, pp.<br />
245-257.<br />
Vieweg, S. (2010) “The Ethics of Twitter Research”, presented at workshop Revisiting Research Ethics in the<br />
Facebook Era:Challenges in Emerging CSCW Research, The ACM Conference, Savannah, Georgia, US,<br />
http://www.cc.gatech.edu/~yardi/ethics-cscw2010_files/AcceptedPapers.htm (accessed on 09/01/2011).<br />
Whiteman, N. (2009) “Internet Research”, Research Methods Lecture Pack, Mres in Educational and Social Research,<br />
Institute of Education, pp. 57-67.<br />
224
Making Constraints and Decisions Explicit to Support<br />
Project-Based Collaborative Learning<br />
Gert Faustmann<br />
Department of Cooperative Studies, Berlin School of Economics and Law,<br />
Germany<br />
gert.faustmann@hwr-berlin.de<br />
Abstract: Learning projects serve as motivational instruments to transfer knowledge and competences relating to<br />
real world problems. Yet, the <strong>learning</strong> effect varies among participants. The anticipated results of a project and<br />
their relation to real life questions are formed by complex requirements and, while the project advances, the<br />
complexity of the results emerging increases. Furthermore, project groups often consist of participants with little<br />
experience in the subject of the project. While the project progresses, the overview of requirements and results<br />
already developed is lost. The <strong>learning</strong> effect can now be raised by making the requirements on the expected<br />
project results as well as the decisions for existing results explicit. In this way, participants understand the<br />
decisions of others engaged in the ongoing project and they are able to align the existing results with the<br />
requirements imposed on them. Additionally, new ways of collaborative <strong>learning</strong> are possible when, for instance,<br />
the task is to develop constraints on or between the respective project results. Former research developed a<br />
concept to keep less experienced project members in the productive process while forcing other project members<br />
to give arguments for their results. The current work describes an approach which aims to integrate the<br />
requirements on results in the <strong>learning</strong> process. While arguments were to include digital documents or specific<br />
parts of these documents for clarification purposes, the incorporation of explicit requirements is now also<br />
manifested in digital documents. The requirements can then be related not only to single result documents, but<br />
also to whole sets of expected documents. The paper gives an example of a software development project where<br />
some design documents have to be developed. It shows how constraints on these documents are presented and<br />
how these constraints affect the collaborative <strong>learning</strong> process.<br />
Keywords: project-based <strong>learning</strong>, cscl, arguments, digital documents, <strong>learning</strong> process<br />
1. Introduction<br />
The integration of <strong>learning</strong> projects into teaching is a common instrument and improves the ability of<br />
students to solve real world problems (Doppelt 2003). Yet, certain problems arise when practicing<br />
project based <strong>learning</strong>:First, the participants’ knowledge of the project in relation to the project task<br />
often differs. While this is a key reason for implementing a “collaborative” approach in <strong>learning</strong>, for<br />
some learners the <strong>learning</strong> effect may stagnate, because the project advances and the results<br />
become more complex making it more difficult for them to understand the new results and their<br />
relationship. Without any explicit help the students will not be able to contribute to the project.<br />
Second, in nearly all domains the expected results share complex requirements and dependencies,<br />
which themselves can be seen as knowledge about the domain space. We can distinguish between<br />
requirements that are related to just a single project result (e.g. the amount of pages for one specific<br />
text document) and overall requirements, that are related to more than one result (e.g. use UML<br />
diagrams for designing a software system). Especially the latter are difficult to learn because they<br />
often contain implicit knowledge of a domain (as in the above example, which diagram types have to<br />
be used and how they interact).<br />
We take a generic approach in order to tackle both sides of the problem. While the participants work<br />
in a <strong>learning</strong> project and contribute result documents, they are committed to add further documents<br />
that explain these results. These documents can be arbitrary digital documents allowing to add<br />
information of any form. So other project members better understand how the existing results were<br />
developed. To manage the complexity of the project constraints, the teacher can choose to add<br />
documents that influence the later development of the result documents. These constraint documents<br />
support the learners in constructing the overall results. With the introduction of supporting documents<br />
for expected and existing results of a project, it is possible to think of new <strong>learning</strong> processes that<br />
integrate these documents in different <strong>learning</strong> phases.<br />
The paper starts with an account of existing approaches that incorporate any kind of metadata into<br />
<strong>learning</strong> processes for reasons of supporting the learners in understanding the project’s results. Then<br />
an example for a <strong>learning</strong> project from the field of software programming is introduced. This will be the<br />
basis for defining the different document types and the way they are used in this concept. Afterwards<br />
some new approaches on how to organize the <strong>learning</strong> process will be presented. The paper ends<br />
225
Gert Faustmann<br />
with an evaluation of the described concepts by presenting a <strong>learning</strong> platform implementing the<br />
described conceptual aspects. Finally, an outlook will give some ideas on evaluating the concept and<br />
on future research.<br />
2. Related work<br />
Today’s <strong>learning</strong> theory favors a constructivistic <strong>learning</strong> approach where <strong>learning</strong> is an active<br />
process and the learner uses her/his knowledge to construct and solve a problem. The teacher acts<br />
as coach, supporting the learners in tackling the complex problem or situation (Baumgartner 2003).<br />
(Gilbert & Driscoll 2002) define the principles of a constructivistic <strong>learning</strong> approach as<br />
A group objective,<br />
Cooperating groups with interactions,<br />
An individual selection of materials and decisions,<br />
The use of integrated tools to store results and to communicate.<br />
Especially wikis as tools play an important role in <strong>learning</strong> environments to support the distributed<br />
construction of documents (Augar et al. 2006) and they are used in different domains as <strong>learning</strong><br />
environments (e.g. journalism (Ma & Yuen 2008)). Moreover, they were investigated in terms of their<br />
ability to support communities (Johnson & Bartolino 2009) and their impact on social and cognitive<br />
processes (Cress & Kimmerle 2008). There is further research about extending wiki systems for<br />
specific <strong>learning</strong> scenarios (Larusson & Alterman 2009) (Pusey & Meiselwitz 2009) (Twu 2009). One<br />
can claim that wikis are a constitutive component of today’s <strong>learning</strong> environments. But there are also<br />
problems when wikis are used in a <strong>learning</strong> environment. (Karasavvidis 2010) reports problems like<br />
the students’ resistance to use a wiki or overwhelming task requirements.<br />
One can find different approaches to implement collaborative <strong>learning</strong> with computer support. Some<br />
use the idea of arguments combined with a <strong>learning</strong> process. The visualization of arguments and of<br />
their overall structure is also investigated in the cscl literature. There are research projects focusing at<br />
computer-supported <strong>learning</strong> where the use of arguments is investigated (Cho & Jonassen 2002)<br />
(González et al. 2007)(Jonassen & Kim 2009). While arguments in these projects are observed as<br />
part of an argumentation process, (Scheuer et al. 2010) gives an overview of the project supporting<br />
use of arguments. The main questions addressed are the visualization of argument structures,<br />
interaction between project participants, the combination of arguments with ontologies and the<br />
analyzation of arguments for automatic feedback. Visualization of the constructed knowledge as well<br />
as of the arguments leading to this knowledge is described in other research (Muller Mirza et al.<br />
2007)(Perit Çakır et al. 2009)(Schwarz & De Groot 2007). A software tool for visualizing argument<br />
structures is described in (Sbarski et al. 2008).<br />
Another interesting approach can be found in the software system SenseMaker (Bell 1997) (Bell &<br />
Linn 2000). SenseMakers gathers web resources to build arguments for statements. The document<br />
structure is a container that holds arguments. These arguments then may themselves contain further<br />
arguments. A knowledge repository based on a semantic web ontology and a resource description<br />
framework is built in (Lubliner et al. 2009). The repository contains course materials. There was a<br />
clear increase in exam result points compared to the group not using the repository. This shows that<br />
knowledge repositories with a internal semantic structure (e.g. hierarchies, ontologies) assist <strong>learning</strong>.<br />
3. Modelling decisions and constraints<br />
3.1 An example of a <strong>learning</strong> project<br />
We will use an example of a <strong>learning</strong> project to explain the basic ideas of arguments, constraints and<br />
digital documents in the following section. The example is an assignment in software programming.<br />
Aside from the program code, some design documents are requested as results from the project<br />
participants. The task is to design and program a flexible menu system which can be used by other<br />
programmers within their projects. The menu system should work on the command line interface, so it<br />
is just a character-based interface to the user. The task document defines<br />
How a menu works (a menu option leads to a sub menu or contains an action that is carried out if<br />
the user chooses this option),<br />
That additional menu options can be integrated without changing the original source code,<br />
226
Gert Faustmann<br />
That results must comprise a class diagram, a sequence diagram and the program code.<br />
Figure 1 shows the task document and possible results. The result documents are related to the subtasks<br />
in the overall task document. In most cases, this relation of the results to the task is clear to the<br />
learners.<br />
Figure 1: Example of a project task with expected results<br />
Usually this is the perspective a student has on a <strong>learning</strong> project of this kind. But, as one can see in<br />
the example, it is not clear why the design is done this way. Questions are: why are there different<br />
classes for the menu options, why is there an interface class for a menu option and why does the<br />
“Menu” class use the getPath() method (as it is modelled in the sequence diagram) to display the<br />
menu options. From a detailed analysis of the program code, these questions could be answered. But<br />
the design principles behind these documents remain unclear.<br />
3.2 Arguments explaining decisions for project results<br />
A first step in clarifying the ideas behind the development of the specific results in a project is to give<br />
explanations for them. We call these explanations ‘arguments’ because often a project participant has<br />
to make one or more decisions when developing a project result. But because arguments support<br />
other project members in understanding results, they need not be formal logical statements. They can<br />
be of very different information types such as lecture slides or parts of a textbook. One can also think<br />
of multimedia documents explaining something, e.g. a procedure with a movie. Figure 2 shows the<br />
basic approach of combining tasks with results by using argument documents.<br />
As depicted in figure 2, only parts of the documents are used for building this “argument network”.<br />
This originates from the fact that often only small parts of a document are important as an explanation<br />
The same applies to task and result documents: only parts of the task document may be solved by<br />
specific result documents (“sub tasks”), and only parts of a result document may need an explanation.<br />
Figure 3 shows the relationship between tasks, results and arguments. Here, they all are parts of<br />
documents, that each belong to exactly one document. The implementation of document parts is<br />
described in detail in (Faustmann 2011).<br />
The class diagram also shows that every result belongs to a task, but must also be associated with at<br />
least one argument. The idea is to force the project participants to provide corresponding arguments<br />
for new results. From the other side an argument only belongs to one result. Different arguments may<br />
227
Gert Faustmann<br />
come from the same source document (and may nearly be the same), but they must be different as<br />
an explanation for different results.<br />
Figure 2: Arguments combining tasks and results<br />
Figure 3: UML class diagram for the integration of document artefacts as arguments<br />
When applied to our example we need to identify a document (or part of a bigger document) that can<br />
serve as an argument for one of the shown results. Let us take a look at the combination of the sub<br />
task “design a class diagram” and the resulting document being a class diagram. Here an argument<br />
document is used to explain the specific design decision implemented in the class diagram. A strategy<br />
pattern allows encapsulating an algorithm with an object. In this way, an algorithm can be easily<br />
exchanged. The introduced argument document is a slide explaining the strategy pattern with a class<br />
diagram and with an additional explanation. In figure 4 it is easy to see, that the structure of both<br />
diagrams is nearly the same. So it is obvious that the MenuItem class encapsulates its action. The<br />
menu system can now manage its options in different objects with each having its specific action.<br />
228
Gert Faustmann<br />
Figure 4: Argument document introducing the strategy pattern<br />
3.3 Constraints revealing requirements on project results<br />
While arguments explain results or show why results are formed the way they are, it is still difficult to<br />
understand the context of project results. When looking at our example, we have two perspectives on<br />
the design documents: On the one hand, every document has to be consistent in itself. This means<br />
that the correct symbols are used, the symbols are correctly assembled and so on. On the other hand,<br />
the diagrams also have to be consistent among each other. One can speak of consistency in relation<br />
to the whole set of diagrams (and also all other documents that can be a result). For example, if a<br />
class is described in the class diagram it should also appear in the source code. While there is no<br />
sense in modelling a class without implementing it, there are also situations that are definitely<br />
erroneous. If an object of a certain class type appears in a sequence diagram, this class must also<br />
appear in the class diagram.<br />
Figure 5 shows the integration of constraint documents and corresponding annotations into the former<br />
system of tasks, results and arguments. Constraints as well as annotations are part of documents. If<br />
an annotation is only text-based the whole ascii document can be the source for this annotation (on<br />
the other hand it is then also possible that e.g. a handwritten document can be used as an<br />
annotation). Constraints belong to results. Usually constraints exist before concrete results are<br />
created. Then we can think of results as containers that have to be filled while the project is<br />
advancing. A constraint has to be related to at least one result document. The function of an<br />
annotation is to match the related constraint document to the special situation of the <strong>learning</strong> project.<br />
Constraints can be very general and so match all situations (“a class in a sequence diagram must<br />
appear in the class diagram”). But they can also be more specific (e.g. “program components against<br />
an interface”) with an annotation giving a hint how to instantiate this constraint (e.g. “use an interface<br />
for the menu actions”).<br />
Again we take a look at the project example now, with the goal to introduce constraint and annotation<br />
documents. Let us assume that there was an initial constraint document that asked for implementing<br />
independent classes representing menu options on the different levels of a menu system. This indeed<br />
is a constraint that affects the class diagram as well as the expected sequence diagram. The menu<br />
system has to test every class-related object that is a menu option, if it is currently visible.<br />
Additionally, the constraint demands in the example that these classes may be loaded while the whole<br />
system is already running. In java there is a reflection mechanism that gives exactly this kind of<br />
functionality. So the annotation to this second (sub-)constraint is a document describing the java<br />
reflection mechanism. If this constraint is bound to the program code, it is easier to understand how<br />
the java code works.<br />
229
Gert Faustmann<br />
Figure 5: UML class diagram for the integration of annotated constraint documents<br />
Figure 6: Constraint demands independent classes, annotation explains loading classes<br />
4. Learning process<br />
The main goal for a <strong>learning</strong> project is to create a set of result documents (or one document that<br />
contains consistent document parts, but according to our approach we stay in the perspective of<br />
multiple documents). Therefore, the main process is to provide new result documents and enrich them<br />
with argument documents thereby making decisions and the results better understandable. But the<br />
explicit modelling of supporting information allows to change the <strong>learning</strong> process in its perspective.<br />
Thus is becomes possible to ask the learners to find relationships between results or give further<br />
context information. Both approaches are presented in the following:<br />
4.1 Result-based project work<br />
Incorporating new results will always be the main work in a project asking for constructive work. The<br />
lerners will work at solutions that are influenced by the project’s assignment and additional<br />
230
Gert Faustmann<br />
constraining documents. One can think of three possible alternatives for providing new results (see<br />
also figure 7):<br />
The learner provides a new result that may be influenced by an already existing constraint<br />
document. She/he is obliged to relate this new result to an argument.<br />
The learner provides a new result, but uses a predefined argument out of a set of argument<br />
documents that was prepared by the teacher. The learner seeks for an applicable argument and<br />
integrates this argument.<br />
Finally, the learner can provide a result where an argument already exists and gives support how<br />
the result ist to be constructed. This would be the highest level of support a teacher can give by<br />
preparing documents.<br />
Here we see that the teacher can provide very different kinds of support in solving a problem. Not only<br />
can she/he provide varying amounts of supporting documents and hints about where to use them, the<br />
teacher may also provide further documents while the project is already advancing. So it is possible to<br />
control not only the general advancement of the project but also the technical elaboration of the given<br />
task (e.g. give a hint to use another design pattern to plan the software system).<br />
Figure 7: Options to provide new results<br />
Again we refer back to our example. According to the three options, possible activities and example<br />
documents are defined here:<br />
The constraint forcing the learner to think about loading classes while the program is running will<br />
surely point to parts of the program code. Perhaps a sample solution for the reflection mechanism<br />
could demonstrate the idea of the project specific program code.<br />
If sample solutions (also for the reflection mechanism) were prepared by the teacher, the learner<br />
can seek for them and use one as an argument for her/his program code.<br />
If there is more than one document for the program code (think of different packages or modules)<br />
then a sample solution could point to a specific module or package.<br />
4.2 Learning context information<br />
Additionally, there are further options for providing documents in the <strong>learning</strong> process (see also figure<br />
8): The learner can provide additional arguments, constraints and annotations to already existing<br />
result documents in the project.<br />
This could mean the following activities in our example:<br />
The learner provides an additional sample solution to the program code already being in the<br />
system.<br />
The learner provides a specific constraint that was asked for by the teacher. The learner knows<br />
that there is a constraint sought-after for specific results.<br />
231
Gert Faustmann<br />
The learner her/himself formulates a constraint, e.g. for the source code in the system. This could<br />
be, for example, a hint how to use the different classes of the menu system in another software<br />
application.<br />
Finally, it may be necessary to adapt a constraint to a specific <strong>learning</strong> situation by providing an<br />
annotation.<br />
Figure 8: Options to provide documents different from results<br />
There are other alternatives, but they all follow the same structure: the teacher may ask for a new<br />
document or the student may her/himself provide a new document without being asked to. When<br />
asked for a document, the teacher can provide a set of possible documents or encourage the student<br />
to search for appropriate documents on his/her own.<br />
5. Arguments and constraints in the WeCoLAr system<br />
Implementing the described conceptual approach is difficult because one of the core requirements is<br />
the use of arbitrary digital documents for results and supporting documents. Additionally, these<br />
documents have to be visualized in the system (also as links with the possibility to use a third party<br />
application for opening) and they must be loadable in a native document format so that the learner<br />
can work on them. If a document is repeatedly used in a <strong>learning</strong> project, it should nevertheless not<br />
be multiply stored in the system.<br />
The web-based <strong>learning</strong> platform WeCoLAr (Web-based Collaborative Learning by Arguments)<br />
implements these requirements by using markings in picture documents (see figure 9). All documents<br />
are stored in their native document format (e.g. pdf or html), as well as in a universal picture format<br />
(here jpg). The graphical user interface only shows the pictures of the documents. But pictures of<br />
bigger documents (e.g. pdf files with many pages) would be too bulky to be handled and viewed on a<br />
graphical desktop. Furthermore, only parts of the documents are needed for referencing. So<br />
WeCoLAr uses markings defining a specific area in a document. A marking is stored when a learner<br />
loads a new document into WeCoLAr. This marking can be defined interactively and is specific for<br />
every context.<br />
232
Figure 9: Implementing document parts<br />
Gert Faustmann<br />
When handling these “documents” on the graphical desktop the quality of the picture is satisfying.<br />
They can be zoomed in by using the browser’s zoom functionality. Also, it is possible to change to a<br />
single document view. WeCoLAr also has the functionality to filter the viewed documents on the<br />
desktop, so for instance, one can choose to see only those documents that belong to a certain sub<br />
task (all design documents for the menu system). Figure 10 exemplifies the user interface of<br />
WeCoLAr. In the viewed modus it is possible to drag documents within the argument net and change<br />
the structure of the net.<br />
Figure 10: WeCoLAr user interface<br />
6. Outlook<br />
WeCoLAr is now being tested in real <strong>learning</strong> processes at university level. The example used in this<br />
paper is one of the domains where the WeCoLAr system is used (i.e. introduction to programming,<br />
object oriented programming). But the system will also be put into effect in other domains that are<br />
affected by business processes. One application field will be courses in Macroeconomics.<br />
Future work on the concept of arguments and constraints in <strong>learning</strong> processes will focus on the<br />
control of the process. (Dillenbourg & Tchounikine 2007) describe macro scripts that implement<br />
didactic concepts. This would match the idea of offering different construction activities in WeCoLAr.<br />
Of special interest could be the question of flexibility in controlled process systems. For now WeCoLAr<br />
is controlled by the learners themselves. A <strong>learning</strong> script defines some constraints for this process<br />
233
Gert Faustmann<br />
(i.e. who has to provide what douments, timeline of documents etc.). In (Konig & Paramythis 2010) a<br />
control mechanism is defined that can be adapted according to the situation at hand.<br />
Acknowledgements<br />
The project WeCoLAr is funded by the European Social Fund (ESF 2009000038) as a sub project<br />
within „E(r)lernen: Kompetenzvermittlung zum Einsatz neuer Medien“. I like to thank Liangliang Gu for<br />
his contribution to the WeCoLAr system.<br />
References<br />
Augar, N., Raitman, R. & Zhou, W. (2006) Wikis: Collaborative Virtual Learning Environments. In The<br />
International Handbook of Virtual Learning Environments. pp. 1251-1269..<br />
Baumgartner, P. (2003) Lerntheorien und Lernwerkzeuge. ÖZB, p.5.<br />
Bell, P. (1997) Using argument representations to make thinking visible for individuals and groups. UNIVERSITY<br />
OF TORONTO, p.10-19.<br />
Bell, P. & Linn, M. (2000) Scientific arguments as <strong>learning</strong> artifacts: Designing for <strong>learning</strong> from the web with KIE.<br />
International Journal of Science Education, 22(8), pp.797–817.<br />
Cho, K.-L. & Jonassen, D. (2002) The effects of argumentation scaffolds on argumentation and problem solving.<br />
Educational Technology Research and Development, 50(3), pp.5-22.<br />
Cress, U. & Kimmerle, J. (2008) A systemic and cognitive view on collaborative knowledge building with wikis.<br />
International Journal of Computer-Supported Collaborative Learning, 3(2), pp.105-122.<br />
Dillenbourg, P. & Tchounikine, P. (2007) Flexibility in macro-scripts for computer-supported collaborative<br />
<strong>learning</strong>. Journal of Computer Assisted Learning, 23(1), pp.1–13.<br />
Doppelt, Y. (2003) Implementation and assessment of project-based <strong>learning</strong> in a flexible environment.<br />
International Journal of Technology and Design Education, 13(3), pp.255–272.<br />
Faustmann, G. (2011) PROJECT-BASED COLLABORATIVE LEARNING WITH ARGUMENTS - Flexible<br />
Integration of Digital Resources. In 3rd International Conference on Computer-Supported Education.<br />
Noordwijkerhout, The Netherlands.<br />
Gilbert, N. & Driscoll, M. (2002) Collaborative knowledge building: A case study. Educational Technology<br />
Research and Development, 50(1), pp.59-79.<br />
González, M. et al. (2007) Modelling Shared Knowledge and Shared Knowledge Awareness in CSCL Scenarios<br />
Through Automated Argumentation Systems. In Groupware: Design, Implementation, and Use. pp. 207-<br />
222.<br />
Johnson, K. & Bartolino, J. (2009) Creating Community through the Use of a Class Wiki. In Online Communities<br />
and Social Computing. pp. 471-478.<br />
Jonassen, D. & Kim, B. (2009) Arguing to learn and <strong>learning</strong> to argue: design justifications and guidelines.<br />
Educational Technology Research and Development.<br />
Karasavvidis, I. (2010) Wiki uses in higher education: exploring barriers to successful implementation. Interactive<br />
Learning Environments, 18(3), pp.219-231.<br />
Konig, F. & Paramythis, A. (2010) Closing the Circle: IMS LD Extensions for Advanced Adaptive Collaboration<br />
Support. In 2010 International Conference on Intelligent Networking and Collaborative Systems. 2010 2nd<br />
International Conference on Intelligent Networking and Collaborative Systems (INCoS). Thessaloniki,<br />
Greece, pp. 421-426.<br />
Larusson, J. & Alterman, R. (2009) Wikis to support the “collaborative” part of collaborative <strong>learning</strong>. International<br />
Journal of Computer-Supported Collaborative Learning, 4(4), pp.371-402.<br />
Lubliner, D., Widmeyer, G. & Deek, F.P. (2009) Collaborative <strong>learning</strong> utilizing a domain-based shared data<br />
repository to enhance <strong>learning</strong> outcomes. Interactive Learning Environments, 17(4), pp.351-366.<br />
Ma, W. & Yuen, A. (2008) A Qualitative Analysis on Collaborative Learning Experience of Student Journalists<br />
Using Wiki. In Hybrid Learning and Education. pp. 103-114.<br />
Muller Mirza, N. et al. (2007) Using graphical tools in a phased activity for enhancing dialogical skills: An example<br />
with Digalo. International Journal of Computer-Supported Collaborative Learning, 2(2), pp.247-272.<br />
Perit Çakır, M., Zemel, A. & Stahl, G. (2009) The joint organization of interaction within a multimodal CSCL<br />
medium. International Journal of Computer-Supported Collaborative Learning, 4(2), pp.115-149.<br />
Pusey, P. & Meiselwitz, G. (2009) Heuristics for Implementation of Wiki Technology in Higher Education<br />
Learning. In Online Communities and Social Computing. pp. 507-514.<br />
Sbarski, P. et al. (2008) Visualizing Argument Structure. In Advances in Visual Computing. pp. 129-138.<br />
Scheuer, O. et al. (2010) Computer-supported argumentation: A review of the state of the art. International<br />
Journal of Computer-Supported Collaborative Learning, 5(1), pp.43-102.<br />
Schwarz, B. & De Groot, R. (2007) Argumentation in a changing world. International Journal of Computer-<br />
Supported Collaborative Learning, 2(2), pp.297-313.<br />
Twu, H.-L. (2009) Effective Wiki Strategies to Support High-Context Culture Learners. TechTrends, 53(5), pp.16-<br />
22.<br />
234
A Strategy for the Inductive Generation of Learning Objects<br />
in Low-Tech Contexts<br />
Ana Mª Fernández-Pampillón, Elena Domínguez, José Mª Lahoz, Dolores<br />
Romero, Isabel de Armas, Susana Palmaz and Jorge Arús<br />
Universidad Complutense de Madrid, Spain<br />
apampi@filol.ucm.es<br />
elenadominguez@filol.ucm.es<br />
josemarialahoz@yahoo.com<br />
doloresromerolopez@gmail.com<br />
iarmas@buc.ucm.es<br />
spalmaz@filol.ucm.es<br />
jarus@filol.ucm.es<br />
Abstract: This article proposes a strategy to create inductive and incremental LOs in tertiary education low-tech<br />
contexts such as the field of Humanities. We take a teaching context to be low-tech when the teachers are not<br />
computer specialists and also have poor IT support. This is the case with many university schools in Spain, such<br />
as the Faculty of Philology at the Universidad Complutense de Madrid (UCM), where the work reported here has<br />
been carried out. In a low-tech context, the use of Educational ICT is considerably expensive, if not impossible.<br />
However, university faculty feels more and more compelled to use ICT tools in their teaching, e.g. e-<strong>learning</strong><br />
platforms. In the process of adaptation and integration of ICT into teaching, a basic issue is the transformation of<br />
good teaching materials into digital format in the simplest and most cost-effective way. Good results can be<br />
obtained by applying the Learning Object (LO) model when creating digital teaching materials, but the models<br />
and tools to build LOs are not easy to understand or use without prior knowledge, and their application requires<br />
computer support beyond what is usually available in low-tech contexts. The results of the research carried out<br />
during the last decade thanks to a number of research grants show that, by using the appropriate models, tools<br />
and strategies, it is possible to bring ICT to IT-illiterate teachers and get excellent results regarding the<br />
educational use of ICT by these teachers. In the present work a solution has been designed and tested to<br />
disseminate teaching materials through the generation of LO collections in low-tech contexts. This solution is<br />
based on applying a novel strategy to inductively construct LOs from the original materials. This is done by<br />
applying a model and a repository of LOs, developed and tested in previous works, and a new quality model for<br />
LOs collaboratively developed with the beneficiaries of this strategy, i.e. IT-illiterate Humanities teachers. The<br />
authors of this paper are part of a mixed team of IT-specialist and IT-illiterate Humanities teachers who<br />
developed and applied this strategy. Our paper aims to provide a real and complete picture of the problem and<br />
the solution developed.<br />
Keywords: <strong>learning</strong> objects, repositories, authoring, digital <strong>learning</strong> resources, educational ICT<br />
1. Introduction<br />
Suppose you are a University Humanities teacher. In the current teaching context, class notes and<br />
textbooks are no longer the only necessary materials. Audiovisual media, even just a simple<br />
slideshow, are de rigueur. The increasing use of e-<strong>learning</strong> platforms requires that teachers convert<br />
all their precious analogical resources to a digital format. Furthermore, they need to enlarge their<br />
collections with new materials, as those virtual platforms are intended to provide extra <strong>learning</strong><br />
resources beyond the classroom. However, those materials are not usually readily available, and<br />
instructors have to design their own material, despite their limited knowledge of the required<br />
technology and the lack of technical support.<br />
The development of content-quality materials involves a clear commitment in terms of research, and,<br />
more often than not, the process also requires a considerable investment in time and effort to learn to<br />
handle the tools and computer programs used for the creation of those digital objects.<br />
As a collection grows, a further concern is to find an efficient way to store and manage it. Whether it is<br />
for the author’s own use or with the purpose of sharing materials with others, digital repositories of<br />
Learning Objects (henceforth LOs) are proposed as a solution (Neven and Duval 2002). However,<br />
new problems arise such as the need to comply with the quality requirements of the repository, mostly<br />
directed toward ensuring reusability (Bennett et al. 2006). This proves especially difficult when the<br />
mentioned requirements are expressed in terms that are not intelligible to IT-illiterate users. On the<br />
other hand, some technical knowledge is necessary to understand the quality, metadata and content<br />
models.<br />
235
Ana Mª Fernández-Pampillón et al.<br />
In this paper we present an experiment conducted at the UCM School of Philology. A Project for<br />
Educational Innovation and Improvement of Quality Teaching was undertaken for the collaborative<br />
creation of a Learning Object Repository (LOR) targeted at our faculty members. The paper is<br />
structured as follows. Section 2 presents the strategy, which is based on (2.1) an open Learning<br />
Object Model, (2.2) an accessible quality model for Learning Objects (LOs), (2.3) an easy-to-use<br />
LOR, and (2.4) an inductive methodology to create LO collections. Following this, section 3 sketches<br />
the experimentation and results. Finally, section 4 presents the main conclusions and outlines future<br />
lines of work.<br />
2. Supporting university IT-illiterate teachers to create Learning Objects<br />
The idea of improving the profitability of digital materials appears in the field of Educational<br />
Technology and the related industry with the notion of LO reuse (Wiley 2000). The goal is to increase<br />
profitability by creating educational materials as pieces (LOs) that can be assembled together<br />
(Gibbons et al. 2000). It can also be applied to the case of university teachers who are, at the same<br />
time, developers and users of their educative material. However, the theoretical and technological<br />
knowledge to build LOs is beyond the reach of many faculty members. There are two main ways in<br />
which researchers tackle the quest for the efficient creation and use of LOs by instructors: one of<br />
them is the development of guidelines for the creation of LOs (e.g. Smith 2004; Jesukiewicz 2009);<br />
the other is the creation of devices such as templates and toolkits to guide instructors in the design of<br />
new LOs (e.g. Adorni et al. 2008; Watson 2010a). In spite of the unquestionable help that these<br />
resources represent, in low tech academic contexts there is a main limitation: IT-illiterate teachers can<br />
build LOs only with sufficient staff and infrastructure support. Unfortunately, at the moment this is not<br />
the case in some Spanish Universities and Schools (Uceda and Barro 2010). Therefore, with the aim<br />
of combining the effectiveness of the LO model with the needs of producing specialized digital<br />
academic materials, we have developed and implemented a new, straightforward strategy for the<br />
creation of LOs with minimal or no IT support. It is based on:<br />
Using already-tested existing materials, digital or not, rather than creating new ones.<br />
Applying an inductive methodology for the incremental construction of LOs from these materials<br />
based on applying: (a) an open LO model; (b) an accessible quality model; (c) an easy-to-use LO<br />
repository.<br />
Collaboratively developing the metadata and LO evaluation models best suited to philology.<br />
Providing basic ICT support, on site and long-term, by involving the Philology School’s IT staff in<br />
this kind of projects.<br />
This strategy has enabled the creation of two specialized LO collections: a historical collection with<br />
old multimedia language teaching material , and a recent collection<br />
of philological LOs: (Figure1).<br />
Figure 1: Language labs’ audiovisual LO historical collection<br />
236
2.1 An open Learning Object model<br />
Ana Mª Fernández-Pampillón et al.<br />
To facilitate the construction of reusable LOs, some international metadata standards (Duval 2004) as<br />
well as multiple content models (Verbert and Duval 2008) have been developed. However, the<br />
theoretical and technical knowledge necessary to understand and use LOs is beyond the reach of<br />
many teachers, primarily because: (1) specifications are difficult to understand and use without<br />
assistance, and also, it is unclear which one to use among the multiple content models and metadata<br />
schema, (2) specifications of standards are too general to concisely describe the semantic richness of<br />
specialized LO collections, such as Linguistics, Literature, Archeology, etc. (Boot 2005: 76,77), in<br />
spite of the extension mechanism the standards provide. Finally, (3) the LO support technology –<br />
authoring tools, tools for building LORs and LORs – are unknown and inaccessible to IT-illiterate<br />
teachers.<br />
To deal with the first two problems, an open and flexible LO model has been developed: the Virtual<br />
Object Model (VOM) (Sierra and Fernández-Valmayor 2006). This model allows users (teachers and<br />
researchers) to: (1) easily define specialized metadata and vocabularies as well as using metadata<br />
standards; and (2) freely build LO content by using any resource: other LOs or even resources from<br />
other LOs. From a conceptual point of view, a LO has three parts (Figure 2a):<br />
Object Data. A set of specialized attribute-value pairs to describe the specific features of the<br />
object in its scientific or humanistic domain.<br />
Object Metadata. A set of attribute-value pairs to document and classify the LO. It can be taken<br />
from any standard proposals such as LOM or Dublin Core, or can be user-defined.<br />
Content. A set of resources (digital files) with an individual set of attribute-value pairs describing<br />
each of those resources. Resources and resource metadata are freely added by teachers.<br />
Resources can be: (a) own resources, (b) foreign resources, which are references to resources<br />
owned by other LOs; and (c) LO’s resources, which are references to other LOs. Foreign and LO<br />
resources allow the establishment of basic relationships between LOs and the construction of<br />
more complex LOs by adding simpler ones (Figure 2b).<br />
(a) (b)<br />
Data<br />
…….<br />
Metadata<br />
Resources<br />
…<br />
VO<br />
resource<br />
…<br />
Foreign<br />
resource<br />
Figure 2: (a) VO model; (b) VOs can be linked together by using Foreign and VO resources<br />
At this point, it is worth mentioning that the VOM is compatible with the standard IMS Content<br />
Packaging specification to package and distribute LOs (IMS 2003). This functionality is already<br />
implemented in the OdA Repository (see section 2.3). However, at this moment there is no support for<br />
the most general and recent distribution specification IMS Common Cartridge (IMS 2011).<br />
Using the VOM, teachers who have no knowledge of either LO models or authorizing tools can easily<br />
build LOs tailored to their needs (Figures 3, 4 and 5). However, from a design standpoint the model<br />
does not ensure content modularity. For this reason, we developed a tool for assessing the quality of<br />
materials which, applied as self-evaluation during the construction process, guides content design<br />
toward modularization, accessibility and interoperability.<br />
237<br />
…
Ana Mª Fernández-Pampillón et al.<br />
Figure 3: Modular content of a LO from the Philology collection<br />
Figure 4: Example of domain-specific metadata (Second Language Teaching)<br />
2.2 An accessible quality model for Learning Objects<br />
A LO Quality Model is useful not only as an evaluation tool itself but also to address the construction<br />
of LOs if applied during the construction of digital <strong>learning</strong> materials. This type of evaluation, called<br />
process evaluation can be used to formatively assess the planning, design, development and<br />
implementation of LOs (Williams 2000). Thus, if a quality evaluation tool is applied as part of the task<br />
of creating digital teaching materials in order to improve their quality, it can guide teachers in the<br />
creation of LOs. However, the quality evaluation models available for instructors (Williams 2000;<br />
Taveira and Azevedo 2010; Watson 2010b), are not always easy to understand or apply by the mostly<br />
238
Ana Mª Fernández-Pampillón et al.<br />
IT-illiterate Humanities teachers nowadays. Firstly, many educational software quality models focus<br />
on educational aspects at the expense of technical aspects (Vidal et al. 2010: 343). Secondly, these<br />
models are not user-friendly evaluation tools, convenient for instructors with no theoretical knowledge<br />
or expertise in the creation of LOs. Thirdly, there is a lack of consensus regarding the concept of<br />
quality applied to teaching materials (Dondi and Moretti 2007).<br />
Figure 5: General metadata in the IEEE-LOM standard<br />
To overcome this problem in low-tech contexts, we have developed an accessible Quality LO tool,<br />
called COdA (Fernández-Pampillón et al. 2011a). Under COdA criteria, a LO is good quality if it helps<br />
instructors and students to learn more and in less time, and if it is used frequently ― which ultimately<br />
means that it is reusable. To apply COdA, teachers have to fill in a questionnaire of ten quality criteria<br />
with the help of a short manual with instructions and examples. COdA is based on LORI (E-Learning<br />
Research and Assessment Network 2010), and educational content quality evaluation models<br />
developed in Spanish Universities, notably the UNED (Universidad Nacional de Educación a<br />
Distancia 2011), UCM Virtual Campus (UCM-OCW, 2011), HEDOAR (Morales et al. 2009) and the<br />
University of Murcia (DIGITUM 2011). It also includes basic content accessibility recommendations<br />
from W3C (Chisholm and Vanderheiden 1999) and the IMS Global Learning Consortium (IMS<br />
guidelines 2002).<br />
2.3 An easy-to-use Learning Object Repository for specialized LO collections<br />
To organize and storage the specialized collections of LOs, LORs have been used. A LOR stores<br />
both LOs’ content and metadata, either by storing them physically together or separately in an on-line<br />
database. Typically, A LOR uses fixed metadata scheme (usually IEEE-LOM or Dublin Core profiles)<br />
to support simple and advanced queries as well as browsing through the material by subject discipline<br />
(Neven and Duval 2002). However, LORs do not permit the custom and dynamic definition of domainspecific<br />
databases for the authoring of specialized LO collections or their searching according to their<br />
specific characteristics (Tzikopoulos et al. 2009). Furthermore, in Spain there is a lack of university<br />
LORs (REBIUN 2007). Therefore, teachers end up installing and maintaining their own educational<br />
and scientific collections of digital materials, which is far too demanding for the IT-illiterate teacher.<br />
OdA repository is an easy-to-use LOR for the management and definition of specialized LOs. OdA is<br />
a UCM software tool in whose development some of the authors of this paper participated. Its<br />
development was funded through national research projects and its purpose is to provide teachers,<br />
researches and students with a simple and flexible computer system to disseminate their educational<br />
and research materials (Sierra and Fernández-Valmayor 2006). Apart from the typical LOR’s basic<br />
functionalities, OdA also provides:<br />
239
Ana Mª Fernández-Pampillón et al.<br />
The possibility to dynamically define customized data, metadata and content for LO collections<br />
according to the knowledge domain and the specific didactic needs of users (Figure 6);<br />
Because of (1), an intuitive mechanism for domain experts (teachers and researches) for the<br />
location and selection of LOs based on their discipline’s specialized attributes and terminology.<br />
Hence, when exploring and searching LOs in OdA, students use language and conceptual<br />
domain classifications of each discipline, which helps to improve the <strong>learning</strong> of terminology and<br />
conceptual structure of that domain.<br />
The possibility to export LOs to an IMS Content Package to publish them in other e-<strong>learning</strong> tools<br />
such as Learning Management Systems (e.g. Moodle).<br />
Figure 6: Metadata schema management interface<br />
Finally, a minimal set of technical requirements is needed, concerning both IT equipment and<br />
skills. OdA can be installed in a personal computer with a self-installing version.<br />
Therefore, OdA is a software solution to easily create and manage specialized LO collections in<br />
tertiary education low-tech contexts (where teachers are not computer specialists and usually have<br />
poor IT support). It offers an alternative to international repositories, with greater functionality and<br />
interoperability but more complex to use, less flexible to precisely store and manage specialized LOs<br />
and requiring larger equipment and staff infrastructures.<br />
2.4 An inductive methodology to create Philology LO collections<br />
The incremental construction methodology of LO collections is based on the above-presented<br />
elements – an open LO model, an accessible quality tool and an easy-to-use LOR – to inductively and<br />
systematically generate tertiary LO collections in six stages:<br />
Creation/reuse. The teacher or team of teachers create or digitize their teaching materials<br />
according to the usual procedures.<br />
Inductive creation of metadata schema. Each faculty member involved in the collection defines a<br />
metadata record for their materials. A minimum set of metadata to describe and classify all the<br />
initial materials is selected from all the metadata records.<br />
Evaluation. Teachers evaluate the quality of the materials they have created, or are creating, by<br />
means of the COdA tool. Once evaluated, they can continue with this step, or stop if the<br />
evaluation has been satisfactory.<br />
Fine-tuning. If during the evaluation the material does not meet some criteria, it is modified so as<br />
to obtain a higher score. After fine-tuning the material, the teacher usually returns to step 3 to reevaluate<br />
the new version.<br />
240
Ana Mª Fernández-Pampillón et al.<br />
Storage and Documentation. Instructors store LOs in the repository. This action involves the use<br />
of the set of metadata created in step 2 to describe the LO collection. If new metadata are<br />
needed, teachers add them. Changes or deletion are agreed on by the different teachers working<br />
on the same collection.<br />
Teachers can edit or delete their LOs and metadata schema at any time. They can also include<br />
the result of the evaluation as one more LO resource.<br />
3. Experimentation and results<br />
The proposed strategy was tested in the UCM School of Philology because: (i) it is a low-tech context,<br />
(ii) we had institutional support, and (iii) there was a demand from faculty members for help to reliably<br />
create good digital teaching materials for use in the UCM’s Virtual Campus and the new digital<br />
language labs (Fernández-Pampillón et. al. 2011b).<br />
3.1 Participants<br />
Experiments were conducted with a group of 33 participants belonging to three different types: (i)<br />
faculty from nine different philology departments; (ii) the School IT Technicians and (iii) UCM eeducation<br />
researchers.<br />
Participants were organized into three working groups:<br />
1) Teaching Resources Team (TRT), with 20 faculty members in charge of testing the strategy.<br />
Only one of the team members had advanced IT skills; two of them had average IT skills; the<br />
other seventeen had low to intermediate IT skills. None of the members had LO expertise.<br />
2) Modelling Team (MT), responsible for developing the quality guidelines tool (COdA), and also<br />
in charge of inducing the general descriptive model of teaching resources prepared by the TRT.<br />
All group members had low to intermediate IT skills. None of the members has LO expertise.<br />
3) Technical Team (TT), responsible for providing the necessary support to use the LO repository,<br />
advising the other teams on technical matters and training the School’s technician in the<br />
maintenance of the repository (backup and possible failures). Only one of the members had LO<br />
expertise.<br />
3.2 Methodology<br />
The experiment was carried out in five phases:<br />
Phase 1. Preparation of a quality evaluation tool draft by the MT.<br />
Phase 2. a) Preparation of two samples of digital teaching materials and validation of quality<br />
guidelines by the TRT (COdA guidelines). By the end of this phase, the existing version of the COdA<br />
tool was considered to be final.<br />
Phase 3. Writing up the documentation and classification of materials by the MT, done inductively. By<br />
the end of this stage, the existing version of the basic documentation model (metamodel) was<br />
considered to be final.<br />
Phase 4. Building the LO collection in the OdA repository. By the end of this phase the preparation of<br />
the first LO collection of the Philology School was considered to be complete and ready to be open to<br />
the rest of the faculty.<br />
Phase 5. Strategy evaluation. The following results are collected and analyzed: a) the LO collection<br />
content models; b) the LO metamodel and surveys on the usability of OdA and COdA with feedback<br />
from the TRT team’s teachers. Conclusions are then drawn.<br />
3.3 Analysis of results<br />
LO collection Content Model. Concerning the content model, this has been observed to evolve as<br />
teachers gain experience in the use of the repository regardless of their initial IT skills. They are<br />
very simple models at first: a list of own-resources (Figure 3, above); more complex LOs are then<br />
created which include other related LOs forming small networks of LO resources (Figure 7).<br />
241
Ana Mª Fernández-Pampillón et al.<br />
Figure 7: Example of a LO representing a language course consisting of other LOs (book and audio)<br />
With respect to the quality evaluation tool (COdA), beyond our initial expectations teachers have<br />
reported that, through evaluation, they have managed to differentiate <strong>learning</strong> object from<br />
teaching material, which was not always clear before. The implementation of some minor<br />
changes ― i.e. the incorporation of a template to grade an exercise and thus give feedback to the<br />
student, or a clearer statement of the didactic goals in the LO to raise the punctuation of some<br />
criteria ― appeared to improve the reusability of the final product (Figure 8).<br />
However, not all the COdA criteria were easily applicable. For instance, the specification of<br />
accessibility has been reported at times not to be suitable for audiovisual objects targeted at auditory<br />
perception tasks (see Arús et al. 2011).<br />
Figure 8: COdA criteria evaluation results (from 1 to 5)<br />
Concerning the quality of the LOs (Figure 9), around 84% of the teaching materials that have<br />
been built according to the Quality Learning Object (QLO) building procedure can actually be<br />
considered QLOs (3 out of 35 do not meet the reusability criterion).<br />
242
Ana Mª Fernández-Pampillón et al.<br />
Figure 9: LO quality evaluation results (from 1 to 5)<br />
Finally, with respect to the use of the flexible LO metadata model, it has been observed that<br />
teachers prefer developing specialized classification vocabularies and attributes rather than using<br />
the more general ones provided by the standards. Furthermore, data and metadata schema have<br />
been dynamically changed as the collection grows.<br />
4. Conclusions and future work<br />
This work is a contribution to the efficient creation of good digital teaching resources in tertiary<br />
education by IT-illiterate or semi-illiterate teachers with few IT and technical staff resources. It is a<br />
strategy for the inductive construction of LOs which has been created and tested in collaboration with<br />
IT-illiterate Humanities teachers.<br />
In the generation of our LOR, faculty members were not given any predetermined content model or<br />
metamodel. Instead, they devised their own by means of an inductive method to obtain the minimal<br />
most specific data and metadata scheme that suited the needs of all the participants in the project. By<br />
so doing, we ensured that every scheme element was really necessary and that it could be<br />
understood by end-users. Furthermore, this methodology implies that the LOs and the models are<br />
scalable and can be expanded to suit future needs.<br />
As seen, a quality-assessment tool was also designed to outline certain criteria for teachers to selfevaluate<br />
their LOs and thus enhance their quality and, ultimately, their usefulness. Since the wording<br />
comes from the instructors themselves, the usability of the tool is guaranteed. Interestingly enough,<br />
the need of self-assessment fosters the reflection on one’s own LOs, which not only helps to improve<br />
the objects but also facilitates the intuition of the mere difference between a LO and a more traditional<br />
teaching material, which many instructors did not grasp at the beginning. It encourages the habit of<br />
sharing teaching materials, something not very frequent among instructors in Spanish universities. In<br />
this regard, we suggest that the institutions and committees working on the quality of education<br />
consider and value quality self-evaluation as a further indication of quality, very economical and with<br />
an important positive effect on the quality of teaching.<br />
The LO-building methodology presented allows a high degree of flexibility, which makes it especially<br />
well suited for supporting author-driven, incremental and open approaches to LO production. It is<br />
particularly appropriate in a university environment where teachers are responsible for the creation of<br />
their specialized teaching material with little or no IT support. Indeed, in these scenarios the first<br />
stages of the LO building process were devoted to produce very simple LO counterparts of the preexisting<br />
<strong>learning</strong> materials. Later stages produced higher-order LOs of more abstract nature (e.g.<br />
foreign language courses containing texts, audios, presentation and exercises related to different<br />
aspects of the <strong>learning</strong> process). During the successive stages the metadata model was also<br />
extended, fine-tuned and corrected to solve inconsistencies and redundancies.<br />
243
Ana Mª Fernández-Pampillón et al.<br />
Compared to other approaches, our proposal provides a solution accessible to any instructor or team<br />
of instructors with limited computer skills as well as human and technological support. The<br />
experimental results are satisfactory to the extent that 1) the feasibility of the procedure has been<br />
proven by means of a realistic project to build a LO collection in a non-technological field such as that<br />
of Philology; and 2) the procedure has succeeded in improving the quality of teaching materials in<br />
both the pedagogical and technological aspects.<br />
It is not possible, however, to ensure that by applying this approach 100% of the materials in a<br />
collection created by a group of teachers will be LOs, since, as shown by the first experimental<br />
results, the success (84% in our case) depends on 1) the degree of involvement of instructors in<br />
improving the quality of their materials; 2) having some sort of technical support so as to know how to<br />
apply IT quality criteria such as accessibility and interoperability.<br />
The next step in this research will be the study of the use (and reuse) of LO collections in the new<br />
electronic <strong>learning</strong> environments in non-technological fields of knowledge. In this sense, we will test<br />
the integration of the LO repository OdA into the UCM Virtual Campus with the aim of using it in virtual<br />
sessions, in the classroom or in the language lab. If IT-illiterate faculty incorporate LOs both into their<br />
classroom and virtual teaching in a natural way, and if the LOR keeps growing by the addition of new<br />
LO collections, then the LO creation strategy will have proven its usefulness in the creation, use and<br />
dissemination of digital research and teaching materials in non-technological domains.<br />
Acknowledgments<br />
The research presented here has been funded by the Universidad Complutense de Madrid through<br />
the Proyecto de Innovación Educativa y Calidad de la Docencia 268-2010 UCM; and by the Spanish<br />
Ministry of Science and Innovation through the following two research projects: Arquitecturas<br />
Avanzadas en Campus Virtuales (TIN2009-14317-C03-01/TSI), and Un Enfoque Generativo para la<br />
Construcción de Herramientas de Producción y Despliegue de Objetos Educativos en el Campus<br />
Virtual (TIN2010-21288-C02-01).<br />
References<br />
Arús, J., Fernández-Pampillón, A. M., Lahoz, J. M., Domínguez, E. and De Armas, A. I. (2011) “Learning Object<br />
Management for IT-Illiterate Instructors”, EDULEARN10 Proceedings, [online],<br />
http://library.iated.org/view/ARUSHITA2011LEA.<br />
Adorni, G., Brondo, D. and Coccoli, M. (2008) "Design and Implementation of a User Friendly Environment for<br />
Learning Objects Creation”, IFIP 20 th World Computer Congress, IFIP TC 3 ED-L2L Conference, Milano,<br />
[online], http://www.springerlink.com/content/a8763767281k2220/fulltext.pdf.<br />
Boot, E. (2005) “Novice and Experienced Instructional Software Developers: Effects on Materials Created with<br />
Instructional Software Templates”, Building-block Solutions for Developing Instructional Software, PhD,<br />
Open Universiteit Nederland Heerlen, [online],<br />
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134.6936&rep=rep1&type=pdf.<br />
Chisholm, W.; Vanderheiden, G. (1999) Tabla de Puntos de Verificación para las Pautas de Accesibilidad al<br />
Contenido en la Web 1.0 [Spanish version]. W3C, [online],<br />
http://www.discapnet.es/web_accesible/wcag10/full-checklist.html.<br />
DIGITUM (2011) Depósito Digital Institucional de la Universidad de Murcia, [online], http://digitum.um.es/xmlui/.<br />
Dondi, C. and Moretti, M. (2007) ELearning Quality in European Universities. UNIQUe, [online],<br />
http://unique.europace.org/pdf/WP1-report-v5_FINAL.pdf.<br />
Dublin Core Education Application Profile (2007) Working Draft of v0.4, [online],<br />
https://docs.google.com/Doc?id=dn8z3gs_38cgwkvv.<br />
Duval, E. (2004) “Learning Technology Standardization: Making Sense of it All”, Vol 1, No. 1, Computer Science<br />
and Information Systems, [online], http://www.comsis.org/pdf.php?id=nnn-1102.<br />
E-Learning Research and Assessment Network (2010) “About LORI”, [online],<br />
http://www.elera.net/eLera/Home/About%20%20LORI/.<br />
Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., Cazorla Vivas, M. C., De Armas Ranero, I. and<br />
López-Varela Azcárate, A. (2011a) “Herramienta para la revisión de la calidad de Objetos de Aprendizaje<br />
universitarios (COdA): Guía del Usuario. v.1.0. Documento de trabajo”, [on-line],<br />
http://eprints.ucm.es/12533/.<br />
Fernández-Pampillón, A. Sevilla, J. and Poves, A. (2011b) “Conclusiones finales y acciones futuras”, El<br />
Laboratorio de Idiomas y la Enseñanza-Aprendizaje de Lenguas, Editorial Complutense, Madrid.<br />
Gibbons, A. S.; Nelson, J. and Richards, R. (2000) “The Nature and Origin of Instructional Objects”, The<br />
Instructional Use of Learning Objects, ed. D. A. Wiley, [online],<br />
http://reusability.org/read/chapters/gibbons.doc.<br />
IEEE (2002) IEEE Standard for Learning Object Metadata. IEEE Standard 1484.12.1-2002, [online],<br />
http://standards.ieee.org/findstds/standard/1484.12.1-2002.html.<br />
244
Ana Mª Fernández-Pampillón et al.<br />
IMS (2003) “Content Packaging Public Draft 2. IMS”, [online], http://www.imsglobal.org/content/packaging/.<br />
IMS (2011) “Common Cartridge. Version 1.1 Final”, [online], http://www.imsglobal.org/cc/index.html.<br />
Jesukiewicz, P., dir. (2009) 14-08-2009-last update, SCORM, 2004. Version 1.1. Testing requirements. Sharable<br />
Content Object, [online],<br />
http://www.adlnet.gov/Technologies/scorm/SCORMSDocuments/SCORM%202004%204th%20Ed%20V1.1/<br />
Documentation%20Suite/SCORM_2004_4ED_v1_1_TR_20090814.pdf.<br />
LOM-ES (2010) “LOM-ES application profile for standardized Digital Learning Objects metadata” UNE-<br />
71361:2010 AENOR, [online],<br />
http://www.aenor.es/aenor/normas/normas/fichanorma.asp?tipo=N&codigo=N0044685&PDF=Si.<br />
Morales, E., Garcia, F., Rego, H. and Moreira, T. (2009) “Learning Objects Quality Management for e-Learning<br />
Systems”, International Journal of Technology Enhanced Learning, nº 1, pp. 193–204.<br />
Neven, F. and Duval, E. (2002) "Reusable Learning Objects: A Survey of LOM-Based Repositories", Proceedings<br />
of the tenth ACM international conference on Multimedia (MULTIMEDIA '02) ACM, New York, pp. 291.<br />
REBIUN (2007) Encuesta sobre la Situación de los Repositorios de Materiales Didácticos y Objetos de<br />
Aprendizaje en las Universidades Españolas, [online], http://www.rebiun.org/doc/RepositoriosE.doc.<br />
Sierra, J.L. and Fernández Valmayor, A. (2006) "A Heritage Dissemination Approach for the Production and<br />
Maintenance of Repositories of Learning Objects", [online], http://www.e-ucm.es/drafts/e-UCM_draft_57.pdf.<br />
Smith, R.S. (2004) “Guidelines for Authors of Learning Objects (White Paper)”, [online],<br />
http://archive.nmc.org/guidelines/NMC%20LO%20Guidelines.pdf.<br />
Taveira, P. and Azevedo, I. (2010) "Supporting the Reuse of Learning Objects through the Use of IMS LD<br />
Templates", Proceedings of the 9th European Conference on eLearning, eds. P. Escudeiro and C. Vaz de<br />
Carvalho, [online], http://www.academic-conferences.org/pdfs/ECEL_2010-Booklet.pdf.<br />
Universidad Nacional de Educación a Distancia (2011) Portal de Cursos Abiertos de la UNED. Open course<br />
Ware, [online], http://ocw.innova.uned.es/ocwuniversia.<br />
UCM abierta, [online], https://www.ucm.es/campusvirtual/ucmabierta/index.php.<br />
Uceda, J.; and Barro, S. (dirs) (2010) “Evolución de las TIC en el Sistema Universitario Español 2006-2010”.<br />
Conferencia de Rectores de las Universidades Españolas. Comisión Sectorial TIC, [online],<br />
http://www.crue.org/export/sites/Crue/Publicaciones/Documentos/Universitic/UNIVERSITIC2010b.pdf.<br />
UCM-OCW (2011) UCM OpenCourseWare, [online],<br />
https://www.ucm.es/campusvirtual/ucmabierta/index.php?ac=inicio.<br />
Verbert, K. and Duval, E. (2008) “ALOCOM: A Generic Content Model for Learning Objects, International Journal<br />
on Digital Libraries, Vol. 9, Num 1, pp 41-63. SpringerLink, [online],<br />
http://www.springerlink.com/content/j57176h118312386/.<br />
Vidal, C., Alejandra Segura, N., Pedro Campos, S. and Sánchez-Alonso, S. (2010) “Quality in Learning Objects:<br />
Evaluating Compliance with Metadata Standards”, eds. S. Sánchez-Alonso and I. N. Athanasiadis,<br />
Metadata and Semantic Research. Proceedings of the 4th International Conference, MTSR 2010,<br />
Springer, Berlin, [online], http://www.springerlink.com/content/x512216255073457/fulltext.pdf.<br />
Tzikopoulos, A., Manouselis, N., and Vuorikari, R. (2009) “An Overview of Learning Object Repositories”, ed. T.<br />
Halpin, Selected Readings on Database Technologies and Applications, pp. 85-94.<br />
Watson, J. (2010a) "A Case Study: Developing Learning Objects with an Explicit Learning Design, Electronic<br />
Journal of e-Learning, vol. 8, pp. 41-41-50, [online], http://www.ejel.org/Volume-8/v8-i1/v8-i1-art-5.htm.<br />
Watson, J. (2010b) "Many from One: Repurposing Learning Objects to Meet Discipline-Specific Needs", IADIS<br />
Journal on WWW/Internet, vol. 8, no. 1, pp. 16-16-26, [online], http://eprints.soton.ac.uk/170457/.<br />
Wiley, D.A. (2000) "The Nature and Origin of Instructional Objects. " in The Instructional Use of Learning Objects:<br />
Online Version., ed. D.A. Wiley, [online], http://reusability.org/read/chapters/wiley.doc.<br />
Williams, D.D. (2000) "Evaluation of Learning Objects and Instruction Using Learning Objects", The Instructional<br />
Use of Learning Objects: Online Version, ed. D.A. Wiley, [online],<br />
http://reusability.org/read/chapters/williams.doc.<br />
245
Cognitive Communication 2.0 in the Classroom –<br />
Resonance of an Experience in Higher Education<br />
Sérgio André Ferreira 1 , Cornélia Castro 1 and António Andrade 2<br />
1<br />
School of Education and Psychology, Portuguese Catholic University, Oporto,<br />
Portugal<br />
2<br />
School of Economics and Management, Portuguese Catholic University,<br />
Oporto, Portugal<br />
sergioandreferreira@gmail.com<br />
corneliacastro@gmail.com<br />
aandrade@porto.ucp.pt<br />
Abstract: The communication in the classroom is often carried out on a one-to-many basis, with the teacher,<br />
before an audience of dozens of students, playing a traditional role. In this context, still commonly practiced, it is<br />
pertinent to introduce mechanisms of interaction mediated by technology, since research shows a significant<br />
correlation between the use of technology, the time spent in social media and the engagement of students. In<br />
fact, literature shows that social media has attracted the interest of academics more likely to use technology in<br />
education, who thereby seek new ways to motivate their students to a more active <strong>learning</strong>. The adoption of<br />
cognitive communication 2.0 morphology, in traditional contexts such in communication one-to-many, is a<br />
challenge to be overcome. We consider so of great importance creating and evaluating resources and<br />
pedagogical practices that are aligned with this new paradigm. In this study, we intend to make a contribution to<br />
understanding the problematic of the morphology of cognitive communication in the context of the classroom in<br />
Higher Education, with the integration of web 2.0 tools. On the approach to the problem, we have explored a<br />
PowerPoint presentation with the integration of the micro blogging tool Twitter, as a basis for addressing the<br />
characteristics of cognitive communication 2.0. For data collection a questionnaire was designed, based on<br />
literature, and intended to evaluate several dimensions of the resource used, namely: i) pedagogical issues, ii)<br />
technological aspects, iii) cognitive <strong>learning</strong>; iv) interactions in the classroom; v) positive behaviour in the<br />
classroom; vi) negative behaviour in the classroom. The goals of this study are: i) to validate the instrument for<br />
data collection, ii) to assess the perceptions of students regarding the effects of the resource and pedagogical<br />
practices used in the classroom dynamics iii) to set in context and to relate the cognitive communication 2.0 in<br />
the classroom with other components of the Hybrid Institutional Personal Learning Environment. This is an<br />
exploratory type research, since it seeks to provide a greater familiarity with the problem and to identify<br />
dimensions and items to be included in the questionnaire. The data collected will be processed under a<br />
quantitative perspective. Considering, therefore, the nature of the study we did not seek to establish correlations<br />
between variables, but only to identify trends, using descriptive statistics. It is expected that the results obtained<br />
will contribute to the articulation of web 2.0 tools with traditional cognitive communication in the classroom, in<br />
such a manner that positive impacts will result in pedagogical and technological effectiveness and thus in<br />
students <strong>learning</strong> achievements.<br />
Keywords: classroom; cognitive communication; <strong>learning</strong>; micro blogging; Twitter; web 2.0<br />
1. Introduction<br />
Information Technologies (IT), specially the “web phenomena”, have contributed to changing the way<br />
people work together, share resources, co-produce, co-act and get involved in activities that benefit<br />
all (Fuchs et al., 2010). Nowadays, expressions such as “collaborative <strong>learning</strong>”, “<strong>learning</strong><br />
communities”, “media in education”, “social media” and other similar ones, are essential in educational<br />
investigation. However, research on these topics focuses on online environments or face-to-face<br />
groups of limited size. Studies in which these principles are applied to a classroom with dozens of<br />
students are rare.<br />
In fact, research shows that the classroom has been losing its historic centrality in favour of new<br />
agglutinating poles such as the Personal Learning Environment (PLE) and the Social Learning<br />
Network (SLN), usually associated with spaces outside the classroom – Cloud Learning Environment.<br />
However, despite the development of on-line <strong>learning</strong> systems and b-<strong>learning</strong>, classroom <strong>learning</strong> is<br />
still largely dominant, and the organization of activities continues to have the classroom <strong>learning</strong> as<br />
the nucleus. Hence the importance of designing activities and creating resources that, in conjunction<br />
with this new paradigm, promote the exploration of the potential of the new ways of <strong>learning</strong> that<br />
dominate the Cloud Learning Environment.<br />
246
Sérgio André Ferreira et al.<br />
This article aims to contribute to the study of the issues related with the cognitive communication<br />
morphology in classroom lectures, as part of Higher Education, where often the communication is<br />
done from one-to-many. To do so we started with a PowerPoint presentation with the integration of<br />
micro blogging tool Twitter, as a basis for addressing the characteristics of communication 2.0 in<br />
classroom in lectures of a more expository nature.<br />
The methodology of data collection used was a questionnaire built based on literature review, with<br />
which we intended to assess various dimensions of the resource used in classroom lectures and to<br />
verify if the integration of Twitter in the presentation contributes to the upgrading of a cognitive<br />
communication 1.0 (one-way communication, one-to-many, low or non-existent interaction) to<br />
cognitive communication 2.0 (many-to-many, interaction between all the participants). The results of<br />
this study are preliminary and intended to serve as a preliminary approach to the subject.<br />
2. Related work: Change challenges and pedagogical innovation in<br />
institutions of higher education through technology<br />
New ways of communicating and interacting in a society immersed in technology are intrinsically<br />
linked to the imposition of new ways of teaching and <strong>learning</strong>, which results in the redefinition of<br />
political and pedagogical models. Social and economic factors call for the use of technology as<br />
pedagogical support. Mark Prensky (2001) was the first to use the term "digital natives", which is<br />
associated with expressions such as "residents", "Generation Y" or "Net Generation”. These students<br />
grow up in environments immersed in technology and have different preferences and skills in key<br />
areas related to education, particularly in making use of the immense potential of web 2.0 (Castañeda<br />
& Soto, 2010; Kennedy et al., 2009), especially social software tools (blogs, micro blogs, sites of<br />
video sharing, social media, wikis or podcasts), which facilitate not only the emergence of<br />
communities of users, but also the involvement in social media.<br />
Despite the characteristics of this new generation of students, we cannot assume that all who reach<br />
higher education already possess the necessary skills to use web 2.0 technologies such as <strong>learning</strong><br />
tools (Castañeda & Soto, 2010). On the other hand, it is also observed that the Higher Education<br />
Institutions (HEIs) are still inadequately prepared to work with students who have completely different<br />
technical skills and <strong>learning</strong> preferences (Bennett, Maton, & Kevin, 2008).<br />
To address this complex situation, the HEIs must create programmes and define methodologies that<br />
enhance the use of the enormous educational potential of web 2.0. Thus, it is expected that students<br />
will develop <strong>learning</strong> skills in this context and increase their motivation and, as a result, will increase<br />
the chances of achieving good academic results.<br />
In the current scenario, it matters that HEIs reconcile their conception of the educational process with<br />
new ways of <strong>learning</strong> and student expectations. The HEIs should not overlook the fact that IT offers<br />
the students the opportunity to control and manage their own <strong>learning</strong> beyond the institutional vision.<br />
The globalization of the sources of knowledge, that brings the world to the school and the growing<br />
importance of social media and collaborative work of smart mobs (Tapscott & Williams, 2008)<br />
emphasize the value of the Social Learning Network (SLN), which creates the need for the HEIs to<br />
evolve to a Hybrid Institutional Personal Learning Environment (HIPLE) architecture, as a bridge<br />
between the vision of the institution and the Personal Learning Environment (PLE) of the student.<br />
The introduction of technology in the facilitation of cognitive function in education means a challenge<br />
for applied research with very complex and slow progress. Among the changes that have been<br />
introduced in training models under e-<strong>learning</strong>, b-<strong>learning</strong> and face-to-face, it is in the latter where the<br />
pace of change is slower. A. D. Figueiredo (2009) states that the HEIs’ face-to-face model is also the<br />
most traditional one and, in essence, it is based on four functions: i) transmission of content provided<br />
by lectures, often masterful, ii) application of concepts, iii) group work and iv) evaluation. These<br />
functions are shown in table 1.<br />
Therefore, the classroom continues to be a place of excellence in the communication of knowledge<br />
that includes multiple educational concepts. In particular, there is, above all, the transmission of<br />
information from one-to-many, dialogue, teamwork and role-playing. When technology is used it is<br />
explored in the context of the laboratory. Personal computers, tablet PCs or smart phones are poorly<br />
explored. In this context, highly practiced, it is pertinent to introduce mechanisms of interaction<br />
mediated by technology, since research shows a significant correlation between the use of technology<br />
247
Sérgio André Ferreira et al.<br />
and time spent with social media and students engagement (Chen P., 2010; Junco, Heiberger, &<br />
Loken, 2010).<br />
Table 1: Pedagogical face-to-face dominant model in the HEIs (Figueiredo, 2009)<br />
Pedagogical face-to-face model<br />
Transmission of content Lectures<br />
Lectures<br />
Application of concepts<br />
Lectures and practical lessons<br />
Practical lessons<br />
Group work<br />
Laboratories<br />
Projects<br />
Tests/exams<br />
Evaluation<br />
Projects<br />
Essays and Presentations<br />
In fact, it has been observed that the social media – a collection of Internet web sites, services and<br />
practices that support collaboration, community building, participation and sharing – has attracted the<br />
interest of academics more likely to use technology in education and who seek new ways to motivate<br />
their students to a more active way of <strong>learning</strong> (Junco, et al., 2010).<br />
In this study we focus on the context of the lectures, where teachers are faced with numerous<br />
students and have the need to activate mechanisms of direct instruction, either to save time, or<br />
because this is the teaching model that they are comfortable with. Electronic presentations as a way<br />
to transmit knowledge are a means commonly used in these classes (James, Burke, & Hutchins,<br />
2006).This is a reusable resource, a facilitator of discourse organization and an integrator of multiple<br />
media which can serve different <strong>learning</strong> styles, as well as becoming a stepping stone of motivation<br />
and it is also conducive to note taking by students.<br />
These presentations are massively materialized on technologies such as PowerPoint and Prezi which<br />
have mechanisms to support text, video, image, flash animation and sound, but they also have the<br />
ability to interact with the so-called web 2.0 systems, such as the micro blogs. A PowerPoint or a<br />
Prezi presentation can therefore be linked with the micro blogging application Twitter, allowing the<br />
teacher the opportunity to speak to his students.<br />
Recent research about the use of Twitter in academic work shows that although 85 % of<br />
undergraduates have a Facebook account, teachers prefer to integrate Twitter into the process of<br />
teaching and <strong>learning</strong> (Junco, et al., 2010). In the category of micro blogs, Twitter, designed in 2006<br />
by Jack Dorsey, allows users to share messages up to 140 characters. This system also allows<br />
sending messages to a direct channel specifically created to exchange information (# hashtags) and<br />
the vote for alternative options which are placed under review (by vote tweet @ x key_word).<br />
Individual or business initiatives (Elliot, 2011) have developed ADD-INS for Prezi and PowerPoint,<br />
allowing the following dynamics for those who have a Twitter account (teachers and students):<br />
Creation of a channel (not compulsory) for comments on the presentation that is being made;<br />
Students can comment directly on Twitter what they see and listen in the classroom as well as<br />
what they read from sharing with peers (virtual classroom);<br />
The teacher may have prepared additional comments for each slide, that hidden in "notes", can<br />
be sent to Twitter whenever it is projected;<br />
The teacher can capture and project in all, or in some of the screens, what is being shared on<br />
Twitter;<br />
The teacher can ask multiple-choice questions that are answered on Twitter, and the percentages<br />
of the responses for each option can be projected in a slide.<br />
This model of communication in the classroom will increase the level of participation by: i) providing<br />
voice and turn to all the students, ii) facilitating and asking for the participation of more reserved<br />
students in oral participation iii) engaging the <strong>learning</strong> community in discussions about the theme and<br />
iv) exploring the acuity of young people to use IT.<br />
248
3. Evaluation methodology<br />
Sérgio André Ferreira et al.<br />
As a methodological approach to the problematic of the morphology of cognitive communication 2.0 in<br />
the context of the classroom, we used a PowerPoint presentation with the integration of Twitter. The<br />
exploitation of this resource was made during lectures to several masters classes, at the Portuguese<br />
Catholic University – Regional Center of Porto (Catholic – Porto). In table 2 the demographic<br />
characteristics of the participants in this study are shown.<br />
Table 2: Demographic aspects of the investigation<br />
Total 122<br />
Gender Male: 29% Female: 71%<br />
Age<br />
Group<br />
Training<br />
Undergone<br />
20 to 29 years old: 37 %<br />
30 to 39 years old: 26 %<br />
40 to 49 years old: 20 %<br />
50 to 59 years old: 17%<br />
Marketing: 3 %<br />
Services: 3 %<br />
Education and Music: 15 %<br />
Computing and Education: 3 %<br />
Pedagogical Supervision: 22 %<br />
Religious Sciences: 4 %<br />
Wounds and Tissue Viability: 4 %<br />
Infection and Health Care: 8 %<br />
School Management and Organization: 15 %<br />
Childhood Education and Special Education: 23 %<br />
The way the operationalization of the resource was made is represented, in schematic form, in Figure<br />
1: the integration of Twitter in PowerPoint, with reference to a hashtag #, through which students<br />
could ask and answer questions, vote on matters presented and answer multiple choice questions.<br />
Thus, the electronic presentation, not only had the traditional function of transmitting information from<br />
one to many, but it was also intended to foster interaction content-students, teacher-students,<br />
students-students. The integration of Twitter in an electronic presentation, enables, potentially, a<br />
paradigm shift in teaching: to the one-way communication teacher-class is added the value of the<br />
interaction teacher-student-content. The feedback given by students is an important item because it<br />
allows the teacher to suit his speech to the class and answer students' questions and comments that<br />
appear in real-time presentation.<br />
Figure 1: Integrating cognitive technologies in communication<br />
At the end of the sessions, the participants were asked to fill a questionnaire to evaluate the resource.<br />
The convenience sampling technique was our choice. Although not representative of the population ,<br />
this sampling technique had the advantage of being a fast and simple one and, therefore, suitable for<br />
preliminary studies, as in this case.<br />
The questionnaire was made based on literature review (Hu, 2011; James, et al., 2006; Kurilovas,<br />
2007; Nesbit, 2007; Nokelainen, 2006) and consisted of 41 items spread across six dimensions: i)<br />
pedagogical aspects, ii) technological aspects, iii) cognitive <strong>learning</strong> iv) interactions in the classroom,<br />
v) positive behaviour in the classroom; vi) negative behaviour in the classroom. The six evaluated<br />
dimensions include: i) issues that students identify as central in the quality of digital <strong>learning</strong><br />
resources associated to more direct teaching, ii) the perceived effects by students in the field of<br />
<strong>learning</strong>, behaviour and attitudes. A Likert scale of five points was used.<br />
4. Presentation of results<br />
The overall results of the questionnaire indicate that the respondents recognize the pedagogical and<br />
technological potential of the resource in the six assessed dimensions, as well as its positive effects<br />
on the quality of <strong>learning</strong> and type of interaction. In Figure 2 the assessment on pedagogical aspects<br />
249
Sérgio André Ferreira et al.<br />
is presented. In the seven analyzed items, the large majority of respondents evaluated the<br />
pedagogical aspects of the course with level four, "agree" and level 5 "strongly agree." The global<br />
average of the seven items corresponds to 55 % of responses at level 4 and 22 % at level 5, and the<br />
importance of level 1 and 2 is negligible (0 % and 4 %, respectively).<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. Scientifically rigorous<br />
2. Pedagogically appropriate<br />
3. It has the correct sequence<br />
4. It facilitates the explanation of abstract concepts<br />
5. It is presented clearly (spelling, grammar, ...)<br />
6. It is easy to understand the goals<br />
7. Appropriate level of detail<br />
Figure 2: Pedagogical aspects<br />
As far as the "technological aspects" is concerned (Figure 3), the respondents commented on the<br />
appropriateness of the use of technology, design, usability, interface, added value compared to<br />
printed material, the potential of technology in facilitating <strong>learning</strong>, building concepts and skills<br />
development. The average of the eight items of this dimension indicates that 52 % of students "agree"<br />
and 32 % "strongly agree" that the technology used was appropriate and that potentiated <strong>learning</strong>.<br />
Like in the previous dimension, the number of respondents that gave unfavourable levels of answers<br />
(levels 1 and 2) is negligible.<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. Use of technology appropriate to the content<br />
2. The design is appealing<br />
3. The design facilitates the clarification of the<br />
content<br />
4. The resource is easy to use<br />
5. The interface facilitates navigation<br />
6. The resource is an added value compared to<br />
printed material<br />
7. The used technology facilitates <strong>learning</strong><br />
8. The technology helps building concepts and<br />
developing skills<br />
Figure 3: Technological aspects<br />
Regarding "cognitive <strong>learning</strong>" (Figure 4), if one considers the average of the 11 items on this<br />
dimension: 82 % of the respondents said that they "agree" (47 %) or "strongly agree" (35 %) that the<br />
resource has positive effects. In the average of the 11 items, the value of the terms "strongly<br />
disagree" and disagree" is located at 2 %. However, 7% of the respondents "disagree" and 1 %<br />
"strongly disagree" that the resource facilitates taking notes (item 3).<br />
The potential of the evaluated resource in the "interactions in the classroom" are also recognized by<br />
the students. The average of the five presented items in Figure 5 indicates that 70 % of respondents<br />
"agree" (40 %) or "strongly agree" (30 %) with the positive effects of the resource on this dimension.<br />
The item 1 is the one that meets the highest percentage of negative and neutral answers: 2 %<br />
"strongly disagree" 11 % "disagree" and 35 % "do not agree nor disagree" that the resource leads to a<br />
better knowledge of the classmates in the classroom.<br />
250
Figure 4: Cognitive <strong>learning</strong><br />
Sérgio André Ferreira et al.<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. It helps to improve understanding<br />
2. It facilitates memorizing information<br />
3. It facilitates note taking<br />
4. It favours a more appropriate pace of<br />
presentation<br />
5. It facilitates fluency when transferring<br />
information in the classroom<br />
6. The examples presented become clearer<br />
7. It fosters a better coordination between the<br />
classroom activity and the texts<br />
8. It makes <strong>learning</strong> more motivator<br />
9. It allows the definition of several ways of<br />
<strong>learning</strong><br />
10. It favours the immersion in <strong>learning</strong> and<br />
encourages further <strong>learning</strong> of contents<br />
11. Mobilizes prior knowledge<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. It leads to a better knowledge of classmates in<br />
the classroom<br />
2. Promotes the will to participate in class<br />
3. It facilitates a closer relationship with the<br />
teacher and classmates<br />
4. It facilitates discussion in the classroom<br />
5. It allows participation via a technological<br />
channel<br />
Figure 5: Interaction in the classroom<br />
The results in the dimension "positive behaviour in the classroom" (Figure 6) are in agreement with<br />
those of the other dimensions. Considering the average of the 7 items, 75% of respondents "agree"<br />
(49 %) or "strongly agree" (26 %) that the resource has positive effects on behaviour in the classroom.<br />
The item 1, which states that the resource helps taking better notes in class, holds the highest<br />
number of negative evaluations (1 % "disagree" and 15 % "strongly disagree"). This result is aligned<br />
with item 7 "it stimulates coming to class to take notes (15 % of respondents answered "disagree") as<br />
well as with item 3 in Figure 4.<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. It helps taking better notes during the lectures<br />
2. The visual images presented in the resource<br />
help to remember the contents during exams<br />
3. It allows a better understanding of the keypoints<br />
highlighted during lessons<br />
4. It enhances attention in the classroom<br />
5. This resource is used as a support to group<br />
work, negotiation and dialogue in classroom<br />
6. It helps keeping interest alive during lectures<br />
7. Encourages coming to class to take notes on<br />
important topics<br />
Figure 6: Positive behaviour in the classroom<br />
In the dimension "negative behaviours in the classroom", the scale should be read in reverse: the<br />
more favourable views about the resource are located on level 1 and the less favourable in level 5.<br />
251
Sérgio André Ferreira et al.<br />
Looking at Figure 7 we can conclude that: i) 36 % "agree" and 19 % "strongly agree" that the resource<br />
increases the possibility of keeping side conversations while the teacher presents the subject, ii) 8 %<br />
"agree" and 8 % "strongly agree" that the resource reduces the motivation to be present in class. This<br />
figure (16 %), although low in percentage terms, assumes an important meaning and is not aligned<br />
with the very positive evaluation made to other dimensions and iii) 24 % "agree" and 7 % "strongly<br />
agree" that they are more likely to skip school if they know that the resource will be available on the<br />
web.<br />
Scale<br />
1. Strongly disagree to 5. Strongly agree<br />
NOP. No opinion<br />
Legend<br />
1. It increases the possibility of keeping side<br />
conversations while the teacher presents the<br />
subject<br />
2. It reduces the motivation to be present in class<br />
3. The probability to miss class is higher if the<br />
students know that the resource will be available<br />
on the web<br />
Figure 7: Negative behaviour in the classroom<br />
Table 3 shows the average of responses in each dimension for each course attended. Only courses<br />
with 10 or more students were taken into consideration as it was assumed that courses with fewer<br />
students didn’t have any statistical significance. However, it is important to state that the same class<br />
could integrate students from various courses, which increases the dimension of students in each<br />
session. What was meant by this analysis was to identify possible differences in opinions according to<br />
the type of training. The students from School Management and Organization and Childhood<br />
Education and Special Education courses are the ones that, overall, make a more positive<br />
assessment of the resource used. On the same scale of 1 to 5, the average score for items 1 to 5<br />
(item 6 was not considered in this average as it presents a reverse scale) was 4,3. The less<br />
favourable reviews were noted in the course Pedagogical Supervision, with an average of 3,7.<br />
Although the type and size of the sample do not make it possible to establish correlations between<br />
variables, the results of this preliminary study indicate that there is material for further analysis in this<br />
field.<br />
Table 3: Average of results in each dimension per course<br />
Training<br />
Undergone<br />
Pedagogy<br />
Technology<br />
Cognitive Learning<br />
Education and Music 3,8 4,1 3,9 3,8 3,8 3,2 3,9<br />
Pedagogical<br />
Supervision<br />
3,8 3,9 3,9 3,4 3,6 2,8 3,7<br />
Infection and Health<br />
Care<br />
4,1 4,0 4,4 4,2 4,0 2,1 4,1<br />
School Management<br />
and Organization<br />
4,0 4,4 4,5 4,3 4,3 2,5 4,3<br />
Childhood Education<br />
and Special Education<br />
4,2 4,4 4,3 4,0 4,2 2,4 4,3<br />
An analysis of differences in various dimensions, by gender, was made in this study. In dimensions 1<br />
to 5 no significant differences are visible. Gender disparities are only visible in dimension 6, which<br />
refers to negative behaviour in the classroom (Table 4). As shown in the three items considered, male<br />
respondents considered that the resource has more negative effects.<br />
252<br />
Interaction in class.<br />
Positive behaviour in<br />
class,<br />
Negative behaviour<br />
in class,<br />
Average<br />
items 1 to 5
Table 4: Significant differences per gender<br />
Sérgio André Ferreira et al.<br />
Men Women<br />
1. It increases the probability to keep side conversations while the teacher<br />
presents the subject<br />
3,7 3,2<br />
2. It reduces motivation to be present in class 2,7 2,2<br />
3. The probability to miss classes is higher if it is known that the resource<br />
will be available on the web<br />
3,1 2,5<br />
5. Results discussion and conclusions<br />
Constructivism as a philosophy and pedagogy is now widely accepted and is close to most<br />
contemporary theories of teaching and <strong>learning</strong>: "constructivism", "situated <strong>learning</strong>", "social<br />
cognition", "activity theory", "distributed cognition", "ecological psychology", and "case-based<br />
reasoning" (Jonassen, Howland, Moore, & Marra, 2003). Education policies, including the Bologna<br />
process, give expression to these pedagogical models of constructivist nature. In line with this<br />
perspective is the way new generations of students learn: in a more flexible way, not just playing the<br />
role of passive consumers of information, but as active builders in their <strong>learning</strong> process. The social<br />
media provided by web 2.0 support this new philosophy of <strong>learning</strong> based on community building,<br />
participation and sharing.<br />
There are several studies about the use of the web 2.0 tools potential as an approach to more<br />
constructivist philosophies. However, few are the studies focused on classroom lectures, where the<br />
teacher faces a class-group, consisting of several dozen of students with the need for direct<br />
instruction, to be able to present a large volume of information in the minimum time. In this context,<br />
which occurs often in higher education, dialogue and interaction are limited.<br />
Figure 8 depicts the morphology of the communication process in a lecture to large groups, in which<br />
the electronic presentation only allows one-way communication, the interaction is not favored and the<br />
student is sent to a more passive role.<br />
Figure 8: Cognitive communication morphology 1.0 in the context of a lecture<br />
Mayer (Pennsylvania State University, 2010) admits the possibility of using resources more<br />
associated with direct instruction to achieve more constructivist approaches. In this study we sought<br />
to determine whether the integration of the web 2.0 tool Twitter micro blogging in a PowerPoint<br />
presentation contributed to the upgrade of a cognitive communication 1.0 to a cognitive<br />
communication 2.0 in the context of a lecture (Hu, 2011). The applied questionnaire allowed to assess<br />
various dimensions of the resource used in the classroom and to verify the changes in the<br />
morphology of cognitive communication. The results indicate that students recognize the potential of<br />
the resource used in the dynamics and dimensions of a cognitive communication situation 2.0, in<br />
particular with regard to: i) pedagogical aspects, ii) technological aspects, iii) cognitive <strong>learning</strong>; iv)<br />
interactions in the classroom and v) positive behaviour in the classroom.<br />
253
Sérgio André Ferreira et al.<br />
In Figure 9 the results of the resource at interaction level and the direct effects on the student’s<br />
activity are summarized. Twitter integration favours a multidirectional communication and an increase<br />
of the interaction between teacher-student, student-student and student-content. This leads to an<br />
upgrade of the level of cognitive communication from version 1.0 to 2.0. The findings corroborate the<br />
results obtained when applying this questionnaire to a smaller sample (Ferreira, Castro, & Andrade,<br />
2011) and are aligned with other studies, namely: i) the positive effects on <strong>learning</strong> (Balanskat,<br />
Blamire, & Kefala, 2006; James, et al., 2006; Junco, et al., 2010) and ii) the relation between the use<br />
of technology and the student involvement in school activities (Balanskat, et al., 2006; Chen P., 2010;<br />
Junco, et al., 2010).<br />
Figure 9: Morphology of cognitive communication 2.0 in the context of a lecture<br />
For future studies, it is suggested i) to extend the sample in order to validate the questionnaire and to<br />
allow the study of correlations between variables and ii) to develop a more systemic way of research,<br />
integrating the issue of cognitive communication 2.0 in the context of the classroom in HIPLE context.<br />
References<br />
Balanskat, A., Blamire, R., and Kefala, S. (2006) The ICT Impact Report: A review of studies of ICT impact on<br />
schools in Europe, European Schoolnet in the framework of the European Commission’s ICT Cluster,<br />
Brussels, Belgium.<br />
Bennett, S., Maton, K., and Kevin, L. (2008) "The ‘digital natives’ debate: A critical review of the evidence", British<br />
Journal of Educational Technology, Vol. 39, No.5, pp. 775-786.<br />
Castañeda, L., and Soto, J. (2010) "Building Personal Learning Environments by using and mixing ICT tools in a<br />
professional way", Digital Education Review, No.18, pp. 9-25.<br />
Chen P., L. A. D. G. K. (2010) "Engaging online learners: the impact of web-based <strong>learning</strong> technology on<br />
college student engagement", Computers and Education, Vol 54, No.4, pp. 1222--1232.<br />
Elliot, T. (2011) "SAP web 2.0", [online], http://www.sapweb20.com/ (acessed on 20/05/2011).<br />
Ferreira, S. A., Castro, C., and Andrade, A. (2011). "Morfologia da Comunicação Cognitiva 2.0 em sala de aula<br />
no âmbito do Ensino Superior", In Proceedings of CISTI 2011 – 6ª Conferência Ibérica de Sistemas e<br />
Tecnologias de Informação, pp. 294-300, Chaves, Portugal, June.<br />
Figueiredo, A. D. (2009) "Estratégias e Modelos para a Educação Online", In G. L. Miranda (Ed.), Ensino Online<br />
e Aprendizagem, Lisboa, Relógio d´Água, pp. 33-55.<br />
Fuchs, C., Hofkirchner, W., Schafranek, M., Raffl, C., Sandoval, M., and Bichler, R. (2010) "Theoretical<br />
Foundations of the web: Cognition, Communication, and Co-Operation. Towards an Understanding of web<br />
1.0, 2.0, 3.0", Future Internet, Vol. 2, No.1, pp. 41-59.<br />
Hu, W. (2011) "Mayer's SOI model" [online], http://www.personal.psu.edu/wxh139/SOI.htm (acessed on<br />
20/05/2011).<br />
James, K. E., Burke, L. A., and Hutchins, H. M. (2006) "Powerful or Pointless? Faculty Versus Perceptions of<br />
PowerPoint Use in Business Education", Business Communication Quarterly, Vol. 69, No.4, pp. 374-396.<br />
Johnson, L., Smith, L. R., Willis, H., Levine, A., and Haywood, K. (2011) The 2011 Horizon Report, The New<br />
Media Consortium, Austin, Texas, USA.<br />
Jonassen, D., Howland, J., Moore, J., and Marra, R. (2003) Learning to solve problems with technology, Merril<br />
Prentice Hall, New Jersey.<br />
254
Sérgio André Ferreira et al.<br />
Junco, R., Heiberger, G., and Loken, E. (2010) "The effect of Twitter on college student engagement and<br />
grades", Journal of Computer Assisted Learning, Vol. 27, No.2, pp. 1-14.<br />
Kennedy, G., Dalgarno, B., Bennett, S., Gray, K., Waycott, J., Judd, T., Chang, R. (2009) Educating the Net<br />
Generation – a handbook of findings for practice and policy, Australian Learning and Teaching Council,<br />
Melbourne.<br />
Kurilovas, E. (2007) "Digital library of educational resources and services: evaluation of components", [online],<br />
http://bit.ly/mz7UH1 (acessed on 20/05/2011).<br />
Nesbit, J. K. B. T. L. (2007) "Learning Object Review Instrument (LORI) Version 1.5", [online],http://bit.ly/laXZHE<br />
(acessed on 20/05/2011).<br />
Nokelainen, P. (2006) "An empirical assessment of pedagogical usability criteria for digital <strong>learning</strong> material with<br />
elementary school students", Educational Technology and Society, Vol 9, No. 2, pp. 178-197.<br />
Pennsylvania State University (2010). Mayer's SOI model”, Available:<br />
http://www.personal.psu.edu/wxh139/SOI.htm (acessed on 20/05/2011).<br />
Prensky, M. (2001) "Digital Natives, Digital Immigrants", On the Horizon, Vol 9, Nº 5, pp.1-6.<br />
Tapscott, D., and Williams, A. D. (2008) Wikinomics: A Nova Economia das Multidões Inteligentes, Quidnovi,<br />
Lisboa.<br />
Vilelas, J. (2009). Investigação – O processo de construção o conhecimento, Edições Sílabo, Lisboa<br />
255
To What Extent Does a Digital Audio Feedback Strategy<br />
Support Large Cohorts?<br />
Rachel Fitzgerald<br />
University of Northampton, Northampton, UK<br />
Rachel.Fitzgerald@northampton.ac.uk<br />
Abstract: The UK National Student Survey (NSS) regularly highlights student dissatisfaction with feedback,<br />
identifying factors such as timeliness; personalisation; specific information on how to improve in a clear and<br />
understandable manner and the level of detail given to students related to <strong>learning</strong> outcomes. For a large cohort,<br />
achieving these targets can be difficult and students are more likely to be at risk of receiving rushed and vague<br />
feedback as lecturers strive to return results as quickly as possible. In his research into the use of digital audio for<br />
feedback in the JISC Sounds Good Project, Bob Rotherham suggests that audio may be a way to assist<br />
“lecturers looking for a way of giving students good quality feedback on their work whilst saving time” Rotherham<br />
(2008, p1). Saunders et al (2005) suggests that a good teacher is one will take advantage of ICT opportunities in<br />
order to enrich the students experience, and this research evaluates the potential to provide meaningful, quality<br />
feedback to a large group of first year students on a Information Management module at Northampton Business<br />
School, via digital audio files. Using an action research methodology, this first cycle of research evaluates the<br />
process from the creation of the digital audio files right through to the personalised approach of returning the files<br />
to the students via the virtual <strong>learning</strong> environment. This paper analyses the effect of the experience on the<br />
lecturers involved and through subsequent group discussion and questionnaires, this research also evaluates the<br />
thoughts of the students involved and considers the overall impact on both home and international students.<br />
Initial results indicate that for large cohorts there is no simple answer but electronic feedback is certainly seen as<br />
more personal, more beneficial and digital audio may have some unexpected benefits for stakeholders. This<br />
paper will identify how this method will be reviewed for a further action research cycle.<br />
Keywords: digital audio feedback, feedback and assessment, action research<br />
1. Introduction<br />
In his research into the use of digital audio for feedback in the Sounds Good Project, Bob Rotherham<br />
asks “are Lecturers looking for a way of giving students good quality feedback on their work whilst<br />
saving time? Of course they are” Rotherham (2008, p1).This statement led to critical reflection on the<br />
student experience of feedback on an Information Management module of over 100 first year<br />
students, where assessment feedback is provided solely by the module leader. This is a 20 CAT<br />
Credit (approx 10 ECTS), Level 4 long thin module taken by students from range of disciplines<br />
including Business, Information Science, Accounting, Marketing and Human Resource Management.<br />
The module embeds core study skills and acts as a transition to develop first year understanding of<br />
academic skills in context. Delivery is face to face and assessment is online. The University Virtual<br />
Learning Environment (VLE) is used to support the students throughout the module via seminar<br />
activities, additional reading and assessment; the students become quite familiar with the VLE<br />
throughout the programme. The assessment in question requires the students to submit a research<br />
project proposal that is graded on independent research, appropriate reference skills, academic<br />
writing and presentation skills. It is a first step towards an overall project in the module so feedback is<br />
critical to developing a successful project.<br />
Rotherham’s (2009a) Sounds Good funded project was successful on a small-scale and students<br />
participating commented on how personal the approach was and that they felt that they understood<br />
their feedback more than with a traditional hand-written approach. Rotherham found that the process<br />
had taken him less time than if he had written feedback scripts. These results suggested win-win, so<br />
this research was undertaken to evaluate if a similar approach could work for a large cohort and if it<br />
was a solution that might work for other situations for example international distance <strong>learning</strong> (DL).<br />
The module leader has the autonomy to change practice for this module and is very much part of the<br />
research, from engaging students with the process, marking all the assessments, creating audio<br />
feedback, observing the impact and reflecting on the experience. Where the researcher wants to<br />
improve their practice, increase their student achievements and is completely engaged in the process,<br />
Ferrence (2000) suggests that action research is the most appropriate methodology.<br />
256
2. Literature review<br />
Rachel Fitzgerald<br />
The concept of using technology to support the process of feedback is not new, Harvey & Mogey<br />
(1999) mooted the use of technology as a way for students to access feedback as and when it was<br />
required and Rust (2001) advocated the use of audiotape to comment on students’ work, suggesting<br />
that this would take less time than written feedback and the students would gain more from the tone<br />
of their tutor than they would from written comments, which Rotherham (2009b) found to be the case<br />
using digital audio for the JISC Sounds Good project.<br />
The potential for audio to impact on international students is something that Copley (2007) suggests<br />
is related to having access to ‘teachers’ voice. Kaplan-Leiserson (2005 p2) identifies opportunities for<br />
time efficient feedback via podcasts and similarly mentions the benefit of access to the “professor’s<br />
voice”. Sounds Good provided students with digital audio feedback to evaluate if audio feedback<br />
would provide richer feedback to students and save assessors time, Rotherham (2009b). Results<br />
from this project indicate that students were positive about the personal nature of audio feedback and<br />
that it did save time but only if the assessor was comfortable using the technology. Fish and Lumadue<br />
(2010) suggest that audio feedback has such a significant impact on the student experience, that<br />
tutors engaged in distance <strong>learning</strong> programmes should consider it as a way to support students.<br />
This is all very positive, however audio feedback is “not a magic bullet for assessment woes”<br />
Rotherham (2009a, p9) and unless the student has been directed to have their original paper in front<br />
of them, they cannot see what the marker is referring to, during the one way conversation, Kerr and<br />
McLaughlin (2008). More fundamentally, feedback, whether audio or not, must be purposeful.<br />
Feedback is considered effective when it is timely, relevant, encouraging and offers suggestions for<br />
improvement, Brown (2001); Rotherham (2009b). Gibbs (1981) suggests that even if the feedback<br />
tells students what to do there is no real reason that they will follow that advice, no matter how many<br />
times they listen to it. Improving feedback for students is an ongoing study, in the Assessment series<br />
for LTSN, Brown (2001) considers the impact of assessment and notes that the purpose of feedback<br />
is to motivate students, to inform how well they have done and how to improve. Rani and Yahya<br />
(2009) advise that the process of feedback should develop the student experience, and suggest that<br />
lecturers should provide feedback that will stimulate <strong>learning</strong>. Rust (2001) suggests that the tutor<br />
should relate the feedback to the <strong>learning</strong> outcomes of the assessment and the feedback should<br />
show students their strengths and weaknesses in relation to outcomes, feedback should also indicate<br />
how they can improve in the future, without this level of detail, Rust implies, students will just become<br />
weaker. Although Price (2007, p9) warns of the dangers of using too much detail without clarity as it is<br />
more likely to mislead or confuse students and is a “guarantee that they will be less likely to engage<br />
with feedback”<br />
Not engaging with feedback is an issue, there is a general perception that students are failing to<br />
understand the feedback they receive, and in some cases are just not reading it, Holmes and<br />
Papagergiou (2009); Rani & Yahya (2009). Bols (2010) in a nod to the National Student Survey<br />
results (NSS) suggests that this is a result of lack of timeliness from the tutor and that students have<br />
generally moved on by the time they receive feedback, as it is no longer relevant. Holmes and<br />
Papageorgiou (2009) researched student perceptions and expectations of feedback and pick up on<br />
the requirement for timely feedback, they also suggest that students perceive feedback as particularly<br />
crucial to their first year experience. The NUS (2010), Charter of Feedback and Assessment<br />
encourages more timeliness in the return of feedback and make a suggestion that students should<br />
have access to more personalised feedback, particularly for their first assessment of each year and<br />
that the students should be able to state their choice on what form this should take.<br />
3. Background of the research<br />
The National Student Survey regularly highlights student dissatisfaction with feedback, Lipsett (2007),<br />
this usually relates to timeliness, although other factors have been identified such as the level of detail<br />
given to students related to <strong>learning</strong> outcomes and specific information on how to improve in a clear<br />
and understandable manner. Being aware of the issues with feedback is important to this research as<br />
the assignment at the heart of the process is part one of two in a first year module in Information<br />
Management. Feedback from the initial assignment is intended to be used by the students to support<br />
and enhance the second assignment, so it is important that they read and understand the feedback<br />
and apply it to their next assignment. There are over a hundred students enrolled on the module, so<br />
257
Rachel Fitzgerald<br />
consideration of the most appropriate way to provide meaningful and memorable feedback is an<br />
issue, particularly as the students are first years and generally in need of academic skills guidance.<br />
Ensuring that the students can access the feedback easily is the first hurdle, Gibbs & Simpson (2004)<br />
suggest that if the students cannot then see the relevance of feedback to any of their other work, they<br />
will just check the grade and chuck the rest in the bin. Other studies demonstrate that students do not<br />
register feedback as feedback unless it is made absolutely apparent, Rani & Yahya (2009).<br />
Assessment is more effective when the criteria is absolutely clear to students, Gibbs (1981), Rani &<br />
Yahya (2009), Holmes & Papageorgiou (2009), therefore feedback should relate to the criteria to<br />
make it understandable and relevant to each individual student but with large student numbers this is<br />
difficult to achieve, and has the potential to lead to perfunctory feedback, Brown (2001). Even with<br />
high student numbers, there is an institutional expectation that turnaround from assignment<br />
submission to receiving feedback will be three weeks, which sometimes leads to rushed, ineffective<br />
feedback. Against this context, it seems a good time to try something new in an attempt to avoid the<br />
pitfalls of repetition whilst maintaining some aspect of individuality for students. As Saunders et al<br />
(2005) suggest, a good teacher is one will take advantage of ICT opportunities in order to enrich the<br />
students experience.<br />
4. Action Research as a methodology<br />
Action Research (AR) is a way for practitioners to engage in meaningful research, that develops the<br />
researcher professionally, Koshy (2005). AR is a cyclical process, O’Leary (2004), as knowledge<br />
emerges from research, so do new questions or new practices to try and the cycle begins all over<br />
again but with ongoing developing awareness.<br />
Figure 1: The Action Research cycle adapted from Kemmis & McTaggart (2005 p564)<br />
For this research, the researcher is interested in how to make feedback more effective for students<br />
with a view to transforming practice and for informing practice for colleagues. The initial cycle plan is<br />
to use audio feedback as an alternative to written feedback and to observe the impact for both myself<br />
as tutor and for the students and to evaluate and critically reflect on the process to inform how to<br />
approach feedback during the next cycle of research.<br />
5. Research method and data collection<br />
For this research the tutor and students were engaged in the process from the outset, students were<br />
informed in advance that personal feedback would be online via a digital audio file and that transcripts<br />
of the audio would be available to anyone who required it. During initial discussions students<br />
confirmed that they had never tried this method before and normally received traditional written<br />
feedback. The turnaround for the project was swift, the assessment was due in mid-Nov and the<br />
audio feedback needed to be available to students early in December. Due to time limitations, data<br />
gathering was done via classroom discussion, an online survey and through evaluation of the number<br />
of times feedback was accessed on the VLE. O’Leary (2005) recommends undertaking a pilot before<br />
258
Rachel Fitzgerald<br />
running full data collection, but with the time factor and a Christmas holiday in the midst of the data<br />
gathering, it was decided to run the full survey. Although not ideal, as the first stage of an AR cycle it<br />
was considered that all data would be of interest and any limitations would serve to improve the next<br />
cycle of research. Survey, as a method was chosen specifically because it suited the time constraints<br />
and students with erratic attendance patterns. An online survey was deployed in the virtual <strong>learning</strong><br />
environment where students access all module materials (and assessment feedback), this seemed<br />
ideal for gathering many responses and opinions about the process. Awareness of the students also<br />
informed the researcher to add a number of response categories to the survey to encourage them to<br />
get involved. This allowed the survey to be uncomplicated and swiftly completed with an option to add<br />
qualitative opinions for general analysis. The type of survey questions affect the quality of data<br />
received, O’Leary (2005), but as this data was gathered to get a general consensus, it was expected<br />
to generate questions for further cycles of research.<br />
To participate, students submitted their assignments electronically. Prior to recording the audio, the esubmissions<br />
were read and graded with some rough notes made about the work, then a selection<br />
were sent (via email) to be second marked. Typed comments from the second marker were<br />
incorporated into the rough notes and then the feedback recording for all students took place over two<br />
days. Seven survey questions were put together to measure the students’ perception of these areas<br />
and to understand if they found the experience more personal. In addition, the survey was designed<br />
to collect data about the impact of the process on international students, with a view to informing<br />
practice in the DL area. In terms of the feedback that was to be recorded, Gibbs & Simpson (2004)<br />
suggests that poor feedback is feedback that is backward looking and addresses issues that will not<br />
be required again. With the feed-forward aspect of the assessment in mind, feedback addressed the<br />
student personally and commented on meeting the <strong>learning</strong> outcomes and included clear guidance<br />
about academic writing, referencing and points to expand on for the next assignment and then they<br />
received the grade.<br />
6. Critical reflection on the process<br />
Recording feedback required little technical knowledge, just the use of a headset and Audacity<br />
software. After a trial recording to test, each recording took less than two minutes, the feedback was<br />
quite specific and therefore time efficient. To keep a personal touch, feedback was in a natural tone<br />
and intonation, and spoken as if talking directly to the student. In a study of 15 students, Merry and<br />
Orsmond (2007) highlight the importance of tone for audio feedback, but what may have been easy<br />
with just 15 students was not the case with over 100 and it was difficult maintaining a natural or even<br />
enthused tone throughout the process. The feedback was similar for many students so it became<br />
perfunctory and repetitious. Rotherham (2008, 2009a) suggests that audio feedback reduces the time<br />
spent on feedback and this was also the case in this research, however while it is easier and quicker<br />
to verbally explain a concept, when one uses typed feedback there is an opportunity to cut and paste–<br />
with large student numbers, this can be a useful tool. In addition, while the recordings saved time,<br />
there was a lot more time spent uploading each individual file to a secure area on the VLE, this<br />
administrative process added at least two days to the process and meant that the feedback was a day<br />
late being returned. This would be a barrier to using audio feedback for these students in the next<br />
cycle.<br />
When the feedback was available to the students, a notice was posted to the VLE informing students<br />
and offering a transcript if required. To get a transcript students had to email me, in all around fifteen<br />
students requested a transcript of their feedback. Using the report tool in the VLE, the graph in Fig 2<br />
shows significantly increased activity on the day the feedback was available (7 th Dec 2010) and in the<br />
subsequent days, this activity tails off during the Christmas vacation and then returns to what can be<br />
considered normal access during Jan 2011. A classroom discussion with the students indicated that<br />
they had no trouble accessing the audio file and most found it to be a novel approach and were happy<br />
to accept this as a new method of feedback. All indicated they were aware that they could have a<br />
transcript on request, but generally didn’t feel the need to request one.<br />
Through the VLE we can see how often each student on the module accessed their feedback, the<br />
VLE report indicated two interesting points that will need further investigation:<br />
The majority of students on the module accessed the file at least twice, with two students<br />
accessing the folder over twenty times. Rotherham (2009b) suggests that students find it harder<br />
to skim through audio feedback and therefore need to go back to listen again to clarify their<br />
understanding. However it could also mean that the students found my voice difficult to<br />
259
Rachel Fitzgerald<br />
understand, although the classroom discussion and the small number of requests for transcripts<br />
indicate otherwise –this requires further evaluation.<br />
A small number of students (that submitted work) did not access their feedback, of these just two<br />
requested a transcript –this is curious as there is no other way to access the grade. I suspect that<br />
these students will access their feedback closer to submission date for the next assignment and<br />
will include this as part of my observation in the next cycle of research.<br />
Figure 2: Graph of hits on feedback on VLE over December 2010<br />
7. Evaluation of the survey<br />
The survey was completed by 51 students, less than half of those enrolled on the module but a<br />
reasonable number for a general consensus, the survey returned both quantitative data as well as<br />
qualitative.<br />
QUESTION 1: DID YOU LISTEN TO THE FEEDBACK ON YOUR FIRST ASSIGNMENT?<br />
Figure 3: Representative of responses to question 1<br />
Although the report indicates that a number of students did not access their feedback, in the survey<br />
there was only one negative response. This is a limitation of this cycle that needs to be looked at in<br />
further detail, a further cycle should consider whether the students just do not engage ever or if they<br />
are just waiting until it is required for the next assignment.<br />
QUESTION 2: IN COMPARISON TO OTHER FEEDBACK THIS METHOD IS DIFFERENT; DID IT<br />
FEEL MORE PERSONAL TO YOU?<br />
The students on this module normally receive traditional written feedback from other tutors so<br />
answers to this question indicate high approval from the students for the personal aspect of the<br />
feedback. They receive feedback in a personal folder; the audio was personally addressed to them<br />
260
Rachel Fitzgerald<br />
too. When comparing those that replied yes to this against the qualitative data, it is notable that the<br />
students repeat the word personal to describe their general thoughts on receiving audio feedback.<br />
Providing a personal touch appears to be a positive outcome of using this form of feedback<br />
Figure 4: Representative of responses to question 2<br />
Comments from Students who found the method more personal<br />
“I found it was more personal to me and gave more information on how I could make my<br />
assignments better. I think that it is a very good method”<br />
“it was so clear and personal and helped me understand where I went wrong with my<br />
assignment!!.”<br />
QUESTION 3: DID THE FEEDBACK SUGGEST WAYS OF IMPROVING YOUR WORK THAT YOU<br />
PLAN TO USE FOR YOUR NEXT ASSIGNMENT<br />
Figure 5: Representative of responses to question 3<br />
The response mechanism used for Q3 is a Likert scale. Most of the responses to this question were in<br />
the positive range, students indicated that the feedback had offered ideas to improve their work Not<br />
many of the students that responded negatively to this question added qualitative feedback, however<br />
one student that did, indicated that they were not fully aware of the purpose of the feedback.<br />
Comment from Student about being suggested ways of improving work<br />
Would be good to be told what can be done to get a higher grade on the next<br />
assignment”<br />
261
Rachel Fitzgerald<br />
Although this is only one comment, this may be something to look at in terms of the language used in<br />
feedback. Perhaps there is a need to be absolutely specific in the feedback.. eg to get a higher grade<br />
you will need to … but then again as Gibbs (1981) acknowledges, students often don’t follow advice<br />
even when it is given with the best intentions.<br />
QUESTION 4: DID YOU RE-READ YOUR ASSIGNMENT HAVING LISTENED TO THE<br />
FEEDBACK?<br />
Figure 6: Representative of responses to question 4<br />
Responses to Q4 suggest that many students did not re-read their assignment either along with or<br />
after hearing the feedback. It would be interesting to ask a similar question to students who had<br />
received written feedback to see if the responses are similar, Gibbs and Simpson (2004) suggest that<br />
students are not interested in feedback if they perceive there is requirement for them to take action<br />
but it was hoped that use of audio would change this. As the audio file and the original submission are<br />
separate from each other, this has proven to be ineffective at encouraging (all) students to read their<br />
work with new understanding. It remains to be seen if the feedback has an impact on the second<br />
assignment.<br />
QUESTION 5: DID YOU FEEL YOU HAD GUIDANCE THAT WILL HELP YOU WITH YOUR NEXT<br />
ASSIGNMENT FOR THIS MODULE?<br />
Figure 7: Representative of responses to question 5<br />
This question was actually similar to Q3, so unsurprisingly, the answers are also similar (a pilot survey<br />
would have highlighted this) however the corresponding qualitative comments are more detailed.<br />
Comments from Student when asked if they had guidance to help with next assignment<br />
262
Rachel Fitzgerald<br />
“Was rather unpleasant as I had to wait a while before you stated my overall grade.<br />
Perhaps you should have stated my grade and then move on to explain why the piece of<br />
work deserved it.”<br />
This was an interesting observation; the grade was deliberately left until the end so that the student<br />
had to listen to the feedback in order to get a grade to avoid the scenario of just accessing the grade<br />
and ignoring everything else.<br />
Comments from Student when asked if they had guidance to help with next assignment<br />
“Audio files are all very well and good, however simply take up such amount of space<br />
and time and are not as beneficial as say yourself passing back the assignment with<br />
comments (too many lecturers with handwriting that cannot be read). If students require<br />
further help, they can request for your time during a seminar. Doing all of the audio files<br />
must have taken you hours.”<br />
While delighted with the concern, this highlights a crucial point – would students be as happy with<br />
written feedback as long as it was typed? Merry and Orsmond (2007) found the majority of students<br />
engaged on their project were happy with just audio, only two from fifteen students mentioned wanting<br />
both written and audio together but this research is showing that the use of audio is not having any<br />
transformative effect.<br />
Comments from Student when asked if they had guidance to help with next assignment<br />
“I consider that his method is very informative and personal. The comments are brief but<br />
however concise and do highlight areas of improvement. I personally took more notice of<br />
this than sometimes assignments being handed back in a written format - not always<br />
digestible and often too many comments to take on board”<br />
If all comments matched this, audio feedback would be the winner! Kerr & McLaughlin (2008) had<br />
similar feedback from their students, receiving comments like “I properly listened to what the marker<br />
was saying” but this is not a common response from all participants in the study.<br />
QUESTION 6: WOULD YOU CONSIDER YOURSELF TO BE AN INTERNATIONAL STUDENT?<br />
Thirteen students identified themselves as an international student; their corresponding qualitative<br />
comments included:<br />
Comments from International Students about receiving Audio Feedback<br />
“Because I am poor in English, I still have some words cannot understand immediately<br />
and clearly. However, I think that this make me feel more personal to me and I am happy<br />
in the radio.”<br />
“I prefer my feedback with hard copy”<br />
“it was all right”<br />
“i really liked it and i think it was much better than any other feedback so far.”<br />
“I like the idea of giving our feedback on an audio file. The feedback was clear and<br />
satisfying.”<br />
“For me, there is no difference between audio and text feedback.”<br />
Overall there are mixed responses from the international students about audio feedback, but the<br />
personal aspect comes up again here. This is a very small sample of students so research is required<br />
before recommending this method as for DL students.<br />
QUESTION 7: DO YOU HAVE ANY OTHER COMMENTS ABOUT AUDIO FEEDBACK?<br />
Other General Qualitative Comments about Audio Feedback<br />
“I like the way feedback was given”<br />
“I would rather prefer a written evaluation than and oral evaluation in form of the audio<br />
file”<br />
263
Rachel Fitzgerald<br />
“The survey was helpful in a way but it didn't make things clear in depth.. I'd prefer a<br />
written feedback instead for the same reason that my mistakes were not justified in a<br />
clear manner.. overall satisfied with it!”<br />
“it felt a bit brief, I felt that had it been written there would have been comments<br />
throughout the assignment and therefore more feedback.”<br />
“For me, there is no difference between audio and text feedback”<br />
Of the rest of general qualitative comments the response is again not particularly in favour of audio, if<br />
anything it suggests that written feedback is more purposeful for these students. It would be<br />
interesting to have measured whether these responses came from students that received the initial<br />
recordings whether tone may have been more enthused or the end of the recordings when the tone<br />
may have been more perfunctory.<br />
8. Conclusions and general reflection<br />
The JISC guide to effective practice lists digital audio as a tool for “improving the quality of feedback”<br />
JISC (2009 p19) but as Rust (2001) advises, when class sizes go up there is likely to be more<br />
“superficial feedback” returned to the student. This research established that this is also the case<br />
when using audio feedback although it is packaged in a more personal way. Feedback from the<br />
students through the various data collection methods suggests that they found the experience more<br />
personal and therefore valued the approach over the more traditional scrawl of handwritten feedback.<br />
Kerr & McLaughlin (2008) suggest that regardless of feedback style, quality personal feedback makes<br />
students feel cared for and that they are thought of as individuals, and as this is difficult to achieve in<br />
large cohorts, this must be something that needs to be considered, when looking at method. The next<br />
assignment on this module will receive traditional feedback as the numbers are too great to go with<br />
audio feedback again. If I were to use audio again, I think I would prefer to work with a smaller cohort<br />
and evaluate the impact across different assignments for the same module as I think the quality of the<br />
recordings would be better and less repetitive and it would offer opportunity to consider the impact of<br />
tone on how the student receives the feedback. Considering a student <strong>learning</strong> styles and their<br />
reception to method of feedback may also warrant further study. In a future cycle I would look for an<br />
alternative to the VLE, possibly Dropbox or another Web 2.0 shared space, and find a method that<br />
incorporated audio and text. This research suggests that audio feedback alone would not support an<br />
international DL student but needs a combination of audio and written feedback, so a future project<br />
may consider the Fish & Lumadue (2010) project that uses Jing, software that enables a tutor to<br />
merge a text response with audio, something they recommend as a 21 st century yet simple approach.<br />
Kerr and McLaughlin (2008) say that the tutors involved in their research would not repeat their<br />
attempt at video feedback. The general view, they report, is that the exercise was pointless in terms of<br />
time and benefit. From this cycle, I would say that Audio Feedback is a simple method but more<br />
suited to smaller groups of students. It can be adopted with ease and requires very little technical<br />
skills. The process of recording feedback is faster than writing it and it offers a personal response to<br />
students that is hard to achieve with any written method. Brown (2001) says one of the problems with<br />
feedback is that it is not always there when students need it; digital audio offers a solution to this, as it<br />
can be listened to anytime, although this could also be the case with typed electronic feedback. What<br />
does set Audio Feedback apart is the voice and tone of the tutor talking to the student, making<br />
feedback less ambiguous and providing a level of personalisation that fits in with NUS<br />
recommendations and gained the approval of the students involved in this research and even if the<br />
only benefit is that the student feels the marking is personal, then the technique is worth further<br />
investigation, Kerr & McLaughlan (2008).<br />
References<br />
Bols, A. (2010) What Students Want: feedback on assessment; Educational Developments; SEDA; 11:4; 7<br />
Brown, G. (2001) Assessment: A Guide for Lecturers. LTSN Assessment Series: No 3; York, LTSN Generic<br />
Centre.<br />
Copley, J. (2007) Audio and Video Podcasts of lectures for campus-based students: production and evaluation of<br />
student use; Innovations in Education and Teaching International; 44:4; 387-399<br />
Ferrance, E. (2000). Action research. Providence, RI: Brown University, Northeast and Islands Regional<br />
Educational Laboratory.<br />
Fish, W., Lumadue, R. (2010) A Technologically Based Approach to Providing Quality Feedback to Students: A<br />
Paradigm Shift for the 21 st Century; <strong>Academic</strong> Leadership The Online Journal [online]<br />
http://www.academicleadership.org/article/A_Technologically_Based_Approach_to_Providing_Quality_Feed<br />
264
Rachel Fitzgerald<br />
back_to_Students_A_Paradigm_Shift_for_the_21st_Century<br />
Accessed (28/1/11)<br />
Gibbs, G. (1981) Teaching Students to Learn: A student centred approach. Milton Keynes: Open University<br />
Press.<br />
Gibbs, G., & Simpson, C. (2004) Conditions under which Assessment Supports Students’ Learning: Learning and<br />
Teaching in Higher Education; 1:1<br />
Harvey, J. & Mogey, N. (1999). Pragmatic issues when integrating technology into the assessment of students: In<br />
Brown, S., Race, P. & Bull, J. (1999) (Eds), Computer-assisted assessment in higher education; London;<br />
Kogan-Page.<br />
Holmes, K., Papageorgiou, G. (2009) Good Bad and Sufficient: Students’ expectations, perceptions and uses of<br />
feedback; Journal of Hospitality, Leisure, Sport & Tourism Education; 8:1: 85-96<br />
JISC (2009) Effective Practice in a Digital Age; Bristol; JISC Innovation Group<br />
Kaplan-Leiserson, E. (2005) Podcasting in <strong>Academic</strong> and Corporate Learning. Learning Circuits [online]<br />
https://www.astd.org/LC/2005/0605_kaplan.html, Accessed (8/1/11)<br />
Kemmis, S; MacTaggart, R. (2005) Participatory Action Research: Communicative action and the public sphere<br />
in Denzin, N. and Lincoln, Y. (Eds) The Sage Handbook of Qualitative Research, London, Sage.<br />
Kerr, W., McLaughlin, P. (2008) The Benefit of Screen Recorded Summaries in Feedback for Work Submitted<br />
Electronically; Loughborough University [online] http://hdl.handle.net/2134/5363<br />
Accessed (16/1/11)<br />
Koshy, V. (2005) Action research for improving practice: A practical guide. London: Paul Chapman Publishing.<br />
Merry, S., Orsmond, P. (2007) Students’ Attitudes to and Usage of <strong>Academic</strong> Feedback Provided Via Audio Files;<br />
Bioscience Education eJournal [online] http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.pdf,<br />
Accessed (25/1/11)<br />
NUS (2010) Charter of Feedback and Assessment [online]<br />
http://www.nusconnect.org.uk/asset/news/6010/FeedbackCharter-toview.pdf, Accessed (30/1/11)<br />
O’Leary, Z. (2004) The Essential Guide to Doing Research. London: Sage<br />
O’Leary, Z., (2005) Researching Real World Problems. London: Sage<br />
Price, M. (2007) Should we be giving less feedback; Centre for Bioscience Bulletin [online]<br />
http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf Accessed (20/1/11)<br />
Rani, N., Yahya, A. (2009) Engaging Learners in their studies via feedback; Teaching and Learning Open Forum<br />
[online] http://www.curtin.edu.my/TLForum2009/images/TLOF09-18.pdf Accessed (20/12/10)<br />
Rotherham, B. (2008) Using an MP3 to give feedback on student assignments; Educational Developments;<br />
SEDA; 8:2 [online] http://sites.google.com/site/soundsgooduk/downloads/MP3_recorder_for_feedback.pdf?<br />
Accessed (10/1/11)<br />
Rotherham, B. (2009a) It wasn’t me, guv!; Educational Developments. SEDA; 10:1<br />
Rotherham, B. (2009b) Sounds Good: Quicker, better assessment using audio feedback; JISC Final<br />
http://www.jisc.ac.uk/media/documents/programmes/usersandinnovation/sounds%20good%20final%20repo<br />
rt.doc, Accessed (15/1/11)<br />
Rust, C. (2001) A Briefing on Assessment of Large Groups, Assessment Series No 12; York; LTSN Generic<br />
Centre.<br />
Saunders, M., Charlier, B., Bonamy, J. (2005) Using Evaluation to create ‘Provisional Stabilities’; Evaluation the<br />
International Journal of Theory, Research and Practice; 11:1; 37-54<br />
265
Messages of Support: Using Mobile Technologies to<br />
Support the Transition of Students on Articulation Routes<br />
From Higher National Level to Degree<br />
Julia Fotheringham and Emily Alder<br />
Edinburgh Napier University, Edinburgh, UK<br />
j.fotheringham@napier.ac.uk<br />
em.alder@napier.ac.uk<br />
Abstract: This paper explores the role mobile technologies can play in supporting students’ transition to second<br />
and third year of university degree study along articulation routes from Higher National Certificate (HNC) and<br />
Higher National Diploma (HND) study at college. Articulating students face particular challenges associated with,<br />
typically, adjusting to a step up in the level of their academic studies, acclimatising to an unfamiliar academic<br />
culture, and integrating into an existing cohort of students. ‘Message of Support’, a project funded by the<br />
Edinburgh, Lothians, Fife and Borders Regional Articulation Hub (ELRAH), developed a range of SMS, podcasts,<br />
and DVD resources, drawing on the voice and experience of existing students, in order to support new students<br />
and staff in their respective parts in the articulation journey. Through a process of action research, it was found<br />
such resources can aid the transition process by offering timely reassurance and information to students as well<br />
as valuable development materials for staff. Responses to the challenges of using mobile technologies as<br />
support mechanisms for articulating students were also identified.<br />
Keywords: articulation; student support; SMS; mobile technologies; podcast; ELRAH<br />
1. Introduction<br />
The creation of articulation pathways into university has become a strategic priority for many Higher<br />
Education Institutions (HEIs). Articulation describes a particular route through higher education by<br />
which an initial qualification (in Scotland, usually HNC or HND qualifications at Scottish Curriculum<br />
Quality Framework (SCQF) levels 7 and 8 respectively ([SCQF 2009]) gains a student access to a<br />
university degree with advanced standing, directly into second or third year. Such progression often<br />
reflects a formal agreement or partnership between institutions (Gallacher 2006). A successful<br />
articulation route relies not only on careful curricular alignment between the courses in the two<br />
institutions but also depends upon staff at both ends of the articulation journey ensuring that students<br />
are as well prepared as possible to respond to the academic and personal challenges that await them<br />
upon arrival at university (MacAskill 2010).<br />
This paper explores the role of mobile technologies in helping to address some of the challenges<br />
faced by articulating students. The Message of Support project, funded by the Edinburgh, Lothians,<br />
Fife and Borders Regional Articulation Hub [ELRAH] (see www.elrah.ac.uk/Elrah.htm) developed<br />
podcasts, SMS texts and a DVD to provide point-of-need support for articulating students from<br />
Scottish colleges, and professional development for academic staff at Edinburgh Napier University.<br />
While ELRAH is a partnership of many institutions, the Message of Support was focussed on<br />
Edinburgh Napier University.<br />
In this paper we describe how interviews with students who had made successful articulation journeys<br />
to Edinburgh Napier University were used to create a series of podcasts for new students and a DVD<br />
resource for professional development for academic staff. We also discuss our experience of using<br />
SMS texts to encourage students to engage with various strands of academic support provision. For<br />
many students, the first few weeks in a new University environment can result in a severe loss of<br />
confidence (Christie et al 2008) which may result in withdrawal from their programme. Our experience<br />
at Edinburgh Napier confirms that students welcome advice and encouragement from other students<br />
in whose footsteps they follow, recognize the value of structured support such as academic skills<br />
workshops, but also feel strongly about the contribution of academic staff to restoring and building<br />
their confidence and helping them to adjust to the unfamiliar academic and social demands of<br />
university life. Our project confirmed that the deployment of a range of different technologies at<br />
different stages in the articulation journey and amongst the different partners in the articulation<br />
process can enhance a student’s experience of joining a degree programme as a direct entrant.<br />
266
2. Background and context<br />
Julia Fotheringham and Emily Alder<br />
2.1 Supporting student transitions to university study<br />
New university students of all kinds face challenges as they make the transition to university study<br />
(Lowe and Cook 2003). For Scottish undergraduates, the traditional ‘first year’ (SCQF Level 7) is a<br />
period during which students build social networks, familiarise themselves with institutional systems<br />
and learn how to learn as a university student (Tait and Godfrey 2001). Research has shown that<br />
direct entry students articulating to year 2 or year 3 (SCQF Level 8 or 9) face particular challenges.<br />
These include integration into a cohort of students who have already learned together for one or two<br />
years, the step up to a more advanced level of study, and acclimatisation to a different academic<br />
culture.<br />
Cree et al (2009: 896) describe a picture of ‘dislocation and loss’ amongst students struggling to<br />
adjust to degree study. Barron and d’Annunzio-Green (2009) point to a catch-up phase in which<br />
students not only must adjust to these differences but familiarise themselves with the one or two<br />
previous years of <strong>learning</strong> undertaken by their peers; they highlight the importance of maintaining<br />
student support and building students’ academic self-confidence during the early months of their<br />
degree study. However, since articulating students progress from an HNC or HND, their academic<br />
career begins long before they join the university, and thus developing academic skills and confidence<br />
must also begin well before that point (MacAskill 2010). Pike and Harrison (2011) emphasise the<br />
importance of a ‘smooth’ transition, to which good communication between institutional partners is<br />
essential. Continuity of support between levels is thus one of the keys to successful transition.<br />
Amongst the difficulties identified by articulating students in this research was lack of awareness of<br />
what to expect from university study. New students attach considerable value to hearing from<br />
previous articulating students about their experiences (MacAskill 2010; Kivlichan and Chirnside 2011).<br />
Further, Pike and Harrison (2011) suggest that raising awareness among new direct entrants that<br />
their concerns are shared by peers could reduce feelings of isolation.<br />
Research points to the key role that academic staff play in supporting the student transition. This<br />
indicates a need for staff development so that academic staff are aware of articulating students in<br />
their classes and equipped to address their needs. Bingham and O’Hara (2007) found that students<br />
felt some of their Higher Education (HE) lecturers were unaware of the prior <strong>learning</strong> they had<br />
experienced; Pike and Harrison (2011) recommend that university teaching teams should be aware of<br />
new direct entrants in their classes and be prepared to respond to their needs.<br />
2.2 Institutional context<br />
At Edinburgh Napier University, student feedback has indicated a similar range of challenges<br />
associated with direct entry to university. While the majority of direct entry students settle in well to<br />
university in the first trimester, a recent survey found that direct entrants were much more (46%) likely<br />
to have experienced difficulties with the transition to university than year one students (18%)<br />
(Kivlichan and Chirnside 2011). In particular, issues were encountered with use of IT systems,<br />
challenges with particular modules, producing lab reports, time management, and personal or<br />
financial matters. When asked what further help the university could have provided, responses<br />
included more information (e.g. timetables, course content, and explanations of the amount of work<br />
required) provided while students were still at college, examples of assignments, and hearing from<br />
existing direct entry students about their experiences (Kivlichan and Chirnside 2011: 14).<br />
Research by Howieson and Croxford (2011) into the experience of direct entry students across four<br />
universities (Edinburgh Napier, Heriot-Watt, Queen Margaret and Stirling) in the ELRAH partnership<br />
supports these findings. After three months of degree study, direct entry students to year two or three<br />
were less likely (47%) to feel very or quite well prepared for their course than college students<br />
entering year one (72%). Direct entrants felt that advance preparation before starting university was<br />
key, including awareness raising about available support, experience of aspects of degree study, and<br />
integration with existing students. Online information and materials were also resources for many<br />
students (Howieson and Croxford 2011). Mobile technology, therefore, becomes a crucial tool in<br />
enabling models of transitional support to be consistently available to students passing to a new stage<br />
of their education.<br />
267
Julia Fotheringham and Emily Alder<br />
2.3 Role of mobile technology in student support<br />
Traxler and Kukulska-Hulme (2005: 1) acknowledge the potential for mobile <strong>learning</strong> to address the<br />
needs of ‘the new constituency of learners’, given the flexibility and near ubiquity of mobile<br />
technologies. Data from a recent Edinburgh Napier survey of student usage of mobile technologies<br />
(2010, unpublished) confirms this; 99.5% of our survey respondents own a mobile phone/device.<br />
Recent mobile student support pilot studies in universities and colleges reveal three main consistent<br />
themes; first, students value the use of mobile technologies in their interactions with the University<br />
(Riordan and Traxler 2005,); secondly they feel socially connected by these interactions (Lunsford<br />
2010; Mentor 2011) and thirdly, institutional strategies for mobile student support are likely to feature<br />
different mobile devices and materials (Lunsford 2010).<br />
Riordan and Traxler (2005) highlight technical and pedagogical issues around effective and<br />
appropriate use of SMS texting. Student acceptance and engagement is enhanced where SMS<br />
interventions are short, personalised and focussed (Riordan and Traxler 2005). Elsewhere, SMS<br />
student-support initiatives are motivated by differing administrative and pedagogical priorities. The UK<br />
Council for International Student Affairs (http://www.ukcisa.org.uk/pmi/case_studies_support.php)<br />
provide a collection of best practice case studies which describe projects where podcasts and SMS<br />
are used to provide support for international students. For example, the Loughborough College<br />
initiative is designed to assist its International Office to keep touch with students, whereas Sheffield<br />
Hallam’s initiative responds to international student demand for support materials on arrival in the UK.<br />
While these case studies provide examples of mobile student support practice, the students’<br />
experiences of these initiatives are not described in any detail.<br />
Mentor (2011) explores the potential for SMS texts to offer a sense of social connectedness to<br />
students experiencing isolation from their college community. Although connections between the<br />
perception of emotional well-being and improved academic performance still require further<br />
exploration (Mentor 2011), Tinto’s seminal model of student integration (1975) continues to provide a<br />
frame of reference for understanding the connections between social interaction, integration with<br />
academic systems and student retention. Models of student retention and engagement have evolved<br />
since then (Tinto 2007) to reflect a broader range of institutional contexts and student circumstances<br />
including online and distance <strong>learning</strong> students (Rovai 2002), but the concept of involvement, (or<br />
engagement) remains a key element for almost all mobile-mediated student support initiatives.<br />
The benefits of mobile accessible materials for enhancing social connectedness for distance <strong>learning</strong><br />
students at the Open University (OU) (Lunsford 2010) appears to be broadly consistent with Mentor’s<br />
(2011) findings. The OU investigation comprised several strands of activity each making different use<br />
of mobile technologies and <strong>learning</strong> materials. Lunsford (2010) observes the important affective<br />
impact of the interactions mediated by mobile technologies, where students report feeling more<br />
connected and involved with the organisation as a result. An important outcome of that project is a<br />
‘model of student support using handheld technologies’ (Lunsford 2010). This model offers a flexible<br />
but coherent framework recognizing that different parts of the organisation will make use of mobile<br />
technologies in different ways. A particular mobile device may be more useable in one context than in<br />
another (Kukulska-Hulme 2005).<br />
2.4 Action research<br />
A commitment to improvement is of paramount importance in any action research project (McNiff and<br />
Whitehead 2010). Despite the differences amongst the various typologies of action research offered<br />
in the literature, most share a common way of thinking about the role of the researcher as practitioner<br />
(Cohen 2000), the value of collaboration in offering differing perspectives (Reil 2011), the cyclical and<br />
systematic nature of enquiry that is undertaken in the name of action research (Norton 2009) and the<br />
inherent contribution that transparent reflection makes to every part of the research and its outcomes<br />
(McNiff and Whitehead 2010).<br />
3. Methodology<br />
Messages of Support explored the potential of mobile technologies to improve the transition<br />
experience of students coming to university as direct entrants from Scotland’s colleges. Messages of<br />
support, information and encouragement were embedded in a suite of podcasts and in a timeline of<br />
SMS texts messages. The target audience for podcasts and SMS were students in Lothians, Fife and<br />
Borders colleges, as well as students just arrived at Edinburgh Napier University. Locating the<br />
268
Julia Fotheringham and Emily Alder<br />
podcasts within the University student portal would have made them inaccessible to college students<br />
given the need for a matriculation identity. As a result, we hosted the podcasts on Podbean, an openaccess<br />
podcast hosting service, and distributed the text messages from an SMS server based in<br />
Edinburgh Napier University, accessible online from various college locations. Action research,<br />
carried out by practitioners on their own practice, presented a strong methodological framework with<br />
which to underpin our collaborative exploration of the potential benefits of mobile accessible materials<br />
for transition support. We established a project team of practitioners from colleges and University and<br />
although the membership of the team varied as the project developed, the mixture of expertise and<br />
professional focus ensured that at every stage the reflective process was collaborative and<br />
participative. Norton’s (2009) five-step action research process provided the most useful way of<br />
sharing the practicalities of our different but related reflective cycles of activity. We started by<br />
producing a suite of podcasts for students (Table 1) and generating a timeline of SMS texts (Table 2,)<br />
then out of the experience of Reflective Cycles 1 & 2 we produced a DVD for staff (Table 3).<br />
Table 1: Reflective cycle 1 (podcasts)<br />
Step 1 Identifying the issue Research provided evidence of the challenges that students face joining<br />
university as direct entrants from college.<br />
Step 2 Thinking of ways to Interest in mobile technologies and the rise of mobile <strong>learning</strong> initiatives<br />
tackle the issue across the university and the HE sector provided a catalyst to develop a<br />
series of podcasts for direct entrants.<br />
A project team of academics and student support staff in colleges and the<br />
university established the podcast topics (see Figure 1), which<br />
complemented other existing university student support podcasts. This<br />
project sought to recognize the value of student voice in providing<br />
reassurance and authentic material for the podcasts.<br />
Step 3 Doing it Students who had joined second or third year of undergraduate degree<br />
programmes were interviewed. The interview questions were informed by<br />
Tinto’s model of student integration (1975).<br />
We asked questions about our interviewees’ experience of academic<br />
integration during their first trimester (whether or not they were enjoying<br />
their subject and coping with academic assignments) and in relation to<br />
social integration (whether or not they had friends and the nature and<br />
extent of the contacts that they had with university staff). We asked them<br />
what advice they would give to students who would follow them from<br />
college to university.<br />
Student interviews were recorded, edited and developed into a suite of<br />
podcasts.<br />
Podbean site (http://college2uni.podbean.com) was developed and<br />
podcasts were launched on a time-released basis.<br />
Publicity posters helped to publicize the initiative.<br />
Step 4 Evaluating it We counted the number of visits to the site, and the number of ‘hits’ on<br />
each of the podcasts.<br />
Each podcast has an online ‘comments’ form on which participants were<br />
invited to submit feedback or to suggest other podcasts.<br />
Invitation to participate in interviews or focus group discussions<br />
Informal comment and feedback from members of the project team<br />
Step 5 Modifying practice We address this in more detail in the Discussion. There is no doubt that<br />
when using open access platforms such as Podbean, it is essential for<br />
evaluation purposes to find ways of making contact with the students who<br />
are likely to be accessing the site.<br />
Table 2: Reflective cycle 2 (SMS texts)<br />
Step 1 Identifying the<br />
issue<br />
Step 2 Thinking of ways to<br />
tackle the issue<br />
As for podcasts in Table 1.<br />
As part of other projects, ELRAH staff produced a timeline of interventions to<br />
facilitate successful articulation. A series of SMS text messages could be<br />
deployed across the span of that timeline, from the process of application to<br />
university until the end of the first university trimester.<br />
The working group drew upon the expertise of the University’s Lead IT<br />
Developer. The University’s SMS interface system enables bulk text<br />
messages to be sent directly to individual students’ personal mobile<br />
numbers. The SMS system is web-enabled, and access is thus not restricted<br />
269
Julia Fotheringham and Emily Alder<br />
to the university campus. This enables both college and university staff to<br />
operate the system to reach students at all stages.<br />
Step 3 Doing it A series of SMS text messages was designed according to the timeline<br />
described in Step 2. The SMS messages aimed to support students’ sense<br />
of belonging to the university and to reinforce their induction programme with<br />
reminders of activities such as academic skills workshops or where to find<br />
advice on exam techniques.<br />
Students in the target group at college or direct entrants to university were<br />
invited to register by sending a text message to the SMS system. Students<br />
would receive no more than 2 SMS messages each month, between July<br />
and December.<br />
Eleven messages were issued between July and December, variously<br />
targeted to students while at college, and then after matriculation to<br />
university<br />
Step 4 Evaluating it Analysing the number and range of students taking part in the initiative.<br />
Evaluations from project team on the method and management of the SMS<br />
interface.<br />
Step 5 Modifying practice We explain in the discussion how we have adapted our approach for the<br />
academic session 2011/12.<br />
Table 3: Reflective cycle 3 (DVD)<br />
Step 1 Identifying the<br />
issue<br />
Step 2 Thinking of ways to<br />
tackle the issue<br />
The first round of interviews (for the podcasts) told us that some articulating<br />
students reported serious difficulties during their first year as direct entrants,<br />
including social isolation and problems with assimilating the HE academic<br />
culture.<br />
We recognized that the student narratives could make a powerful impact on<br />
academic staff. A DVD was selected as a suitable format for a professional<br />
development resource.<br />
We anticipated that the material on the DVD was likely to be sensitive and<br />
not appropriate for student use, so in order to limit access, the resource has<br />
not been web-enabled.<br />
Step 3 Doing it Both student and staff accounts were used in order to maximise the impact<br />
of the DVD. We re-interviewed and filmed students from the podcast<br />
interviews. We also interviewed one new member of academic staff who had<br />
valuable experience of working with groups of students that included a<br />
significant proportion of direct entrants.<br />
The DVD has been used for the purpose of <strong>Academic</strong> Induction and as a<br />
resource for CPD for staff in the university.<br />
Step 4 Evaluating it Feedback from participants at <strong>Academic</strong> Induction and from members of the<br />
project team<br />
Step 5 Modifying practice Use of the DVD with new groups of staff and with programme teams will be<br />
monitored and re-evaluated throughout the forthcoming academic session.<br />
We explore this further in the discussion.<br />
4. Results<br />
The evaluation of each of the three cycles of activity represented the most challenging aspect of our<br />
project. The table below takes Step 4 (Evaluating it) from each of the reflective cycles documented<br />
above and illustrates the results associated with each of the evaluation methods.<br />
Table 4: Results for reflective cycle 1 (podcasts)<br />
Evaluating it Results<br />
Counting visits to the site and<br />
hits on each podcast<br />
Comments forms for each<br />
podcast<br />
Invitation to participate in<br />
interviews or focus groups to<br />
Between January and July 2011 the College2uni Podbean site<br />
(http://college2uni.podbean.com/) received 1618 visits. The number of<br />
‘hits’ on the podcasts vary from the least popular (Staying on for<br />
Honours) with 13 hits and the most popular (Personal Development Tutor<br />
[PDT]) with 337 hits.<br />
None were completed.<br />
In May an email was issued to all 2 nd and 3 rd year students who came to<br />
Edinburgh Napier University from College in 2010, inviting them to offer<br />
270
discuss their experience of the<br />
podcasts or to provide feedback<br />
by way of email.<br />
Julia Fotheringham and Emily Alder<br />
feedback about their experience of the ‘college2uni’ podcasts. We<br />
received no responses.<br />
Evaluating it Results<br />
Feedback from articulation<br />
support staff and from others<br />
working in Scotland’s colleges<br />
Three articulation support advisors who had actively used the podcasts<br />
responded to the invitation to provide feedback. A consistent theme<br />
emerges from their responses around the importance attached by<br />
students to the student-voice perspective of the podcasts:<br />
‘I think it’s beneficial to hear this information first hand, as in my<br />
experience students will listen to staff, but tend to believe other students.’<br />
The podcasts were well-received by staff working in central support roles<br />
in ELRAH partner colleges. We received requests from all of the colleges<br />
for the publicity posters and several colleges sought permission to<br />
upload the podcasts on their own student portals.<br />
Figure 1: Number of hits per podcast<br />
Table 5: Results for reflective cycle 2 (SMS)<br />
Evaluating it Results<br />
Participants 9 students registered with the SMS system. Of these, 3 articulated to<br />
Edinburgh Napier (plus 1 to year one), 1 to Heriot-Watt, and 2 to an<br />
Edinburgh Napier degree in Carnegie College. The remaining 2 did not<br />
go on to university. This uptake rate was disappointingly low, and<br />
participation will be more actively promoted in the future.<br />
The range of student destinations, however, demonstrated the system’s<br />
potential to reach a diverse and dispersed student population.<br />
A closing message requesting feedback on the messages elicited no<br />
responses.<br />
Administration Participation relied on students contacting the SMS system. In future,<br />
student mobile numbers will be collected and entered, with their<br />
permission, in advance.<br />
Interface Administrators found the interface to be straightforward to use. It allowed<br />
students to be variously grouped (e.g. by year of study, college of origin,<br />
or university course) for ease of targeting messages.<br />
271
Table 6: Results for reflective cycle 3 (DVD)<br />
Julia Fotheringham and Emily Alder<br />
Evaluating it Results<br />
Feedback from academic<br />
induction<br />
5. Discussion<br />
5.1 Podcasts<br />
Feedback from induction participants was positive. Comments were<br />
made about the powerful impact of hearing actual student stories. Other<br />
participants had not previously considered their role in supporting direct<br />
entry students prior to viewing the DVD.<br />
Figure 1 displays the distribution of hits on each podcast. One podcast, ‘What is a PDT?’, received<br />
nearly 5 times as many hits as the next most popular. We speculate this may have been caused by<br />
the acronym and the podcast’s title being posed in the form of a question. If so, this may point to the<br />
importance of demystifying jargon such as PDT (personal development tutor). The other podcasts<br />
average 38 hits. Allowing for the uncertainty over how many users were students, this may suggest<br />
the need of more active and directed promotion of the podcasts to the target group in the future.<br />
As noted in Table 4, the lack of qualitative data from students means we are unable to analyse how<br />
and why students (if indeed they were students) accessed the podcasts, whether or not they<br />
subscribed to the whole series, why there was such a variation in the number of hits on each podcast<br />
(from 13 to 337), nor, perhaps most importantly, how valuable they found the form and content of the<br />
series. We were pleased to support colleges to host the podcasts within their own institutional virtual<br />
environments. However, given the dispersed location of the podcasts across various student portals,<br />
and the anonymity of the College2uni podbean site users, evaluating the usage and value of the<br />
podcast series became unachievable.<br />
5.2 SMS<br />
The SMS project was successful in testing the bulk mailing system, which its administrators found<br />
reliable and straightforward to use. Student recruitment to the SMS system proved a challenge.<br />
Although 6 of the 9 students participating in the initiative progressed to enrol as undergraduates at<br />
Edinburgh Napier University, the sample size is too small to draw any conclusions about its impact.<br />
Students were required to register their mobile number by sending a text message to the system, but<br />
very few students actually took this step despite having expressed interest in taking part. As a result,<br />
opportunities to assess the impact of the SMS support in 2010/11 were limited. Consequently, in<br />
preparation for 2011/12, mobile phone numbers are being collected in advance for entry by the<br />
system’s administrator. The effect of this change is immediately evident: at the time of writing (July<br />
2011), around 80 participants have provided mobile numbers. This indicates that the success of this<br />
strand of the project is dependent not on the technology but on its management.<br />
5.3 DVD<br />
We expected to make regular use of the DVD at academic induction, but the small numbers of new<br />
academic staff and the lack of induction events mitigated against this. However, at the start of<br />
academic session 2012/13 we expect to respond to requests from programme teams who will be<br />
using the resource in preparation for the new intake of direct entrants in September 2012. This should<br />
provide us with ongoing evaluation data.<br />
6. Conclusions<br />
In the Message of Support project, the use of podcasts made the voice and experience of existing<br />
students readily and widely available online to all, circumventing the restrictions of access often<br />
associated with institutional logins.. The podcasts and SMS messages can also promote a sense of<br />
belonging and connectivity to the institution, as well as raising awareness of key topics known to<br />
present challenges to students. Student feedback has highlighted the importance of staff awareness<br />
of the articulating student experience, and the DVD promotes a wider understanding amongst<br />
academic staff of the needs of this student group.<br />
The project offers support that is consistent, inclusive, and longitudinal, spanning the college to<br />
university transition. Mobile technologies offer real advantages by making it as easy as possible for<br />
272
Julia Fotheringham and Emily Alder<br />
students to engage on their own terms with open access resources. However, as use of open access<br />
resources grows more widespread, familiar monitoring and evaluation techniques may increasingly<br />
prove to be insufficient. In the rapidly evolving world of mobile technologies, new approaches to<br />
managing these initiatives are required to ensure that their potential for student support can be fully<br />
exploited.<br />
References<br />
Barron, P. and d’Annunzio-Green, N. (2009) ‘A smooth transition? Education and social expectations of direct<br />
entry students’, Active Learning in Higher Education, Vol. 10, No. 7, pp. 7-25.<br />
Bingham, R. and O’Hara, M. (2007) ‘Widening participation on early years degrees: “I realised I could, and would,<br />
do this – and I have!”’, Journal of Further and Higher Education, Vol. 31, No. 4, pp. 311-321.<br />
Christie, H., Tett, L., Cree, V., Hounsell, J., and McCune, V. (2008) ‘“A real rollercoaster of confidence and<br />
emotions”: <strong>learning</strong> to be a university student’, Studies in Higher Education, Vol. 33, No. 5, pp. 567-581.<br />
Cohen. L., Manion, L. & Morrison, K. (2000). Research Methods in Education, 5 th edition, Oxon:<br />
RoutledgeFalmer.<br />
Cree, V., Hounsell, J., Christie, H., McCune, V., and Tett, L. (2009) ‘From Further Education to Higher Education:<br />
Social Work Students’ Experience of Transition to an Ancient, Research-Led University’, Social Work<br />
Education, Vol. 28, No. 8, pp. 887-901.<br />
Edinburgh Napier University (2010) Survey of Mobile Technology Use at Edinburgh Napier University,<br />
unpublished.<br />
Gallacher, J. (2006) ‘Blurring the boundaries or creating diversity? The contribution of the further education<br />
colleges to higher education in Scotland’, Journal of Further and Higher Education, Vol. 30, No. 1, pp. 43-<br />
58.<br />
Howieson, C. and Croxford, L. (2011) Students’ Experience of the Transition from HN to Degree Study. Final<br />
Report to ELRAH. University of Edinburgh.<br />
Kivlichan, N. and Chirnside, A. (2011) Students Calling Students: Trimester 1 2010/11. Final Report, Edinburgh<br />
Napier University.<br />
Kukulska, A. (2005) ‘Mobile usability and user experience’, in A. Kukulska-Hulme and J. Traxler (eds.) Mobile<br />
Learning; A Handbook for Educators and Trainers, Oxon: Routledge .<br />
Lowe, H. and Cook, A. (2003) ‘Mind the Gap: Are students prepared for higher education?’, Journal of Further<br />
and Higher Education, Vol. 27, No. 1, pp. 53-76.<br />
Lunsford, J. (2010) ‘Using handheld technologies for student support’, Journal of the Research Center for<br />
Educational Technology (RCET) Vol. 6, No.1. [online]<br />
http://www.rcetj.org/index.php/rcetj/article/view/82/181 (accessed 7th August 2011)<br />
Macaskill, K. (2010) ‘Student Evaluations of Articulation: A Guide to Institutional Practice’, Paper presented at<br />
Hitting the Ground Running Symposium, Glasgow Caledonian University, Glasgow, June.<br />
McNiff, J. and Whitehead, J. (2010) You and Your Action Research Project. Oxon: Routledge.<br />
Mentor, D. (2011) Supporting students’ connectedness via texting. Educause Quarterly Magazine, Vol. 34, No.1.<br />
[online]<br />
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/SupportingStudent<br />
sConnectednes/225851 (accessed 5th August 2011).<br />
Norton, L. (2009) Action Research in Teaching & Learning, Oxon: Routledge.<br />
Pike, A. and Harrison, J. (2011) ‘Crossing the FE/ HE divide: the transition experience of direct entrants at Level<br />
6’, Journal of Further and Higher Education, Vol. 35, No. 1, pp. 55-67.<br />
Reil, M. (2011) Understanding Action Research, Centre for Collaborative Action Research, [online],<br />
http://cadres.pepperdine.edu/ccar/define.html (accessed 7th August 2011).<br />
Rovai, A. (2002) Sense of community, perceived cognitive <strong>learning</strong>, and persistence in asynchronous <strong>learning</strong><br />
networks, Internet and Higher Education, Vol. 5. pp. 319-332.<br />
Riordan, B. and Traxler, J. (2005) ‘Supporting computing students at risk using blended technologies’.<br />
Proceedings of 4th Annual Conference. Galway, Ireland: LTSN Centre for Information and Computer<br />
Science, pp. 174-175.<br />
Scottish Credit and Qualifications Framework. (2009). SCQF Handbook. [online]<br />
http://www.scqf.org.uk/Resources/ (accessed 28 July 2011).<br />
Tait, H. and Godfrey, H. (2001) ‘Enhancing the Student Experience for Direct Entrants to the Penultimate Year of<br />
Undergraduate Degree Programmes’, Journal of Further and Higher Education, Vol. 25, No. 2, pp. 259-265.<br />
Tinto, V. (1975) ‘Dropout from Higher Education: A Theoretical Synthesis of Recent Research’, Review of<br />
Educational Research, Vol. 45, pp. 89-125.<br />
Tinto, V. (2006) ‘Research and Practice of Student Retention: What next?’, Journal of College Student Retention,<br />
Vol. 8, No. 1, pp. 1-19.<br />
Traxler, J., & Kukulska-Hulme, A. (2005). ‘Evaluating Mobile Learning: Reflections on Current Practice’.<br />
Proceedings of mLearn 2005, [online], http://www.mlearn.org.za/papers-full.html (accessed June 2011)<br />
UK Council for International Student Affairs (UKCISA), ‘Best Practice Case Studies’, [online]<br />
http://www.ukcisa.org.uk/pmi/case_studies_support.php (accessed 7th August 2011)<br />
273
Blended Learning at the Alpen-Adria-Universität Klagenfurt<br />
Gabriele Frankl and Sofie Bitter<br />
Alpen-Adria-Universität, Klagenfurt, Austria<br />
gabriele.frankl@aau.at<br />
sbitter@edu.uni-klu.ac.at<br />
Abstract: The application of blended <strong>learning</strong>, i.e. the combination of face-to-face and eLearning elements, is<br />
one of the university’s declared goals and seen as a chance to cope with the challenges educational institutions<br />
are facing today. After several years of implementing eLearning and blended <strong>learning</strong> at the Alpen-Adria-<br />
Universität Klagenfurt (AAUK), a survey has been conducted to detect the status quo. The survey shed light on<br />
the potential of eLearning and blended <strong>learning</strong> as well as risks, fields of application and contentedness.<br />
Furthermore, the survey focused on the benefits of blended <strong>learning</strong> as well as on reusability. Data were<br />
collected with online and paper questionnaires and 1154 valid answers (collected over a period of three<br />
semesters) were obtained. Students and lecturers were asked to judge, how studying is supported by the various<br />
forms blended <strong>learning</strong> offers. Furthermore, it was investigated how common different forms of blended <strong>learning</strong><br />
are and what the most frequented tools are. Students see great potentials of eLearning and blended <strong>learning</strong> in<br />
flexible <strong>learning</strong> (24 hours a day 7 days a week), in using their study time more efficiently and in being able to<br />
learn at their own pace. In addition, students were asked about their attitude towards mobile <strong>learning</strong>, as this form<br />
of <strong>learning</strong> is planned to be further expanded in the near future. The eLearning service department has<br />
additionally collected suggestions for improvements regarding the eLearning and blended <strong>learning</strong> environment<br />
at the AAUK. Based on these results our already existing service of video-recording lectures has been further<br />
expanded. Apart from that, we have started to offer a “secure exam environment” where online exams can take<br />
place on a grand scale. With this special software students can use their own devices but have no access to any<br />
of their files or to non-specified Internet resources.<br />
Keywords: blended <strong>learning</strong>, potentials, risks, benefits, video recording of lectures, secure exam environment,<br />
online testing<br />
1. Introduction<br />
The Alpen-Adria-Universität Klagenfurt (AAUK), situated on the campus in the Wörthersee region of<br />
Carinthia, Austria, hosts about 10,000 students. It is divided into four faculties: (1) Humanities and<br />
Social Sciences, (2) Interdisciplinary Studies (with additional centres in Vienna and Graz), (3)<br />
Management and Economics, and (4) Technology. The AAUK seeks high quality of teaching<br />
combined with excellent support of individual students. Due to the small number of students, the<br />
personal coaching of students can be guaranteed. (e.g., CHE University Ranking 2009/10) However,<br />
coaching already suffers in highly frequented study programs. Thus, offering up-to-date <strong>learning</strong> and<br />
teaching methods, up-to-date didactic skills of the lecturers and adequate infrastructure is an<br />
important aim of the AAUK. ELearning and blended <strong>learning</strong> are integral parts of the university’s<br />
strategy to offer attractive study conditions. On the one hand-side, there are necessary challenges<br />
required due to financial shortenings of the ministry. Also other universities face challenges that<br />
require rethinking and changes in teaching methods taking advantage of blended <strong>learning</strong> (e.g.<br />
McNaught 2011). However, on the other hand-side, “digital natives“, which enter universities in<br />
increasing numbers, cannot be neglected, since they definitely need the application of new teaching<br />
methods (Prensky 2008).<br />
The university pursues a bottom-up approach concerning the spread of eLearning and blended<br />
<strong>learning</strong>. This means that lecturers can decide autonomously which facets of eLearning and blended<br />
<strong>learning</strong> they regard as useful for their individual teaching contexts. This is especially for the blended<br />
<strong>learning</strong> concept an advantageous precondition. Lecturers can individually and step-by-step convert<br />
their usual teachings to the eLearning environment. As already highlighted by Bakhtiar (2011), the<br />
lecturer is the critical element in distance teaching and <strong>learning</strong>. Students can only profit, if the<br />
lecturer is convinced about his/her teaching methods.<br />
Blended <strong>learning</strong> does not only mix media and methods but integrates a number of ideas for teaching<br />
and <strong>learning</strong>. Thus, it enlarges established educational concepts with the digital flexibility and does<br />
not completely condemn existing structures, attitudes and routines. This process does not overburden<br />
lecturers and students since it offers the possibility to slowly integrate the new methods and<br />
procedures (Reinmann-Rothmeier 2003, p. 30 and pp.42 ff.).<br />
274
Gabriele Frankl and Sofie Bitter<br />
As the results of study by Thompson and MacDonald (2005) suggest, structure and design are critical<br />
quality features for eLearning courses. In transforming their courses, the eLearning service<br />
department at the AAUK offers lecturers assistance for the conceptual design and the planning of<br />
didactic blended <strong>learning</strong> courses, as well as support regarding the technical and organizational<br />
implementation. Additionally, lecturers can apply at the eLearning service department for specially<br />
trained e-tutors who accompany and support the lecturer for one or two terms.<br />
Face-to-face teaching at the AAUK is still very important and should remain as significant as it is.<br />
Results of recent brain research show that the strongest and the best ‘drug’ for people are other<br />
people (Bauer 2008). Therefore, efforts target to optimally combine the benefits of face-to-face<br />
teaching with the advantages of eLearning. Thus, a broad range of blended <strong>learning</strong> activities is<br />
covered at the AAUK, starting from minor additions and extensions of face-to-face teaching to a few<br />
teaching scenarios that are nearly completely based on eLearning. It has to be pinpointed that a vast<br />
majority does use tailor-made concepts of blended <strong>learning</strong> with great contentedness.<br />
1.1 The theoretical potential of eLearning and blended <strong>learning</strong><br />
One potential that we can derive from theory is that eLearning can bridge the technology gap in<br />
education, but in the context of methodology (e.g. Luhmann and Schorr 1979). According to Luhmann<br />
and Schorr (1979), education was based on the communication between teachers and students.<br />
Traditionally, the lecture hall - under the control of the chair holding professor - is the only relevant<br />
organizational unit for the production in higher education. This focus on the traditional classroom<br />
setting is frequently reproduced in the eLearning literature while it might be helpful to consider other<br />
educational contexts and settings as well (Frankl, Pfeffer, Zederbauer, Steinberger 2008). Learning<br />
can only happen actively. However, in a traditional classroom setting, the active part was/is mainly the<br />
lecturer. The learners play/ed the passive part. Due to costs, written homework was exclusively<br />
addressed to the teacher and was not available to students for feedback. In the digital environment<br />
these obstacles can easily be overcome through means of online peer-reviews. Additionally, it is also<br />
possible to jointly work on wikis, ePortfolios or tools like Etherpad. The teacher was not only active<br />
and autonomous, but also a lonesome fighter. The predominant oral culture of the traditional course<br />
resulted in the interaction of the people present. Additionally, technology transfer (the transfer of the<br />
‘how to’ of teaching) and hence the improvement of teaching was very difficult. Teachers normally did<br />
not learn their profession from their peers or from formal training, but rather from past experiences as<br />
students. (Frankl, Pfeffer, Zederbauer, Steinberger 2008)<br />
The situation changed with the evolvement of new digital media in education. These new technologies<br />
allow new forms of communication and cooperation fostering knowledge generation, exchange and<br />
usage. In a blended <strong>learning</strong> environment, it is now possible that teachers share experience and invite<br />
each other for a skype-conference to participate as co-lecturers or guest-lecturers. Further, it<br />
becomes feasible to visit colleagues’ online <strong>learning</strong> courses and give feedback or to share literature,<br />
methods or discuss ideas.<br />
Web 2.0 does indeed show great potential also for higher education (Grosseck 2009). Furthermore,<br />
as highlighted in a study by Lee and Tsai (2011), technology supported <strong>learning</strong> shows better<br />
outcomes than traditional face-to-face <strong>learning</strong>. This holds true for the fields of collaboration, selfregulated<br />
<strong>learning</strong> and information seeking. A detailed picture is provided by Dabbagh (2011),<br />
focusing on the influences of online media tools and on self-regulated <strong>learning</strong> in higher education.<br />
Another study by Cabero et al. (2010) also pinpoints the positive influence of blended <strong>learning</strong> on<br />
students’ <strong>learning</strong> and lecturers’ teachings. Empirical results also reveal that more experienced<br />
students (with online tools) were much more positive towards eLearning. However, the study also<br />
reports that the overall trend was openness towards new tools and/or applications (Lam et al. 2011).<br />
The aim at the AAUK is to establish “<strong>learning</strong> without walls”. To evaluate the status quo, to initiate<br />
future developments and to design the future, we started a survey about eLearning and blended<br />
<strong>learning</strong> at our university.<br />
2. Survey methodology<br />
In the summer term 2009, winter term 2009/10 and summer term 2010, we conducted online surveys<br />
on eLearning and blended <strong>learning</strong> at the AAUK. The aim was to identify the status quo and recent<br />
developments in the eLearning and blended <strong>learning</strong> sector at the university as well as potentials for<br />
improvement. With the data gained through these surveys the eLearning department is in the position<br />
275
Gabriele Frankl and Sofie Bitter<br />
to react to new needs and tailor its services offered to the actual requirements of the university<br />
members. All students and lecturers at the AAUK were invited via email to take part in the online<br />
survey. In addition, during the second and third survey, paper questionnaires were also available at<br />
the secretariats of the various institutes. Therefore, we have also motivated those lecturers and<br />
students of the university who prefer paper-questionnaires. Summing up all three surveys, 1154<br />
people completed the questionnaire. In addition, five qualitative guided interviews with experienced<br />
and didactic well-trained lecturers were conducted. However, they are not included in this study since<br />
it would go beyond the scope of this paper.<br />
As highlighted in table 1, the participation in the survey slightly increased over the three terms (this<br />
might be due to raffles offered with the second and third survey), whereas only the valid responses<br />
have been considered for the evaluations and analyses.<br />
Table 1: Sample characteristics<br />
Summer term 2009 Winter term 2009 Summer term 2010<br />
N % N % N %<br />
Total 432 - 488 - 504 -<br />
Valid 327 100,0 412 100,0 415 100,0<br />
Missing 105 - 76 - 89 -<br />
Male (valid) 136 41,6 197 47,8 189 45,5<br />
Female (valid) 191 58,4 215 52,2 226 54,5<br />
Student 216 66,1 273 66,3 314 75,7<br />
Lecturer 90 27,5 112 27,2 76 18,3<br />
Non-lecturing staff 21 6,4 27 6,6 25 6,0<br />
3. Survey results<br />
3.1 eLearning, blended <strong>learning</strong> and Moodle in general<br />
As Figure 1 shows, an increase among those members of the university who actively use eLearning<br />
and blended <strong>learning</strong> can be recognized at the AAUK. Especially from the summer term 2009 to the<br />
winter term 2009/10 a substantial augmentation can be recorded. However, this increase in using<br />
eLearning and blended <strong>learning</strong> is due to a sponsorship which the eLearning department received<br />
and which was invested to expand the use of the eLearning and blended <strong>learning</strong> activities at the<br />
AAUK, e.g. through the assignment of more e-tutors and through expanding the varieties of the fields<br />
of applications.<br />
Figure 1: Usage of eLearning and blended <strong>learning</strong> among AAUK lecturers and students<br />
* 384 valid answers and one missing answer for this question (total of lecturers and students 385)<br />
** 389 valid answers and one missing answer for this question (total of lecturers and students 390)<br />
276
Gabriele Frankl and Sofie Bitter<br />
In the summer term 2009, 770 (out of 1493; 51,6 %) university courses implemented Moodle (cf.<br />
https://e<strong>learning</strong>.aau.at/moodle or http://moodle.org) in their course outline, in the winter term 2009<br />
this number increased to 823 (out of 1460; 56,4 %) courses and finally in the summer term 2010, 842<br />
Moodle courses (out of 1510 university courses; 55,8 %) were employed, as highlighted in Figure 2.<br />
This statistic goes in line with those respondents (lecturers and students) who answered that they use<br />
Moodle at the university.<br />
Figure 2 Overview of Moodle usage compared to Moodle courses<br />
A more detailed picture provides graph 3 by highlighting the distribution among lectures and students<br />
at the AAUK. Especially among students Moodle become very popular over the three terms. Although<br />
it looks like that lecturers reduced their Moodle use, we cannot confirm this, since the ratio of Moodle<br />
courses to normal courses gives us a clearly different picture. Due to the fact that approximately 56 %<br />
of the university courses use Moodle in the summer and winter term 2010, we assume that the<br />
sample of lecturers is not representative and does not very well reflect the actual situation. In addition<br />
to the online survey (for the last two surveys) paper questionnaires have been distributed reaching<br />
also those lecturers who might be not that technology affine as compared to the summer term 2009,<br />
where solely online questionnaire were distributed.<br />
Generally, we do not know whether the same people completed all three surveys. It has to be<br />
pinpointed that the analyses highlighted in the forthcoming pages do not necessarily reflect a trend<br />
but rather reflect the mood of the respondent group.<br />
Figure 3: Usage of Moodle, the university’s online <strong>learning</strong> platform<br />
*241 out of 306 lecturers and students completed the question<br />
**342 out of 385 lecturers and students completed the question<br />
***342 out of 390 lecturers and students completed the question<br />
Taking a glance at the different faculties and their employment of Moodle it gives a very well defined<br />
picture that humanities and social sciences followed by management and economics are the clear<br />
leaders using this eLearning application as highlighted in Figure 4. It has to be added that Moodle at<br />
the faculty for humanities and social sciences is especially popular amongst the language institutes as<br />
well as at the institute for media and communication. Unfortunately the institute for educational<br />
sciences forms the taillight. The interdisciplinary studies have the smallest number of Moodle users<br />
but also the lowest number of lecturers and students.<br />
277
Figure 4 Moodle usage among the AAUK faculties<br />
Gabriele Frankl and Sofie Bitter<br />
3.2 Usage of eLearning and blended <strong>learning</strong><br />
Exchange of information and materials, time scheduling and messaging, as well as online<br />
assignments and students communicating via means of eLearning are the most popular features<br />
among the users. Less frequently used are <strong>learning</strong> logs and discussions in an online environment.<br />
However, features like video recording of courses and offering the materials via the eLearning<br />
platform Moodle to the students as well as giving feedback and to do online tests is starting to<br />
become more and more popular. For the graphical illustration see Figure 5.<br />
Figure 5 eLearning methods employed<br />
*This question was not included in the survey of the summer term 2009<br />
In the Moodle courses of the three terms, students mostly worked with forums, assignments,<br />
calendars, feedbacks and surveys. They did online group works, online practice activities and tests<br />
and worked collaboratively in wikis. The features least provided were e-portfolios, glossaries, SCORM<br />
<strong>learning</strong> packages, lessons and peer reviews.<br />
Slightly different compared to our results, a study by Miyazoe and Anderson (2010) investigating an<br />
English blended <strong>learning</strong> course at a Tokyo university, highlights that wikis are the preferred blended<br />
<strong>learning</strong> applications before blogs and forums. A possible reason could be the different usability of the<br />
applied wikis. The wiki application in Moodle shows indeed technical weaknesses and a lack of<br />
usability.<br />
3.3 Rating of potentials and risks of eLearning and blended <strong>learning</strong><br />
In our survey, we presented typical potentials of eLearning to students and lecturers (cf. Figure 7).<br />
Overall, students and lecturers see great prospects in eLearning and blended <strong>learning</strong>. Additionally, a<br />
positive trend amongst students and lecturers over the period of three terms can be recognized.<br />
278
Gabriele Frankl and Sofie Bitter<br />
Figure 6: Moodle tools used<br />
*“Materials” were not included in the survey of the summer term 2009<br />
*The figure in the winter term 2009/10 reflects only those users who solely uploaded materials (no<br />
other Moodle tools used); whereas in the summer term 2010 figure also includes users who used<br />
other Moodle tools in addition to providing materials<br />
Students indeed see great potential of eLearning and blended <strong>learning</strong> in flexible <strong>learning</strong> 24 hours a<br />
day 7 days a week, in more efficient use of their study time and in being able to learn at their own<br />
pace. They appreciate the possibility to take over responsibility for their own progress and avoiding<br />
cramming at the end of the semester thanks to the opportunities of continuous assessment. As a<br />
contrast, only few of the lecturers, who were also asked to rate these potentials for students, saw the<br />
latter two potentials. The students see the smallest potential in the support of collaboration through<br />
eLearning and blended <strong>learning</strong>. Whereas lecturers, see the greatest potential of eLearning and<br />
blended <strong>learning</strong> in the exchange of information and materials but do not think that <strong>learning</strong> time can<br />
be used more efficiently. A detailed evaluation of the potentials of eLearning and blended <strong>learning</strong> for<br />
students and lectures gives the following picture:<br />
Figure 7: eLearning and blended <strong>learning</strong> potentials (lecturers vs. students)<br />
Measurement: 4-point frequency scale (1 disagree to 4 agree)<br />
°“Exchange” was not included in the survey of the summer term 2009<br />
279
Gabriele Frankl and Sofie Bitter<br />
Note: Not every respondent gave answer to all items of the question. Therefore, the response rate<br />
varied as follows:<br />
*summer term 2009 (199-223 respondents out of a total 306 students and teachers)<br />
**winter term 2009/10 (232-328 out of 385)<br />
***summer term 2010 (322-333 out of 390)<br />
In the first study (summer term 2009) we explicitly asked to rate predefined potential risks of<br />
eLearning and blended <strong>learning</strong>. The major concern was that there might not be enough opportunities<br />
to get into personal contact with fellow students and lecturers. Additionally, users thought that<br />
eLearning might increase the workload for students. Hardly any users believed that lecturers would<br />
become unnecessary because of eLearning. In the consecutive two studies (winter term 2009/10 and<br />
summer term 2010) only one open question relating to risks was included in the survey to discover<br />
which risks users see without having predefined suggestions. Indeed, among the most commonly<br />
named risks were lacking social contact and anonymity, reduced development of social competences,<br />
suffering from quality of teaching as well as further increasing the distance between students and<br />
lecturers. At this point, the authors would like to pinpoint that the survey reveals that no qualitative<br />
short cuts, due to eLearning respectively blended <strong>learning</strong>, have to be feared. ELearning and blended<br />
<strong>learning</strong> have to be regarded as a chance to enhance quality in teaching and <strong>learning</strong> methods (see<br />
Frankl & Bitter 2011).<br />
3.4 Benefits thanks to eLearning and blended <strong>learning</strong><br />
In addition, we asked lectures to what extend they think that they benefit from eLearning and blended<br />
<strong>learning</strong>. We asked the lectures whether eLearning and blended <strong>learning</strong> supports better structured<br />
courses, helps them to turn down the work load, increased the flexibility of <strong>learning</strong> regarding time<br />
and/or space, as well as reusability and transparency of the courses. We used a 4-point frequency<br />
scale from totally agree to totally disagree. Especially having better overview and structure of the<br />
courses as well as reusability and work load relief (after a certain period of time) were the top benefits<br />
reported by lectures. Figure 8 gives a detailed overview of the different factors and highlights the<br />
mean of the frequency results.<br />
Figure 8: Benefits from the teachers’ perspective<br />
Measurement: 4-point frequency scale (1 totally disagree to 4 totally agree)<br />
We also asked the students to estimate to what extent eLearning and blended <strong>learning</strong> support them<br />
in reaching their <strong>learning</strong> aims. The data revealed that eLearning and blended <strong>learning</strong> had the<br />
greatest impact on successful <strong>learning</strong> thanks to access to important study materials, communication<br />
with and feedback from lecturers as well as fellow students and online self-tests and opportunities of<br />
collaborative work. The study also reveals huge potentials regarding re-academization processes and<br />
improving the quality in <strong>learning</strong> and teaching methods (Frankl & Bitter 2011). However, the smallest<br />
benefit from eLearning and blended <strong>learning</strong> for students is the consideration of individual interests,<br />
individual strengths and weaknesses. Figure 9 gives a overview of the students’ ratings:<br />
280
Gabriele Frankl and Sofie Bitter<br />
A study in the field of eLearning in an accounting class that partially reflects our results, revealed that<br />
student satisfaction was associated inter alia with usefulness and availability of lecture notes, online<br />
assessment, model answers, and online chat (Basioudis & de Lange 2009).<br />
Figure 9: Benefits from the students' perspective<br />
Measurement: 4-point frequency scale (1 disagree to 4 agree)<br />
“time-delayed communication” was not included in the survey of the summer term 2009<br />
4. Implications and consequences of the survey results<br />
The results of our survey have several implications for further activities in the blended <strong>learning</strong> sector<br />
at the Alpen-Adria-Universität Klagenfurt. As a long-term goal we would like to spread blended<br />
<strong>learning</strong> further across the various disciplines. The provision of materials, time scheduling and<br />
messaging, assignments, communication among students and feedback from lecturers to students<br />
are relatively widespread, whereas the opportunities for collaborative and reflexive tasks are relatively<br />
rare, although those taking advantage of this possibilities use it very well and in a very innovative way.<br />
To point out the potentials of working with wikis, keeping <strong>learning</strong> diaries, doing peer-reviews or<br />
working with ePortfolios, we need to collect good-practice examples, develop practical suggestions<br />
and instruction manuals and share and discuss them with the lecturers to explore the potentials of<br />
these forms of <strong>learning</strong> in more courses. Likewise, we have to communicate the differences in the<br />
assessment of particular potentials of eLearning and blended <strong>learning</strong> between students and<br />
lecturers: if lecturers knew that students do see great potentials in continuous assessment and in<br />
taking over more responsibility for their <strong>learning</strong> outcomes, lecturers might dare to modify their<br />
teaching strategies. It is appropriate to source out <strong>learning</strong> and teaching methods to the online<br />
environment in order to foster flexibility and autonomous accessibility of online courses and online<br />
<strong>learning</strong>. However, this also means to use the reduced face-to-face times more efficiently and<br />
advantageously. For example, to get to know each other, to build up trust, to get introduction into new<br />
knowledge areas, to conduct in-depth discussions or to foster reflexion processes. For example,<br />
forums on frequently-asked-questions or video-recorded lectures can be sourced out to support faceto-face<br />
lectures and to gain valuable time.<br />
In the survey of the summer term 2009, students could also express their blended <strong>learning</strong> wishes for<br />
the future. Two thirds said they would appreciate video recordings of lectures so that they can revise<br />
the topics covered in class whenever they want. Almost two thirds said that they would like to get<br />
guest access to online courses of previous semesters so that they can select courses better. About<br />
one third wanted to have an online room independent of concrete courses for communication and<br />
exchange with their fellow students. We have already started to put these suggestions into practice.<br />
Thus, an increasingly popular service that the eLearning service department offers and which has<br />
been expanded is video-recording lectures. The lecturer’s speech as well as the presentation slides<br />
are recorded and provided on Moodle to the students. This allows students to revise content<br />
281
Gabriele Frankl and Sofie Bitter<br />
whenever they want: instead of having to watch an entire 90 minute video, the presentation slides can<br />
be used to navigate exactly to the section students want to watch again.<br />
OpenCourseWare Klagenfurt, inspired by the MIT OpenCourseWare Initiative, is a collection of free<br />
educational resources organized in courses. Since 2005 we are part of this initiative and still the only<br />
university in the German-speaking area participating in it. Finally, we have set up platforms where<br />
students can exchange anything from their personal experiences to lecture notes in a self-organized<br />
way. At the moment we are sustaining such platforms for beginner students in the most popular<br />
subjects at the AAUK as well as for incoming/outgoing students. Additionally, we provide information<br />
and communication platforms for lectures to exchange study programme relevant issues e.g.<br />
methods, literature. Lectures are no lonesome fighters anymore but support each other enhancing the<br />
teaching quality. Additionally, we are planning to expand our <strong>learning</strong> services to support mobile<br />
<strong>learning</strong>.<br />
5. Conclusion<br />
The survey results have given us a first picture of the status quo of eLearning and blended <strong>learning</strong> at<br />
the AAUK. The surveys have revealed the scope of eLearning and blended <strong>learning</strong> as well as its<br />
most common forms. At the same time it helped us to identify facets of eLearning potentials which<br />
have not been fully explored or which have been underestimated. These forms of eLearning can thus<br />
be put into the centre of interest for university-internal training programmes. The students’ ratings of<br />
benefits and risks of eLearning and blended <strong>learning</strong> as well as their wishes have shown the direction<br />
into which to promote further eLearning activities.<br />
Further, we are in the starting phase of offering a “secure exam environment” (SEE) where online<br />
exams can be conducted without being restricted to computer labs. The purpose of this environment<br />
is to enable students to conduct tests online on their own devices without having access to any of<br />
their own files and to the Internet (except if explicitly stipulated by the lecturer). Thus also no other<br />
form of communication among the students is possible e.g. chats, mails. Students who do not have<br />
own mobile devices receive loan devices. This insular testing is possible thanks to special software.<br />
Students have to boot their devices from an USB stick or from a DVD and immediately are directed to<br />
the Moodle log-in screen. They can then navigate to the respective course and open the exam. The<br />
SEE uses the save-exam-browser hindern access to any materials/resources that are not permitted in<br />
the test. Thanks to various Moodle features, answers and questions can be mixed hindering cheating.<br />
The development of the secure exam environment is a major leap forward and constitutes an<br />
enormous workload relief for lecturers and students. Lecturers can decided whether to prompt<br />
automatic feedback (e.g. in case of standardized questions like multiple-choice) or to provide<br />
individual feedback. With standardized questions, students also immediately receive the test scores<br />
and – if applicable – feedback. For a detailed overview on the development and technical as well as<br />
organizational implementation of the secure exam environment, confer to Frankl, Schartner and<br />
Zebedin (2011). A survey, after the first exams, reveals a very positive feedback of students and<br />
lectures about this new way of testing. The negative critique is the unfamiliar handling of the loan<br />
devices. Furthermore, in the next development phase of the SEE, Excel will be integrated. Hence, this<br />
enables certain study programs (e.g. information systems, mathematics) to test common calculation<br />
methods. Up-to-date <strong>learning</strong> also requires up-to-date testing.<br />
Concluding, we would like to add that online collaboration still needs time to settle and get more<br />
advanced. Especially lecturers need time to accustom to these new ways of teaching and to<br />
restructure their previous teaching methods. However, according to Prensky (2008), lecturers do not<br />
need to be scared of implementing new eLearning and blended <strong>learning</strong> methods since there is no<br />
need for them to fully understand the digital technology. The new generation of students, namely the<br />
“digital natives” are already experts for these new <strong>learning</strong> technologies. The role of the lecturers is to<br />
foster curiosity and to activate <strong>learning</strong> processes.<br />
Each terms new students are joining the university being much more connected and feeling part of<br />
the digital era we face. It is of utmost importance to foster, enhance and further develop our services<br />
to optimally support these future students. The mood towards eLearning and blended <strong>learning</strong> at the<br />
AAUK makes us confident to move forward making most efficient use of the potentials eLearning and<br />
blended <strong>learning</strong> offers.<br />
282
References<br />
Gabriele Frankl and Sofie Bitter<br />
Bakhtiar, F. (2011) “The Role of Information and Communication Technologies (Ict) in Distance Education”,<br />
Journal of American Science 7.6, 318-322.<br />
Basioudis, I. G. and de Lange, P. A. (2009) “An assessment of the <strong>learning</strong> benefits of using a Web-based<br />
Learning Environment when teaching accounting”, Advances in Accounting, 25:1, 13-19.<br />
Bauer, J. (2008) Prinzip Menschlichkeit: Warum wir von Natur aus kooperieren, Heyne, München.<br />
Cabero, J., C. Llorente and Puentes, A. (2010) “Online Students' Satisfaction with Blended Learning”.<br />
Comunicar.35, 149-157.<br />
Dabbagh, N. and Kitsantas, A. (2011) “Personal Learning Environments, Social Media, and Self-Regulated<br />
Learning: A Natural Formula for Connecting Formal and Informal Learning”, The Internet and Higher<br />
Education Article, in Press.<br />
Frankl, G., Pfeffer, T., Zederbauer, S. and Steinberger, C. (2008) “e-Collaboration at the Alpen-Adria-Universität<br />
Klagenfurt”, Proceedings of the Apera 2008.<br />
Frankl, G. and Bitter, S. (2011): “Blended Learning at the Alpen-Adria-Universität Klagenfurt. Improving quality<br />
and fostering re-academization”, handed in at the ICL 2011, Pieštany, Slovakia, submitted.<br />
Frankl, G., Schartner P. and Zebedin, G. (2011) „The ‘Secure Exam Environment’ for Online Testing at the<br />
Alpen-Adria-Universität Klagenfurt / Austria”, handed in at the E-LEARN 2011, Honolulu/Hawaii, submitted.<br />
Grosseck, G. (2009) “To use or not to use web 2.0 in higher education?”, Procedia - Social and Behavioral<br />
Sciences, 1 (1), 478-482.<br />
Lam, P., Lee, J., Chan, M. and McNaught, C. (2011) “Students’ Use of E<strong>learning</strong> Strategies and Their<br />
Perceptions of E<strong>learning</strong> Usefulness”, in Global Learn Asia Pacific 2011, Barton, S.-M., Hedberg J. and<br />
Suzuki, K. (eds.), Melbourne, Australia, AACE, 1379-1388.<br />
Lee, S. W. and Tsai, C.-C. (2011) “Students' perceptions of collaboration, self-regulated <strong>learning</strong>, and information<br />
seeking in the context of Internet-based <strong>learning</strong> and traditional <strong>learning</strong>”, Computers in Human Behavior,<br />
27 (2), 905-914.<br />
Luhmann, N. and Schorr, K. E. (1979) „Das Technologiedefizit der Erziehung und der Pädagogik“, Zeitschrift für<br />
Pädagogik, Vol. 25, No. 3, pp. 345-365.<br />
Luhmann, N. (1975) Interaktion, Organisation, Gesellschaft. Anwendungen der Systemtheorie, In Niklas, L. (ed.),<br />
Soziologische Aufklärung 2. Aufsätze zur Theorie der Gesellschaft, Westdeutscher Verlag, Opladen.<br />
McNaught, C. (2011) "The Best of Both Worlds: Effective Hybrid Learning Designs in Higher Education in Hong<br />
Kong" in Hybrid Learning, Kwan, R., Fong, J., Kwok, L.-F. and Lam, J. (eds.), Springer Berlin / Heidelberg,<br />
1-9.<br />
Miyazoe, T. and Anderson, T. (2010) “Learning outcomes and students' perceptions of online writing:<br />
Simultaneous implementation of a forum, blog, and wiki in an EFL blended <strong>learning</strong> setting”, System, 38,<br />
185-199.<br />
Prensky, M. (2010) Teaching Digital Natives. Partnering for Real Learning. Corwin, Thousand Oaks.<br />
Reinmann-Rothmeier, G. (2003) Didaktische Innovation durch Blended Learning: Leitlinien anhand eines<br />
Beispiels aus der Hochschule, Huber, Bern [et al.].<br />
Thompson, T. L. and MacDonald, C. J. (2005), "Community Building, Emergent Design and Expecting the<br />
Unexpected: Creating a Quality E<strong>learning</strong> Experience." The Internet and Higher Education 8.3, 233-249.<br />
Zeit-Online (n.d.) “CHE University Ranking 2009/10”, [online], http://ranking.zeit.de/che10/CHE_en<br />
283
Evaluating the use of Social Networking Sites as a Tool for<br />
Knowledge Sharing for Developing Higher Education in<br />
Developing Countries: An Exploratory Study of Egypt and<br />
Iraq<br />
Elaine Garcia, Ibrahim Elbeltagi, Sawasn Al-Husseini and Ahmed Abdelkader<br />
University of Plymouth, UK<br />
elaine.garcia@plymouth.ac.uk<br />
i.elbeltagi@plymouth.ac.uk<br />
sawasn.al-husseini@plymouth.ac.uk<br />
a.kader006@gmail.com<br />
Abstract: Educational institutions are today facing increasing pressures due to economic, political and social<br />
upheaval. This is only exacerbated by the nature of education as an intangible good which relies upon the<br />
intellectual assets of the organisation in terms of its staff. It is within this context that Social Networking Sites<br />
(SNS) offer an important potential alternative method to manage and share knowledge within educational<br />
institutions. The focus of this research therefore explores the role that SNS could play in relation to the<br />
development of Higher Education (HE) within developing countries with a focus on the countries of Egypt and<br />
Iraq as having one of the highest and lowest rates of internet usage within the Middle East respectively. In order<br />
to gain a better understanding of the potential drivers and barriers to the use of SNS as a knowledge sharing<br />
(KS) tool within educational institutions, within the developing world, a case study approach has been used. This<br />
approach consisted of semi-structured interviews with forty members of staff, both senior managers and<br />
academic staff, within Mansoura University’s Faculty of Commerce in Egypt and Al-Mustansirya University’s<br />
Faculty of Economy and Administration in Iraq. The results show there are both areas of agreement and<br />
difference in the views of staff within both institutions. In both cases issues of culture appear to affect the<br />
participation of female staff and additionally age appears to be a predictor of use. However between the<br />
institutions there appears to be wider differences relating to the existence of KS strategies, the extent to which<br />
staff understand the nature of SNS and technology in general and the extent to which Senior Managers and other<br />
staff views are shared. Overall the results suggest that Mansoura University in Egypt is far more prepared for the<br />
use of SNS as a KS tool and some usage appears to already be occurring. Within Al-Mustansirya University, Iraq<br />
however the usage of SNS as an organisational tool appears to be unlikely to occur within considerable<br />
promotion and changes to strategic drivers.<br />
Keywords: social networking sites, developing countries, case study, higher education, knowledge sharing<br />
1. Introduction<br />
The education sector is today facing an increasingly challenging period. Global recession, increased<br />
competition, globalisation, technological advances and demand for increasing educational quality<br />
(Mathew, 2010) are all leading to a need for education across the world to become increasingly<br />
competitive, efficient and innovative.<br />
Within this context Knowledge Management (KM) represents an imperative area of consideration in<br />
terms of capturing, organizing and sharing organisational knowledge (Liao et al., 2004) to maximise<br />
and fully exploit the intellectual assets institutions hold.<br />
The use of Social Networking Sites (SNS) offers potential to share organisational knowledge within<br />
educational institutions as they allow for the capturing of tacit knowledge and allowing Knowledge<br />
Sharing (KS) to occur which in turn provides greater opportunities for creating new knowledge and<br />
innovative practices.<br />
SNS are also currently appearing to have a substantial impact specifically within developing countries<br />
where a number of political events have recently been reportedly influenced by SNS. These include<br />
Egypt, (Idle & Nunns; 2011; Alexander, 2011; Nabi, 2011) Tunisia, (Delany, 2011), Libya (Beaumont,<br />
2011) and Pakistan. (Sutter, 2011; Hill, 2011, Blair, 2011)<br />
Whilst the growth of Facebook, the largest and most well known SNS has slowed and user numbers<br />
have decreased in the countries in which it was first launched, this has been tempered by growth in<br />
developing countries within the last few months (Arthur, 2011).<br />
284
Elaine Garcia et al.<br />
According to the Spot on PR Report (2010), shown in Figure 1, the number of Arab internet users has<br />
reached more than 65 million whilst Arabic Facebook users are now predicted to total more than 15<br />
million.<br />
Figure 1: Middle East and North Africa’s (MENA’s) top five Facebook communities source: Spot on<br />
PR (2010)<br />
Based on this report the largest number of Facebook users exist within Egypt followed by Saudi<br />
Arabia and in both cases SNS have been used to mobilize collective action toward breakthroughs in<br />
social, educational, economic and political contexts as discussed above.<br />
By comparison in Iraq, internet penetration is amongst the lowest in the Middle East with just over 1%<br />
of the population being able to access the Internet. Therefore the Iraqi use of SNS is relatively low<br />
although there is growth of membership within the country.<br />
It is important to note also that whilst there is an increasing number of Muslim based SNS these<br />
appear to have relatively low membership and are not a preferred means of networking for the<br />
majority of the Arabic population. (Dashan, 2011) The majority of Arabic users appear to prefer sites<br />
such as Facebook which have promoted international use through marketing and the provision of 43<br />
translations of the site. (Facebook, 2011)<br />
The reasons for lower usage in Iraq whilst numerous are likely to include political instability and the<br />
Iraq War which led to the destruction of the ICT infrastructure of the country which is only now more<br />
thoroughly restored.<br />
Due to the clear differentiation between these two countries this paper shall focus on Egypt and Iraq<br />
as the highest and lowest users of SNS within the Middle East.<br />
2. Knowledge Management and Knowledge Sharing<br />
KM and KS have been the subject of much research and scrutiny in recent years. KM can largely be<br />
defined from two perspectives. The first has its roots in the concepts of Artificial Intelligence (AI) and<br />
subsequent Information Systems (IS) developments.<br />
This perspective relies on the ability to codify knowledge in order to share it (Barnes, 2007; Snowden,<br />
2005; Schutt, 2003) and therefore relies on knowledge in explicit form, not dealing with tacit forms.<br />
(Hildreth & Kimble, 2002)<br />
285
Elaine Garcia et al.<br />
The process by which tacit knowledge can be shared is therefore the subject of the second<br />
generation of KM which is particularly relevant where products are largely intangible and often shared<br />
through a process of socialisation. (Nonaka & Takeuchi, 1995).#<br />
It is therefore the nature of SNS as a popular web 2.0 tool which encourages and supports<br />
socialisation that makes systems such as Facebook a relevant proposition for KM.<br />
3. Social Networking Sites<br />
In terms of KS the main benefit from SNS is the manner in which they ‘would let us see who knows<br />
what, who does what and even who controls what and update us continuously on all of these fronts.’<br />
(Donath, 2007)<br />
Additionally, the popularity of existing sites suggests that, the use of SNS would be relatively straight<br />
forward. However the motivation for existing use may not always be compatible with organisational<br />
KM and may lead to conflicted motivation for continued use ultimately affecting the ability of KS to<br />
occur. (OFCOM, 2008; Ardichvilli, Page & Wentling 2003)<br />
The main issue is ‘what makes a social network function isn’t so much the tools as it is the attitude<br />
you’ve got to have to “want to” and that isn’t something you can get out of a box.’ (Suarez, 2007) In<br />
terms of existing users the motivation to use SNS as a tool for KS may not necessarily be compatible<br />
with primary usage. (Ardichivilli, Page & Wentling, 2003)<br />
Further important factors affecting the motivation for SNS use will relate to the desire for individuals to<br />
have ownership, trust and privacy of their information and knowledge.<br />
For non-users privacy concerns will largely relate to issues of personal safety (OFCOM, 2008; Hewitt<br />
& Forte, 2006) and a desire not to share personal information with a wide group, particularly if they<br />
are not chosen but due to shared organisational connections.<br />
For existing users privacy concerns are likely to relate to the combining of personal and public use<br />
particularly as others can inform what is posted and therefore visible about an individual. (Dwyer et al,<br />
2007; Barnes, 2007)<br />
Furthermore privacy concerns could relate to the sharing of proprietary information or even libellous<br />
or defamatory comments across the internet which could prove to be a problem both individually and<br />
organisationally. (APQC, n.d; Go, 2007)<br />
In terms of KS, barriers may also exist from the fundamental way in which these sites operate.<br />
Information posted can become the property of the software company and therefore its unauthorised<br />
reuse cannot be guaranteed. (Jones & Soltren, 2005; Schonfeld, 2009)<br />
Any information posted is also persistent and accumulative making it searchable, replicable and<br />
available to any number of invisible audiences for an indeterminate period of time. (Cain, 2007;<br />
Barnes, 2007, Dwyer et al., 2007)<br />
McConnell (2008) however considers that there is a great deal of “paranoia” concerning the privacy<br />
and inappropriate use of personal data on the internet and a balanced view of these issues is<br />
required.<br />
Whilst a number of barriers are considered perhaps the most prevalent area for consideration must be<br />
the nature of relationships and specifically the manner in which weak ties can be utilised and<br />
strengthened through the use of a SNS. (Donath, 2007; Boyd & Ellison, 2007)<br />
The strength of weak ties within an SNS are likely to lead to a large network especially where existing<br />
users already hold a personal network of friends and family. However it is important also to consider<br />
that networks can become too large and unmanageable.<br />
Donath (2007) places the maximum number of relationships an individual can have at 150. In a large<br />
organisation this could be easily exceeded when considering that the average Facebook user will<br />
already have 120 confirmed friends. (Sandberg, 2009)<br />
286
Elaine Garcia et al.<br />
The nature of organisations and the limited maximum number of relationships could lead to the<br />
creation of silos as homogenous groups remain separated and offline relationships are simply<br />
replicated online, no further enhancing KS or communication across the organisation. (Boyd & Ellison,<br />
2009)<br />
In addition there are likely to be a number of individuals who are less familiar with technology and who<br />
may become segregated from the KS activities of the organisation resulting in a lack of completeness<br />
across the organisation.<br />
The nature of technology acceptance and use represents a fundamental barrier to the use of SNS as<br />
a KM tool and indicates the need for adequate training, technical support and senior management<br />
support to ensure that individuals who may not traditionally use SNS will be encouraged to trial use<br />
within an organisational context.<br />
4. Methodology<br />
Generally previous studies have used a qualitative approach to identify the barriers to the use of SNS<br />
as a KM tool. This approach is most appropriate due to the assumption that access to the reality of a<br />
situation is only possible through social construction. (Smith et al, 2008)<br />
A qualitative approach seeks to answer questions posed by studying different social settings. As<br />
noted by Berg (2009), qualitative techniques make it possible for researchers to participate in<br />
understanding and perceiving others, as well as permitting them to discover how people structure<br />
their daily lives to make them more meaningful.<br />
According to Creswell and Clark (2011) the advantages of utilising this approach are numerous:<br />
It provides a suitable means of studying complex phenomena.<br />
It attempts to interpret and describe people’s personal experiences of phenomena.<br />
It tries to understand the process of interaction between individuals.<br />
It allows participants to share their views.<br />
Therefore the validity of the results of this approach is high (Creswell, 2009).<br />
4.1 Case study<br />
Case studies are increasingly used as research tools (Creswell, 2007) and according to Yin (2009)<br />
investigate holistically and feature real-life events. Moreover, case studies are uniquely strong as they<br />
are able to deal with a variety of evidence, i.e. documents, interviews, etc (Saunders et al., 2009),<br />
thus providing a more detailed view of the issue under consideration (Yin, 2009). In addition<br />
Siggelkow (2007) indicates that case studies are beneficial in order to understand the significance of<br />
the research questions posed and to encourage new ideas.<br />
In order to investigate the drivers and barriers that affect the use of SNS as a KS tool in order to<br />
develop HE in Egypt and Iraq, case studies will be undertaken, one institution in Egypt and one in<br />
Iraq.<br />
4.2 Interviews<br />
In this case study interviews, which Gray (2009) defines as a conversation between people, whereby<br />
one party assumes the role of the researcher, will be used. According to Hair et al. (2007) this method<br />
is helpful where issues are complex as it enables the researcher to receive feedback.<br />
In both institutions semi-structured interviews were conducted with participants enabling the<br />
researchers to explore and investigate opinions from respondents, giving them the opportunity to both<br />
analyse in-depth and expand upon their answer (Gray, 2009)<br />
This method appeared to be suitable as it elicits the assumptions and stories participants hold about<br />
the issues under consideration.<br />
287
4.3 Data analysis<br />
Elaine Garcia et al.<br />
A total of 40 managers within Al-Mustansirya University in Iraq and Mansoura University in Egypt<br />
were interviewed using open-ended questions and consisted of Deputy Deans (DD), Heads of<br />
Department (HD), <strong>Academic</strong> Heads (AH) and academic staff (AS).<br />
The interview results were analysed using narrative analysis. This method is defined by Saunders et<br />
al. (2009) as ‘an account of experience that is told in a sequential way, indicating a flow of related<br />
events that, taken together, are significant for the narrator and which convey meaning to the<br />
researcher.’ This method allows the stories people employ in recounting events to be emphasized<br />
and analysed.<br />
The advantages of this method include factors such as its expedient nature and the way it allows an<br />
examination of the social background of the interviewee, the relationship between individuals and the<br />
interviewee’s views of the organisation (Smith et al., 2008).<br />
5. Case study institutions<br />
5.1 Mansoura University in Egypt – Faculty of Commerce<br />
Mansoura University was established in 1972 under the name East Delta University but had existed<br />
as a branch of Cairo University since 1962 and is now one of 17 State owned Universities within<br />
Egypt.<br />
The Faculty of Commerce was established in 1973/1974 and currently has 22,000 students enrolled.<br />
After the civil unrest within Egypt in January 2011 a small number of academic staff at the institution<br />
established a Facebook group for the purpose of networking and KS within the faculty with an aim to<br />
assist in the development of HE within the institution. Currently this group has 123 members.<br />
5.2 Al-Mustansirya University in Iraq - Faculty of Economy and Administration<br />
Al-Mustansirya University is affiliated to the Ministry of HE and Scientific Research and was<br />
established in 1963.<br />
The Faculty of Economy and Administration was founded in 1963. The Faculty has had a relatively<br />
variable enrolment level in the last few years and current enrolments stand at 733 students.<br />
The institute faces a number of challenges and changes, in terms of its structure, curriculum,<br />
approaches and resources particularly due to political instability over the last two decades.<br />
6. Findings<br />
6.1 Existence of KS strategy<br />
From the interviews, it appears a contradiction exists regarding the existence of a KS strategy within<br />
Al-Mustansirya. Some interviewees for example, claim there is no strategy for KS within the University<br />
whilst others consider a strategy does exist under an alternative name.<br />
Largely those further down the organisation consider a strategy does exist indicating a lack of clarity,<br />
a lack of communication or a desire for interviewees to appear to be aware of potential strategies of<br />
which they have no knowledge.<br />
The Deputy Dean did however state:<br />
“There is no particular strategy, but I think there are recognizable priorities for KS.”<br />
(DD.2)<br />
This may therefore lead to others to believe that these priorities represent the KS strategy within the<br />
organisation. The Deputy Dean also stated that such a strategy was imperative in the future, reflecting<br />
the acceptance of the need for a KS strategy.<br />
Within Mansoura similar results were found with general staff considering a KS strategy exists and<br />
only one senior manager agreeing that it does. Two senior managers did however identify the need to<br />
focus on informal not formal KS providing an indication of some awareness of KM approaches and<br />
strategies.<br />
288
Elaine Garcia et al.<br />
In comparison it is clear there is greater clarity within Mansoura regarding the existence of a strategy<br />
for KS in addition to the need to focus on both formal and informal KS although in both institutions<br />
confusion existed as to the existence of a KS strategy reflecting different results from senior<br />
managers and other staff.<br />
6.2 Acceptance of use<br />
In relation to the acceptance of use, the findings from Al-Mustansirya clearly indicated that the<br />
University does not encourage the use of SNS and resistance to use exists across the organisation.<br />
Staff across the University acknowledged that SNS were becoming widespread amongst student<br />
communities but this led to an expectation that SNS can solely be used for social purposes and are<br />
therefore inappropriate for work use.<br />
Some managers could comprehend that SNS use for exchanging and disseminating knowledge was<br />
useful and thus SNS use could benefit the institution but on the whole inappropriate comments,<br />
issues of trust and privacy in addition to concerns about the low levels of technological capabilities of<br />
staff all act as barriers to use.<br />
It should be noted however that technological capabilities appear to be influenced by age as a Head<br />
of Department within Al-Mustansirya stated:<br />
“Some of the staff do not like to enter these sites due to their age and they lack<br />
understanding about how to use technology...... By contrast, these sites are used<br />
frequently by young staff” (HD1).<br />
In Mansoura University however all interviewees agreed that SNS could have a large role in future KS<br />
and particularly for the dissemination of knowledge, between staff or between staff and students to<br />
the extent that:<br />
“People feel free and relax when they express their views through this medium more than<br />
face to face communication” (SM.1)<br />
The key differences between the two institutions appear to relate to technological capabilities which in<br />
Mansoura ranges between good and acceptable although again there are higher skills levels seen<br />
amongst younger staff that tend to be more familiar with the technology.<br />
In terms of other barriers to use the two institutions are relatively similar with issues of bullying,<br />
security, privacy and trust all prevalent within Mansoura in addition to Al-Mustansirya.<br />
Within Mansoura some concerns were often expressed by staff relating to the triviality of SNS:<br />
“There are a lot of issues raised on SNS that are trivial and to read or comment on it is<br />
just wasting time”. (AS5).<br />
This may, as in Al-Mustansirya, reflect concerns relating to the social nature of SNS and the types of<br />
communications and information that is shared on these sites.<br />
Through the interviews it became clear that the use of SNS is associated with negative images and<br />
culture in the minds of some people generally as a reflection of religious views within the region:<br />
“The negative culture of the users of SNS is a big barrier for use” (AS1)<br />
In practice this means that if you are using SNS you are immoral and this is especially the case for<br />
female users. This view is reflected by the manner in which most female academic users do not put<br />
their real pictures on the sites and are currently underrepresented on SNS.<br />
Within the Mansoura University, Faculty of Commerce group on the facebook which contain 123<br />
members of staff, only five of the forty female members put their photo on the site and of these three<br />
are with their husbands. This was expressed by one of the female members of academic staff who<br />
said that:<br />
“.... One of the most serious issues for female users is the privacy”. (AS9).<br />
Thus it can be concluded that the lack of acceptance of SNS within both institutions is the result of a<br />
lack of technological skill (linked to age), concerns relating to bullying or inappropriate comments,<br />
289
Elaine Garcia et al.<br />
issues of privacy, trust and security, and, the ambiguity of perceived benefits aside from social<br />
relationships.<br />
The issues of age and gender are important considerations within the context of SNS in both<br />
institutions and can be difficult to measure due to the nature of these groups where females and<br />
elderly members of staff are underrepresented within both institutions either due to the familiarity with<br />
the technology or due to the threat of privacy.<br />
Issues of gender are likely to be directly linked to religious, cultural and social norms within Arab<br />
states and therefore may continue to exist or take time to change as political and social change<br />
occurs within the region.<br />
Issues of gender may also be linked to age as due to these sites only recently entering the Arab<br />
world, younger generations are more conversant and comfortable with them and this may result in<br />
greater acceptance as the technology and population matures.<br />
6.3 Motivation for use<br />
In terms of motivation for use within Al-Mustansirya there appears to be a contradiction regarding the<br />
extent to which KS is encouraged within the institution for whilst the Deputy Dean and Departmental<br />
Heads indicated KS is open and encouraged, academic staff claim it is constrained:<br />
“We always hear the term “open culture” at work but if you try to share ideas with<br />
somebody higher than you in the organisational structure you will find a wall” (AS.8).<br />
However, all interviewees agreed motivations for use will include benefits such as networking, selfexpression<br />
and KS enhancement particularly in informal environments.<br />
There was additionally however some resistance to motivation for use due to some of the issues<br />
previously found such as the perception of sites as solely social:<br />
“I consider these sites social. It is custom only to use them only for communicating with<br />
friends and relatives” (AS.7).<br />
However within Mansoura, where some use of SNS is already occurring and common as indicated by<br />
one interviewees who did consider the social aspect of SNS to be useful for KS:<br />
“SNS are useful in increasing the social bond of the staff which increases job satisfaction<br />
and makes the work environment more likable” (AS3).<br />
Furthermore within Mansoura the existing use of SNS appears to a greater understanding of potential<br />
benefits as all interviewees agree that Mansoura is already benefiting from SNS use. These benefits<br />
vary from networking, global KS, speed of communication, constructive feedback, supporting<br />
innovation, generating new ideas, mass opinion polling cheaply, developing more social/collaborative<br />
views of <strong>learning</strong>, improved motivation, engagement , creating a connection to real-life <strong>learning</strong> and<br />
finally flexibility of use as one of the interviewees at Mansoura commented:<br />
“SNS technology is already available free of charge to many people either at home, work<br />
or even while they are on move. This makes it very easy and quick to use” (AS1).<br />
The existing use within Mansoura also highlights the role that senior managers play in motivating staff<br />
to use such sites as whilst senior managers consider that SNS can act as a sensor to evaluate the<br />
popularity of their decisions and as a way to disseminate information to staff:<br />
“SNS are a good way to know the problems within the school and respond to it quickly”<br />
(SM5).<br />
However from observation of the Mansoura Faculty group within Facebook and the interviews<br />
undertaken it would appear that the contributions from senior managers are relatively limited and this<br />
is noticed by staff:<br />
I felt that the use of SNS by senior management within the school is just as a publicrelations<br />
tool rather than a means of communication and solving people problems”<br />
(AS11).<br />
290
Elaine Garcia et al.<br />
It would appear that this lack of contribution is considered to also indicate a lack of commitment from<br />
senior managers and this is considered by academic staff to result in confusion and a lack of clear<br />
direction:<br />
“Lack of purpose and clarity of common goals of using these websites in terms of the<br />
conflict of the nature of use .......will continue in the future until we have more<br />
commitment from both senior management and academic staff” (AS6).<br />
To conclude it is clear that there are no clear policies in place within either institution to adopt and use<br />
SNS as a KS tool however the organic growth of a community of this nature within Mansoura would<br />
appear to suggest that the institution is in the early stages of adoption and may be of significance in<br />
the future.<br />
It is also clear from the results that Arabic culture and social norms play important roles in the use of<br />
SNS within these countries. Females in both countries tended to be more concerned about online<br />
privacy than males. There is, nonetheless, a general acceptance and recognition that SNS encourage<br />
and enable discussion, communication and KS.<br />
7. Discussion<br />
From the findings it is clear that these institutions are at very differing points of development in terms<br />
of SNS usage, KS strategy and technology usage.<br />
To some extent the differences appear to highlight the importance of senior management support for<br />
whilst Mansoura appears to have a KS strategy and SNS policy in place and thereby some<br />
acceptance of SNS use, Al-Mustansirya has neither. This therefore appears to highlight the<br />
importance of senior management support, as identified in the literature, for achieving the use of SNS<br />
for KS.<br />
From both institutions issues of privacy, trust and openness seem to be prevalent. This interestingly<br />
relates to concerns identified in the previous literature but there is also a cultural dimension relating to<br />
issues of gender that are evident within Arab nations as a wider cultural issue. This consideration is<br />
not one which has been given a great deal of attention within the area of SNS literature as the<br />
majority of previous studies focus upon the role of SNS within the developed world; largely within the<br />
USA and UK.<br />
In relation to privacy and trust, as within previous studies, there appear to be a number of concerns<br />
relating to the informal nature of SNS and the manner in which users are required to combine<br />
personal and professional profiles. In addition it would appear that the inclusion of “social” within the<br />
name of such tools gives users a sense that the sites are not serious and cannot be used for work.<br />
This view is supported in both institutions. This therefore suggests that the name of the site is value<br />
laden and represents a barrier to use.<br />
Technological barriers appear to be present within both institutions but are to some extent related to<br />
age. Increased barriers to use relating to technology within Al-Mustansirya may exist more due to the<br />
nature of the organisation, its culture and strategy than necessarily in relation to individual usage<br />
although previous literature has suggested that age may be a factor.<br />
Within Al-Mustansirya for example there is no clear KS strategy and this perhaps reflects that the<br />
institution does not know how knowledge transfer can be accomplished effectively and certainly how<br />
SNS could assist in enabling this to occur.<br />
The two institutions appear to be at different stages of cultural acceptance of SNS as would be<br />
predicted from the number of existing users of Facebook within both countries. Whilst Egypt appears<br />
to be reporting benefits already from using such tools and support comes from the possibilities of<br />
social bonding, free software and easily accessible sites, within Iraq benefits relate to networking,<br />
self-expression and knowledge enhancement but its role outside that of a communication tool for<br />
friends and family is difficult for interviewees to comprehend.<br />
Overall the findings appear to suggest that the higher usage of SNS within Egypt does result in a<br />
greater acceptance and understanding of the role that SNS could play in KS whilst the slower gain in<br />
usage within Iraq seems to result in lower acceptance of formal use within the workplace.<br />
291
8. Conclusions and recommendations<br />
Elaine Garcia et al.<br />
This research has examined the drivers and barriers to using SNS for KS to develop HE in developing<br />
countries. The study found that there are contradictions and differences in viewpoints concerning the<br />
application of SNS as a KS tool and therefore factors which act as both barriers and drivers to usage<br />
within developing countries.<br />
Within both institutions studied there is clear evidence of the need to ensure that KS strategies are put<br />
in place and for this to be achieved greater senior manager support will be vital. Within Mansoura the<br />
manner in which SNS adoption is already occurring and SNS are considered to be valuable for KS<br />
particularly demonstrates the advantages that can be gained. Potentially the sharing of this best<br />
practice within and between the two institutions considered here and more widely than this may be<br />
one way in which the promotion of KS strategies and SNS could be achieved. This would be<br />
particularly effective if SNS was the vehicle used to promote such activities as it would demonstrate<br />
the manner in which SNS enable institutions to create global communities conveniently and cheaply.<br />
Using SNS as the vehicle to share best practice would not however overcome the barriers that exist<br />
due to lack of technological and IT skills particularly seen within the Iraqi University. It is expected that<br />
as the technological infrastructure of Iraq is more fully developed the country will become more<br />
technologically capable however it will be important that technology is highlighted within the institution<br />
and where possible training, assistance and the promotion of technology as a tool to assist in KS will<br />
be undertaken. This, to some extent will depend upon budget constraints and wider investment within<br />
the country. It may however be possible for the institution to make use of mobile technologies, which<br />
are usually more reliable and available throughout the country, to demonstrate the use of SNS and<br />
the ease of use of such technology, particularly to older staff that it appeared were less accepting of<br />
technology use than younger staff.<br />
Whilst senior management support and technological barriers should be relatively easy to overcome<br />
the issues of privacy and trust, particularly relating to cultural and social norms within the region will<br />
be harder to overcome.<br />
Again the development of best practice groups and exposure to SNS will undoubtedly alleviate some<br />
of these concerns, particularly if privacy settings and user controls are explained and a<br />
recommendation for appropriately sensitive privacy settings can be provided. Furthermore if an<br />
institution wide approach were to be taken and agreement concerning pictures and personal details to<br />
be included were agreed by all affected groups, but to be implemented by all staff, this may assist in<br />
both exposing individuals to and determining the appropriateness of SNS use within the institution.<br />
Once again the sharing of best practice between institutions may be useful in enabling issues such<br />
cultural and social norms to be discussed and previous experience by some may be promote future<br />
usage by others through discussion of concerns and the manner in which these may have already<br />
been overcome within another institution.<br />
It is important to note particularly that the use of sites that have been developed specifically for<br />
Muslim communities do not appear to date to have been popular however this may be due to the lack<br />
of interest in such sites due to the technological barriers discussion above. It may therefore be of<br />
interest for institutions with specific social and cultural issues to investigate the use of such sites and<br />
more successful sites such as Facebook as either may represent a potential SNS solution for the<br />
institution and an integration of both may provide an optimum solution if it can be achieved.<br />
Overall there was broad agreement that there were advantages to employing such sites, given that<br />
SNS aim to facilitate KS amongst users. The research therefore concludes that the Iraqi University<br />
should adopt SNS and draw upon their benefits as a KS tool. Additionally, for effective change the<br />
institution should work to achieve clear communication amongst staff and implement strategic plans to<br />
encourage the KS during working hours. The time has come for the institutions not only to dispel the<br />
ignorance of SNS but also to make academic staff aware of their necessity. The adoption of new<br />
strategies to seize SNS is an absolute must – to the irrefutable benefit of senior staff in their bid to<br />
accomplish an elevated echelon of communication with all other affiliates of their institutions.<br />
292
Acknowledgements<br />
Elaine Garcia et al.<br />
This project has been partially funded by Plymouth College of Art<br />
References<br />
Alexander, A. (2011) Internet role in Egypt’s protests, [Online], http://www.bbc.co.uk/news/world-middle-east-<br />
12400319 [Accessed 10/06/11]<br />
APQC. (n.d.) The Growing popularity of Social Networking and Expertise Location, [Online],<br />
http://kmedge.org/homepage.html [Accessed 10/05/08]<br />
Ardichvili, A., Page, V. & Wentling, T. (2003) Motivation and barriers to participation in virtual knowledge-sharing<br />
communities of practice, Journal of Knowledge Management, 7, pp164-177<br />
Arthur C. (2011) Facebook growth slows for the second month in a row, [Online],<br />
http://www.guardian.co.uk/technology/2011/jun/13/facebook-growth-slows-for-second-month [Accessed<br />
13/6/11]<br />
Barnes, S. (2007) A privacy paradox: Social networking in the United States, [Online],<br />
http://www.firstmonday.org/issues/issue11_9/barnes/index.html [Accessed 01/06/08]<br />
Beaumont, P. (2011) The truth about Twitter, Facebook and the uprisings in the Arab world, [Online],<br />
http://www.guardian.co.uk/world/2011/feb/25/twitter-facebook-uprisings-arab-libya [Accessed 10/06/11]<br />
Berg, B. (2009) Qualitative Research Methods For the Social Sciences, 7th edition, Pearson Education<br />
Company: USA<br />
Blair, K. (2011) Behind Bin Laden: The Role of Social Media, [Online], http://socialtimes.com/behind-bin-ladenthe-role-of-social-media_b60489<br />
[Accessed 10/06/11]<br />
Boyd, D. & Ellison, N. (2007) Social Network Sites: Definition, History, and Scholarship, Journal of Computer-<br />
Mediated Communication, No. 13 [Online], http://jcmc.indiana.edu/vol13/issue1/boyd.ellison.html [Accessed<br />
17/05/10]<br />
Cain, J. (2007) Online Social Networking Issues within Academia and Pharmacy Education, American Journal of<br />
Pharmaceutical Education, No. 72 [Online],<br />
http://findarticles.com/p/articles/mi_qa3833/is_200801/ai_n24394683/pg_1?tag=artBody;col1 [Accessed<br />
15/05/08]<br />
Creswell, J. (2007) Qualitative inquiry research design- choosing among five approaches, SAGE Publication<br />
LTD: London<br />
Creswell, J. (2009) Research Design: qualitative, quantitative, and mixed methods approaches, SAGE<br />
Publications, Inc: USA<br />
Creswell, J. & Clark, V. (2011) Designing and conducting mixed methods research, SAGE Publication, Inc: USA<br />
Dashen, M. (2011) Facebook has little to fear from Muslim social networking sites, [Online],<br />
http://www.guardian.co.uk/commentisfree/belief/2011/may/31/muslim-social-networking [Accessed 10/6/11]<br />
Delany, C. (2011) How Social Media Accelerate Tunisia’s Revolution: An Inside View, [Online],<br />
http://www.huffingtonpost.com/colin-delany/how-social-media-accelera_b_821497.html [Accessed 10/06/11]<br />
Donath, J. (2007) Signals in Social Supernets, Journal of Computer-Mediated Communication, No. 13, pp231-<br />
251<br />
Dwyer, C., Hiltz, S. & Passerini, K. (2007) Trust and Privacy concerns within social networking sites: A<br />
comparison of Facebook and MySpace, [Online],<br />
http://csis.pace.edu/~dwyer/research/DwyerAMCIS2007.pdf [Accessed 01/06/08]<br />
Facebook. (2011) Facebook: My Account – Primary Language, [Online],<br />
https://www.facebook.com/#!/editaccount.php?language [Accessed 13/06/11]<br />
Go, A. (2007) The Paper Trail, [Online], http//usnews.com/blogs/paper-trail/2007/2/27/columnist-says-he-wasfired-for-facebook=comments.html<br />
[Accessed 21/07/08]<br />
Gray, D. (2009) Doing research in the real world, SAGE Publication Ltd: London<br />
Hewitt, A. & Forte, A. (2006) Crossing Boundaries: Identity Management and Student/Faculty Relationships on<br />
the Facebook, [Online], http://www-static.cc.gatech.edu/~aforte/HewittForteCSCWPoster2006.pdf<br />
[Accessed 01/06/08]<br />
Hildreth, P & Kimble, C. (2002) The duality of knowledge, Information Research, Vol. 8, No. 1<br />
Hill, A. (2011) Osama bin Laden killed – how a live blogger captured the raid, [Online],<br />
http://www.guardian.co.uk/world/2011/may/02/osama-bin-laden-live-blogger [Accessed 10/06/11]<br />
Idle, N & Nunns, A. (2011) Tahrir Square tweet by tweet, [Online],<br />
http://www.guardian.co.uk/world/2011/apr/14/tahrir-square-tweet-egyptian-uprising [Accessed 10/06/11]<br />
Jones, H & Soltren, H. (2005) Facebook: Threats to Privacy, [Online],<br />
http://groups.csail.mit.edu/mac/classes/6.805/student-papers/fall05-papers/facebook.pdf [Accessed<br />
19/05/08]<br />
Liao, S., Chang, J., Cheng, S. & Kuo, C. (2004) Employee relationship and knowledge sharing: A case study of a<br />
Taiwanes finance and securities firm, Knowledge management and practice, vol.2, pp24 - 34<br />
Mathew V. (2010) Service delivery through knowledge management in higher education, Journal of knowledge<br />
Management practice, vol.11. no. 3, pp1-14<br />
McConnell, B. (2008) Social Networks, from 80s to the 00s, [Online], http://gigaom.com/2008/01/20/socialnetworks-from-the-80s-to-the-00s/<br />
[Accessed 15/05/08]<br />
293
Elaine Garcia et al.<br />
Nabi, A. (2011) Facebook protester: ‘A lot of people love Mubarak and want to defend him, [Online],<br />
http://www.guardian.co.uk/world/2011/may/19/egypt-100-days-on-facebook-protester-mubarak [Accessed<br />
10/6/11]<br />
Nonaka I. O. & Takeuchi, H. (1995) The knowledge-creating company: how Japanese companies create the<br />
dynamics of innovation, Oxford University Press: New York<br />
OFCOM. (2008) A quantitative and qualitative research report into attitudes, behaviours and use, [Online],<br />
http://www.ofcom.org.uk/advice/media_literacy/medlitpub/medlitpubrss/socialnetworking/report.pdf<br />
[Accessed 03/05/08]<br />
Sandberg, S. (2009) How many friends can you have? [Online],<br />
https://www.facebook.com/blog.php?post=72975227130 [Accessed 13/06/11]<br />
Saunders, M., Lewis, P. & Thornhill, A. (2009) Research Methods for Business Students, Pearson Education<br />
<strong>Limited</strong>: England<br />
Schonfeld, E. (2009) Zuckerberg on who owns user Data on Facebook: it’s Complicated, [Online],<br />
http://technews.am/conversations/techcrunch/zuckerberg_on_who_owns_user_data_on_facebook_it_s_co<br />
mplicated [Accessed 13/06/11]<br />
Siggelkow, N. (2007) Persuasion with case studies, Academy of management Journal, vol.50 no.1, pp20-24<br />
Smith, M., Thorpe, R. & Jackson, P. (2008) Management research, Sage Publication Ltd: London<br />
SPOT ON PR. (2010) Middle East & North Africa Facebook Demographics – May 2010, [Online],<br />
http://www.spotonpr.com/wp-content/uploads/2010/05/FacebookMENA_24May10.pdf [Accessed 11/06/11]<br />
Sutter, J. (2011) How bin Laden news spread on Twitter, [Online], http://articles.cnn.com/2011-05-<br />
02/tech/osama.bin.laden.twitter_1_bin-tweet-twitter-user?_s=PM:TECH [Accessed 11/06/11]<br />
Yin, R. (2009) Case study Research- Design and Methods, 4th Edition, SAGE Publications, Inc: Los Angeles,<br />
294
The Relationship Between Mindful Learning Processes and<br />
Course Outcomes in Web-Based Learning<br />
Danny Glick and Roni Aviram<br />
Ben-Gurion University, Beer-Sheva, Israel<br />
glickd@bgu.ac.il<br />
roniav@zahav.net.il<br />
Abstract: Online education has been growing rapidly within all levels of education. Reports published in 2007-<br />
2009 by the Sloan Consortium based on responses from over 2,200 colleges and universities in the US reveal<br />
that institutions report record online enrollment growth on both a numeric and a percentage basis. Nearly 96% of<br />
the very largest institutions, schools with 15,000 enrollments, have some online offerings, and about two-thirds of<br />
the very largest institutions have fully online programs. The Association of International Educators predicts that<br />
by 2020, global higher education demand for seats will reach 200 million. Much of this growth will be in distance<br />
education. While online education is growing rapidly, the impact of online <strong>learning</strong> on course outcomes has been<br />
subject to considerable debate. In 1983, Richard E. Clark famously argued that media have no more effect on<br />
<strong>learning</strong> than a grocery truck has on the nutritional value of the produce it brings to market. Recent research<br />
findings support Clark’s statement revealing that students taking online courses score lower than students in<br />
face-to-face or blended courses. The research hypothesis is that the main reason for the little impact online<br />
<strong>learning</strong> has on <strong>learning</strong> outcomes is lack of Mindful Learning Processes (MLP), a new term coined by the<br />
authors of this study. The authors define MLP as an online course that is based on four interconnected<br />
components: <strong>learning</strong> theory, <strong>learning</strong> environment, <strong>learning</strong> process, and assessment. The goal of this paper is<br />
to check whether the research hypothesis may be accepted. To achieve that, the authors have reviewed case<br />
studies describing online courses published in peer-reviewed journals in the area of educational technology and<br />
e-<strong>learning</strong>. They authors have identified three case studies that fulfilled the MLP requirements stated above, and<br />
analyzed them in terms of <strong>learning</strong> theory, <strong>learning</strong> processes, <strong>learning</strong> environment and assessment. To<br />
measure the impact of the MLP on course outcomes, the authors analyzed these case studies in terms of test<br />
scores and students’ perception of the e-<strong>learning</strong> experience. Analysis of the three case studies reveals that the<br />
outcome of the three case studies was positive in terms of achievement outcomes and students’ opinion about<br />
course. The authors conclude that a Mindful Learning Process may lead to a meaningful and pedagogically<br />
sound course where theory, design, <strong>learning</strong> and assessment harmonize with each other. These findings support<br />
earlier work on instructional technology that found that the reason for the <strong>learning</strong> benefits of computer media is<br />
not the medium of instruction, but the instructional strategies built into the <strong>learning</strong> materials.<br />
Keywords: Mindful Learning Process, web-based <strong>learning</strong>, impact on <strong>learning</strong>, text analysis, review of research,<br />
<strong>learning</strong> outcomes<br />
1. Introduction<br />
Online education is growing rapidly at all levels of education (Allen & Seaman, 2010; International<br />
Association for K-12 Online Learning, 2010; Watson, 2007). A report published by Allen & Seaman<br />
(2010) on Web-based <strong>learning</strong> in higher education, based on responses from over 2,500 colleges and<br />
universities in the United States, reveals that institutions report record online enrollment growth on<br />
both a numeric and a percentage basis. Nearly 5.6 million students were taking at least one online<br />
course during the fall 2009 term, which is an increase of nearly one million students over the number<br />
reported in the report from the previous year. The 21% growth rate for online enrollments far exceeds<br />
the less than 2% growth of the overall higher education student population. Nearly 30% of highereducation<br />
students now take at least one course online (Allen & Seaman, 2010).<br />
Research indicates that educators and decision-makers have embraced online <strong>learning</strong>. Allen and<br />
Seaman (2010) report reveals that 63% of all reporting institutions said that online <strong>learning</strong> was a<br />
critical part of their institution’s long-term strategy. According to Rovai & Downey (2010), many<br />
universities have increased their recruiting efforts to reach a larger and more diverse audience. Some<br />
universities also extend their reach with cross-border initiatives and seek international students in<br />
order to promote enrollment growth and global <strong>learning</strong>. The economic potential of distance education<br />
and academic globalization has attracted numerous higher education providers, many of which<br />
operate on a for-profit basis (Rovai & Downey, 2010).<br />
The Association of International Educators predicts that by 2020, from a year 2000 base, global<br />
higher education demand for seats will double to 200 million, suggesting a growth market in<br />
international higher education. Much of this growth will be in distance education. Consequently, an<br />
increasing number of universities are transforming themselves from single-mode on-campus<br />
295
Danny Glick and Roni Aviram<br />
universities to dual-mode universities as they acknowledge the importance of distance education and<br />
the opportunities it provides (Redden, 2009).<br />
A recent report published by The International Association for K-12 Online Learning (2010) reveals<br />
that online <strong>learning</strong> in K-12 schools is growing explosively. Supplemental or full-time <strong>learning</strong><br />
opportunities are available to students in 48 states across the USA. 27 states, as well as Washington,<br />
DC, have state-wide full-time online schools. Moreover, many virtual schools in the US show annual<br />
growth rates between 20% and 45%.<br />
The number of virtual universities offering full-time academic degrees is growing rapidly. Universities<br />
in Latin America (www.unad.edu.co), North America (www.phoenix.edu), Europe (www.uned.es), Asia<br />
(www.english.cyberkorea.ac.kr) and Oceania (www.curtin.edu.au) offer numerous online academic<br />
courses in a wide range of departments and fields such as languages, education, finance, heath and<br />
engineering.<br />
The impact of technology and online <strong>learning</strong> on education is great. Hutti (2007) claims that<br />
technology has impacted on <strong>learning</strong> at an extraordinary pace, perhaps like no other innovation in<br />
previous years, decades or centuries. Distance education, specifically virtual classrooms, may have a<br />
greater impact on the nature of higher education than any innovation since the invention of the<br />
printing press (Hutti, 2007).<br />
In light of the strategic role online <strong>learning</strong> plays at all levels of education, one would expect webbased<br />
environments to offer quality instruction. However, recent research indicates that students<br />
taking online courses score lower than students in face-to-face or blended courses. Moreover, recent<br />
meta-analyses comparing students’ attitude towards online and classroom instruction reveal that there<br />
is significant effect on overall attitude outcomes in favor of classroom instruction.<br />
2. The little impact online <strong>learning</strong> has on <strong>learning</strong> outcomes and student<br />
satisfaction – review of relevant literature<br />
A meta-analysis published by the US Department of Education (2009) reviewing online <strong>learning</strong><br />
studies from 1996 through 2008, identified 51 independent effect sizes (EF) that could be subjected to<br />
meta-analysis. Meta-analysis is a technique for combining the results of multiple experiments or<br />
quasi-experiments to obtain a composite estimate of the size of the effect. The result of each<br />
experiment is expressed as an effect size, which is the difference between the mean for the treatment<br />
group and the mean for the control group, divided by the pooled standard deviation (Bernard et al.,<br />
2004; Shachar, 2008; US Department of Education, 2009). The meta-analysis found that instruction<br />
combining online and face-to-face elements had a larger advantage relative to purely face-to-face<br />
instruction than did purely online instruction. The mean effect size in studies comparing blended with<br />
face-to-face instruction was +0.35, p < .001. This effect size is larger than that for studies comparing<br />
purely online and purely face-to-face conditions, which had an average effect size of +0.14, p < .05<br />
(US Department of Education, 2009). This finding suggests that blended <strong>learning</strong> has greater impact<br />
on <strong>learning</strong> than does virtual <strong>learning</strong>. Furthermore, the report indentified 28 research papers<br />
comparing the courses outcomes of purely online courses with face-to-face instruction. The findings<br />
reveal that significant difference in favor of online <strong>learning</strong> was found in only 2 research papers. The<br />
remaining research papers (92.8%) found no significant difference in favor of online <strong>learning</strong>. This<br />
finding suggests that, statistically wise, online <strong>learning</strong> is not better than face-to-face instruction.<br />
In another meta-analysis reviewing online and distance <strong>learning</strong> studies from 1985 through 2002<br />
(Bernard et al., 2004), 688 independent effect sizes that could be subjected to meta-analysis were<br />
identified. The meta-analysis found that in terms of achievement outcomes, the mean effect size is<br />
near zero (standard deviation of 0.439). Bernard et al. point out that it is clear from the range of effect<br />
sizes (−1.31 to +1.41) that in almost half of the cases, classroom instruction is far better than distance<br />
education. Furthermore, a comparison of student attitude towards distance and classroom instruction<br />
reveals that there is a small negative but significant effect on overall attitude outcomes in favor of<br />
classroom instruction (g+ = −0.0812). The researchers also found a significant effect in favor of<br />
classroom instruction (g+ = −0.0573) on retention outcomes. The research findings suggest that<br />
distance <strong>learning</strong> is not better than face-to-face instruction in terms of achievement outcomes.<br />
Moreover, students appear to perceive traditional instruction more favorably than online instruction.<br />
296
Danny Glick and Roni Aviram<br />
In another meta-analysis comparing online and distance <strong>learning</strong> studies from 2000 through 2002<br />
(Ungerleider and Burns, 2003), 12 studies that have enough statistical information for inclusion into<br />
the meta-analysis were found. In this meta-analysis the researchers looked at the effectiveness of<br />
online versus traditional forms of instruction by focusing on two measures: achievement measures,<br />
and satisfaction ratings. The researchers found no difference between forms of instruction on grades,<br />
and a significantly negative effect (-0.509) of online instruction on student satisfaction (p < .0001).<br />
Ungerleider and Burns point out that students appear to perceive online instruction less favorably than<br />
traditional instruction, regardless of their test scores grades. In addition, the researchers also<br />
conducted in-depth narrative reviews of 25 studies that did not include enough statistical information<br />
for a meta-analysis. The 25 studies in the narrative reviews present a mixed picture of the<br />
effectiveness of online <strong>learning</strong>. While some studies reported significantly higher levels of<br />
achievements and student satisfaction with the traditional format, other studies showed a reversed<br />
difference. Ungerleider and Burns claim that the number of studies purporting to show an effect on<br />
one method over the other is small and there is nothing present in the studies reviewed to allow the<br />
researchers to make strong statements one way or the other.<br />
Table 1 presents a summary of the meta-analyses discussed in this chapter.<br />
Table 1: Summary of the meta-analyses<br />
Meta-Analysis Years What was Number of Effect Mean<br />
Measured Sizes Identified Effect Size<br />
US Department of Education<br />
(2009)<br />
1996-2008 Learning<br />
Outcomes<br />
51 +0.35 t<br />
+0.14 t<br />
Bernard et al. (2004) 1985-2002 Learning<br />
Outcomes<br />
318 -0.0100<br />
Bernard et al. (2004) 1985-2002 Attitude 262 -0.0812*<br />
Ungerleider and Burns<br />
(2003)<br />
2000-2002 Attitude 4 -0.509**<br />
Ungerleider and Burns 2000-2002 Learning<br />
12 0.000<br />
(2003)<br />
Outcomes<br />
* p < .001; **p < .0001; t Instruction combining online and face-to-face elements had a larger<br />
advantage relative to purely face-to-face instruction (+0.35) than did purely online instruction (+0.14).<br />
Thomas Russell’s No Significant Difference Website (www.nosignificantdifference.org)<br />
provides access to hundreds of research reports that document no significant difference as well as<br />
reports that do document significant difference in student outcomes between technology-enhanced<br />
teaching and face-to-face instruction. Analysis of the reports published on Russell’s Website found 42<br />
research papers comparing online <strong>learning</strong> with face-to-face instruction. Analysis of these research<br />
reports reveals that significant difference in favor of online <strong>learning</strong> was found in 13 research papers<br />
(30.9%). The remaining 29 research papers (69.1%) found no significant difference in favor of neither<br />
of the instruction modes.<br />
To sum up, the literature review suggests that in terms of achievement outcomes, online <strong>learning</strong> is<br />
no better than face-to-face instruction, and is some cases, inferior to classroom instruction. In terms of<br />
student satisfaction, a comparison of student satisfaction towards distance and classroom instruction<br />
reveals that there is a small negative but significant effect on overall attitude outcomes in favor of<br />
classroom instruction. Finally, the meta-analyses found a significant effect in favor of classroom<br />
instruction on retention outcomes.<br />
3. Research hypothesis<br />
The research hypothesis is that the main reason for the little impact online <strong>learning</strong> has on <strong>learning</strong><br />
outcomes and student satisfaction is lack of Mindful Learning Processes (MLP), a new term coined by<br />
the authors of this study. The authors define MLP as an online course that is based on four<br />
interconnected components (see Figure 1):<br />
1) Learning Theory: Learning theories play a key role in any <strong>learning</strong> process. Anderson (2008)<br />
claims that theory allows and even forces educators to see the big picture and makes it possible to<br />
view their practice and their research from a broader perspective than envisioned from the murky<br />
trenches of their practice. This broader perspective helps educators make connections with the work<br />
of others, facilitates coherent frameworks and deeper understanding of their actions, and perhaps<br />
297
Danny Glick and Roni Aviram<br />
most importantly, allows them to transfer the experience gained in one context to new experiences<br />
and contexts.<br />
2) Leaning Environment: A well-designed Web-based <strong>learning</strong> environment and a user interface that<br />
is based on a theory are an integral part of a Mindful Learning Process. Plass (1998) defines user<br />
interface as the part of an application in charge of communication with the learner. Plass argues that<br />
the user interface has a key function because it conveys the functionality of a computer application to<br />
the user, and translates the user's input into a machine-specific format.<br />
3) Learning Process: The <strong>learning</strong> process is a stage in which the learner realizes the course<br />
objectives through interaction with the course platform, content, and communication tools. Anderson<br />
(2003) argues that interaction is a critical component of the educational process, and should include<br />
three types of interaction: student-student; student-teacher; student-content.<br />
4) Assessment: Assessment is the forth component of a Mindful Learning Process. Using traditional<br />
and alterative assessment tools, educators should assess their students’ progress and provide<br />
personalized feedback. Birenbaum (2003) claims that the course assessment tools should reflect the<br />
course goals and objectives.<br />
An online course that wishes to fulfill the MLP requirements must, first and foremost,<br />
be based on a <strong>learning</strong> theory. Bigge and Shermis (1999) claim that action, whether a part of<br />
teaching or any other activity in life, either is linked to a theory or is blind and purposeless. A teacher<br />
who does not make use of a systematic body of theory in his day-by-day decisions is behaving<br />
blindly. Consequently, any purposeful action is governed by theory. Furthermore, the <strong>learning</strong><br />
environment, <strong>learning</strong> process and course assessment criteria should reflect the <strong>learning</strong> theory the<br />
course is based on and harmonize with each other.<br />
Figure 1: The four components of a Mindful Learning Process<br />
The goal of this paper is to check whether the research hypothesis outlined in this chapter may be<br />
accepted. To achieve that, the authors have reviewed case studies describing online courses<br />
published in peer-reviewed journals in the area of educational technology and e-<strong>learning</strong>. They<br />
authors have identified three case studies that fulfill the MLP requirements stated above, and<br />
analyzed them in terms of <strong>learning</strong> theory, <strong>learning</strong> processes, <strong>learning</strong> environment and assessment<br />
tools. To measure the impact of the MLP on course outcomes, the authors analyzed these case<br />
studies in terms of test scores and students’ perception of the e-<strong>learning</strong> experience. The next<br />
chapter provides a description of the three case studies together with the impact of the MLP-based<br />
<strong>learning</strong> environment on course outcomes.<br />
298
Danny Glick and Roni Aviram<br />
4. The impact of MLP on course outcomes – case study analysis<br />
4.1 A constructivist approach to online college <strong>learning</strong> (Rovai, 2004)<br />
This paper evaluates the effectiveness of an online graduate course in education research design<br />
(N=82). This online course, which fulfills the four MLP requirements outlined in chapter 3, was<br />
designed and delivered based on constructivist epistemology. The author of this paper fulfills the first<br />
MLP requirement (Learning Theory) by linking the course to a <strong>learning</strong> theory. The author provides a<br />
detailed description of a constructivist approach to online <strong>learning</strong> and outlines the differences<br />
between traditional and constructivist <strong>learning</strong> environments. With regard to the second MLP<br />
requirement (Learning Environment), the paper fulfills this requirement by designing a <strong>learning</strong><br />
environment that reflects the constructivist <strong>learning</strong> theory the course is based on. The course<br />
<strong>learning</strong> environment includes a wide range of constructivist-based tools such as tools for<br />
collaborative group work, Instructor-Student and Student-Student tools for interactions, tools for<br />
guided discussions, discussion boards for posting the assessment tasks, etc. The course also fulfills<br />
the third MLP requirement (Learning Process) by describing the course constructivist-based <strong>learning</strong><br />
processes which require learners to participate in a problem-oriented discussion forum, and to solve<br />
authentic tasks requiring the evaluation of research-based case study. Finally, the course fulfills the<br />
fourth MLP requirement (Assessment) by assessing students’ progress in the course using<br />
constructivist-based assessment tools such as authentic assessment tasks, analysis of researchbased<br />
case studies, use of rubrics to assess forum discussions and collaborative work, etc.<br />
The research findings reveal that the course outcomes were positive in terms of course grades and<br />
student level of satisfaction. With regard to course grades, course letter grades were distributed as<br />
follows: F, 3 students; C, 3 students; C+, 0 students; B−, 7 students; B, 23 students; B+, 15 students;<br />
A−18 students; and A, 3 students. In other words, 69 students (81.9%) have earned an A or a B<br />
grade. To measure perceived <strong>learning</strong> effectiveness, students were asked to respond to the following<br />
item via an online survey: On a scale of 0 to 9, how much did you learn in this course, with 0 meaning<br />
you learned nothing and 9 meaning you learned more than in any other course you’ve had? The<br />
mean and standard deviation for perceived <strong>learning</strong> among this group of participants were 7.17 (on a<br />
9-point scale) and 1.23, respectively. When comparing this course with other courses (both traditional<br />
and Web-based), students report that they learn more than in most other courses that they have<br />
learned.<br />
4.2 Community building, emergent design and expecting the unexpected: Creating a<br />
quality eLearning experience (Thompson & MacDonald, 2005)<br />
This paper evaluates the effectiveness and factors that influence the quality of an online Synthesis<br />
Seminar, the last of ten courses required to obtain a M.Ed. at the University of Ottawa. This online<br />
course, which was based on a constructivist <strong>learning</strong> theory, fulfills the four MLP requirements. In this<br />
Synthesis Seminar, learners (N=19) reflect on their professional development throughout the M.Ed.<br />
program and demonstrate that they can apply new <strong>learning</strong>s by writing a 25-page research paper.<br />
The knowledge built throughout this <strong>learning</strong> experience was directly related to the learners’<br />
professional and personal interests and constructed as they wrote their paper and shared insights in<br />
the discussion groups. The only content provided were brief online documents designed to support<br />
learners in this constructivist <strong>learning</strong> environment. As mandated by the University, this pass/fail<br />
course was based on assessment of the research paper. To help learners produce the best possible<br />
paper, a series of five cumulative assignments provided opportunities for formative assessment and<br />
enabled the learners to build their papers in stages. Several strategies were employed to encourage<br />
online participation: (a) online participation was included in the list of course requirements to<br />
emphasize its importance; (b) online forums were intended to be useful to the learners and designed<br />
to have the potential to engage a diverse group of people in a meaningful dialogue that would<br />
enhance their research papers; and (c) the focus of each discussion group was highlighted in the<br />
course outline and the forum moderator posted frequent reminders and encouragement.<br />
Constructivists acknowledge that <strong>learning</strong> is active, situated, and social. Consequently several<br />
strategies were implemented to facilitate community in the course. First, learners were grouped into<br />
triads; triad members were expected to give each other feedback on course assignments. Second,<br />
dialogue in the discussion groups was emphasized. Eight one-week discussion groups were<br />
strategically placed throughout the course. Learners were asked to share: aspects of their paper,<br />
299
Danny Glick and Roni Aviram<br />
opinions and insights, their journey as e-learners, and how their research related to challenges in their<br />
professional work.<br />
The research findings reveal that the course outcomes were positive in terms of perception of the e<strong>learning</strong><br />
experience. Students were asked to respond to the following questions: On a scale of 1 to 5,<br />
(a) as a result of this course I have acquired personal and professional growth and (b), the course met<br />
my <strong>learning</strong> objectives, with 1 being Strongly Disagree and 5 being Strongly Agree? The mean for<br />
questions (a) and (b) was 4.3 and 4 (on a 5-point scale), respectively. With regard to achievement<br />
outcomes, all 19 students passed the course (100%).<br />
4.3 Instructional design strategies for intensive online courses: an objectivistconstructivist<br />
blended approach (Chen, 2007)<br />
This paper evaluates the effectiveness of a three-credit online course for graduate education majors<br />
(N=12) regarding the integration of technology in the K-12 curriculum. This online course, which fulfills<br />
the four MLP requirements outlined in chapter 3, was designed and delivered based on a mix of<br />
objectivist and constructivist theories and design strategies. With regard to the first MLP requirement<br />
(Learning Theory), the author provides a detailed description of objectivist and constructivist<br />
approaches to online <strong>learning</strong> and outlines the differences between these two <strong>learning</strong> theories. With<br />
regard to the second, third and forth MLP requirements (Course Design, Learning Process and<br />
Assessment), this course assigned the students a problem-based capstone <strong>learning</strong> task with a real<br />
world context. The <strong>learning</strong> task was designed to provide opportunities for collaboration and social<br />
interaction during <strong>learning</strong> and reflection after <strong>learning</strong>. To scaffold the complexity of the capstone<br />
task and define its structure, objectivist approach was used to break the capstone task into several<br />
subtasks and provide a set of clear and detailed guidance for each subtask to direct group<br />
interactions for completing the problem/task. The culminating task required students to develop a<br />
course plan integrating technology into teaching and <strong>learning</strong>. The culminating task was broken down<br />
into subtasks such as creating a website evaluation instrument, evaluating web resources for<br />
instruction, and developing a web-enhanced lesson plan. Each subtask was designed as a unit. A<br />
project was assigned for each unit. To support collaboration and interaction among members of the<br />
community, a number of strategies were adopted for the design of a social support system such as<br />
forming heterogeneous groups, enhancing effective peer review, opening forums for social<br />
interaction, etc.<br />
The research findings reveal that the course outcomes were positive in terms of students’ opinion<br />
about the online course. Students were asked to respond to the following questions: On a scale of 1<br />
to 4, (a) I find the course <strong>learning</strong> tasks, activities and instruction effective; (b) the course met my<br />
<strong>learning</strong> objectives in terms of personal and professional development, with 1 being Strongly Disagree<br />
and 4 being Strongly Agree. The mean for questions (a) and (b) was 3.68 (SD=.26) and 3.60<br />
(SD=.13), respectively. With regard to the course grades, the average score on the final project was<br />
93.75.<br />
Table 2 below presents a summary of how the above case studies fulfill the MLP requirements. Table<br />
2 also presents the outcomes of each of the three courses in terms of test scores and student level of<br />
satisfaction.<br />
Table 2: Case study summary in accordance with the MLP requirements<br />
Learning<br />
Theory<br />
Learning<br />
Environment<br />
Case Study 1<br />
(Rovai, 2004)<br />
Constructivism<br />
-Tools for collaborative<br />
group work<br />
- Tools for guided<br />
discussions<br />
- Discussion boards<br />
- Tools for Instructor-<br />
Students and Student-<br />
Student interaction<br />
300<br />
Case Study 2<br />
(Thompson &<br />
MacDonald, 2005)<br />
Constructivism<br />
-Tools for guided<br />
discussions<br />
-Discussion boards<br />
- Tools for peer feedback<br />
Case Study 3<br />
(Chen, 2007)<br />
Constructivism &<br />
Objectivism<br />
-Tools for collaboration<br />
-Tools for social<br />
interaction<br />
- Forums<br />
- Tools for peer review
Learning<br />
Process<br />
Assessment<br />
Test<br />
Scores<br />
Student Level of<br />
Satisfaction<br />
5. Discussion<br />
Danny Glick and Roni Aviram<br />
Case Study 1<br />
(Rovai, 2004)<br />
-Problem-oriented<br />
discussion forum<br />
- Research-based case<br />
studies<br />
-Authentic assessment<br />
tasks<br />
- Analysis of researchbased<br />
case studies<br />
- Rubrics<br />
81.9% of students have<br />
earned an A or a B grade<br />
Case Study 2<br />
(Thompson &<br />
MacDonald, 2005)<br />
-Research paper<br />
-Knowledge application<br />
- Group discussions<br />
- Peer feedback<br />
-Research paper built in<br />
stages<br />
All students passed the<br />
course.<br />
Case Study 3<br />
(Chen, 2007)<br />
-Problem-based tasks<br />
- Authentic tasks<br />
- Reflection<br />
- Breaking task into<br />
subtasks<br />
- Peer review<br />
- Project Work<br />
- Developing a course<br />
plan<br />
- Evaluating Web<br />
resources<br />
- Developing lesson plans<br />
93.75<br />
7.17 on a 9-point scale 4.3 on a 5-point scale 3.6 on a 4-point scale<br />
Online education is growing rapidly within all levels of education (Allen & Seaman, 2010; Watson,<br />
2007) and is expected to grow during the coming years (Redden, 2009; Rovai & Downey, 2010). In<br />
addition, Chief <strong>Academic</strong> Officers have made online <strong>learning</strong> part of their long-term strategy (Allen &<br />
Seaman, 2007; Rovai & Downey, 2010). Research, however, indicates that online <strong>learning</strong> has little<br />
impact on course outcomes. The authors of this paper hypothesize that this paradox is a result of lack<br />
of Mindful Learning Process in online education. The hypothesis of this paper is that online courses<br />
that fulfill the MLP requirements as defined in chapter 3 are likely to have positive outcomes.<br />
Analysis of three case studies (Rovai, 2004; Thompson & MacDonald, 2005; Chen, 2007) suggests<br />
that online courses that fulfill the MLP requirements may lead to positive outcomes in terms of test<br />
scores and student level of satisfaction. These case studies fulfill the first MLP requirement – Learning<br />
Theory – by linking the course to a <strong>learning</strong> theory and providing a detailed description of the theory<br />
the course is based on. They also fulfill the second and third MLP requirements – Learning<br />
Environment and Learning Process – by using technologies and applying e-pedagogies which reflect<br />
the course <strong>learning</strong> theory. And finally, the authors of these papers use a variety of alternative<br />
assessment methods to reflect the course <strong>learning</strong> theory.<br />
The research findings suggest that a Mindful Learning Process may lead to a meaningful and<br />
pedagogically sound course where theory, design, <strong>learning</strong> and assessment harmonize with each<br />
other. These findings support earlier work on <strong>learning</strong> theories and online <strong>learning</strong> (Ally, 2008; Clark,<br />
2001; Rovai, 2002) that found that the reason for the <strong>learning</strong> benefits of computer media is not the<br />
medium of instruction, but the instructional strategies built into the <strong>learning</strong> materials, and that the<br />
development of effective online <strong>learning</strong> materials should be based on proven and sound <strong>learning</strong><br />
theories.<br />
References<br />
Allen, E. I. and Seaman, J. (2007). Making the Grade: Online Education in the United States, 2006, The Sloan<br />
Consortium, Needham.<br />
Allen, E. I. and Seaman, J. (2010). Class Differences: Online Education in the United States, 2010, The Sloan<br />
Consortium, Needham.<br />
Ally, M. (2008). “Foundations of Educational Theory for Online Learning”. In T. Anderson (Ed.), Theory and<br />
Practice of Online Learning, 2 nd ed. (pp 15-44), AU Press, Edmonton.<br />
Anderson, T. (2003). “Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction”,<br />
International Review of Research in Open and Distance Learning, Vol 4, No. 2.<br />
Anderson, T. (2008). “Towards a Theory of Online Learning”. In T. Anderson (Ed.), The Theory and Practice of<br />
Online Learning (pp 45-74), AU Press, Edmonton.<br />
301
Danny Glick and Roni Aviram<br />
Bernard, R. M., Abrami, P. C., Lou, Y, Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A., Fiset, M. and Huang,<br />
B. (2004). “How Does Distance Education Compare with Classroom Instruction? A Meta-Analysis of the<br />
Empirical Literature”, Review of Educational Research, Vol 74, No. 3, p. 379-439.<br />
Bigge, M. L. and Shermis, S. S. (1999). Learning Theories for Teachers (6th Ed.), Longman, New York.<br />
Birenbaum, M. (2003). “New Insights into Learning and Teaching and their Implications for Assessment”. In<br />
M. Segers, F. Dochy and E. Cascallar (Eds.) Optimizing New Methods of Assessment: In Search of<br />
Qualities and Standards. (pp 13-36), Kluwer, Boston.<br />
Chen, S. J. (2007). “Instructional Design Strategies for Intensive Online Courses: An Objectivist-Constructivist<br />
Blended Approach”, Journal of Interactive Online Learning, Vol 6, No. 1, pp 72-86.<br />
Clark, R. E. (2001). “A Summary of Disagreements with the ‘Mere Vehicles’ Argument. In R. E. Clark (Ed.),<br />
Learning from Media: Arguments, Analysis, and Evidence (pp 125–136), Information Age Publishing,<br />
Greenwich.<br />
Hutti, D. (2007). “Online Learning, Quality, and Illinois Community Colleges”, MERLOT Journal of Online<br />
Learning and Teaching, Vol 3, No. 1, pp 18-29.<br />
iNACOL. (2010). “Fast Facts about Online Learning”, [online], International Association for K-12 Online Learning,<br />
http://www.inacol.org/press/nacol_fast_facts.pdf.<br />
Plass, J. L. (1998). “Design and Evaluation of the User Interface of Foreign Language Multimedia Software: A<br />
Cognitive Approach”, Language Learning & Technology, Vol 2, No. 1, pp 40-53.<br />
Redden, E. (2009). “In Global Recession, Global Ed Still Growing”, [online], Inside Higher Ed.,<br />
http://www.insidehighered.com/news/2009/05/29/international. .<br />
Rovai, A. P. (2002). “Building Sense of Community at a Distance”, International Review of Research in Open and<br />
Distance Learning, Vol 3, No. 1.<br />
Rovai, A. P. (2004). “A Constructivist Approach to Online College Learning”, Internet and Higher Education, Vol<br />
7, No. 2, pp 79-93.<br />
Rovai, A. P. and Downey, J. (2010). “Why Some Distance Education Programs Fail while Others Succeed in a<br />
Global Environment”, The Internet and Higher Education, Vol 13, No. 3, pp 141-147.<br />
U.S. Department of Education. (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-<br />
Analysis and Review of Online Learning Studies, Washington, D.C.<br />
Shachar, M. (2008). “Meta-Analysis: The Preferred Method of Choice for the Assessment of Distance Learning<br />
Quality Factors”, The International Review of Research in Open and Distance Learning, Vol 9, No. 3.<br />
Thompson, T. L. and MacDonald, C. J. (2005). “Community Building, Emergent Design and Expecting the<br />
Unexpected: Creating a Quality eLearning Experience”, Internet and Higher Education, Vol 8, No. 3, pp<br />
233-249.<br />
Ungerleider, C. and Burns, T. (2003). “A Systematic Review of the Effectiveness and Efficiency of Networked ICT<br />
in Education”, A state of the field report to the Council of Ministries of Education, Canada and Industry<br />
Canada, October.<br />
Watson, J. F. (2007). “A National Primer on K-12 Online Learning”, [online], NACOL,<br />
http://www.nacol.org/docs/national_report.pdf.<br />
302
Researching and Sharing – Business School Students<br />
Creating a Wiki Glossary<br />
Andrea Gorra and Ollie Jones<br />
Faculty of Business and Law, Leeds Metropolitan University, UK<br />
gorra@leedsmet.ac.uk<br />
o.jones@leedsmet.ac.uk<br />
Abstract: Often students find that the terminology involved in a new subject can be a barrier to <strong>learning</strong>, Some<br />
theorists term this 'cognitive load' - a high amount of information that the brain has to process before it can begin<br />
to construct new knowledge. Glossaries have been used in many different educational contexts; however<br />
traditional glossaries are passive and have less capability to promote student engagement. This paper outlines a<br />
small scale study in a UK business school that used a small wiki situated within the Virtual Learning Environment<br />
to encourage students to construct their own glossary at the start of a module. Whilst many studies have looked<br />
at use of wikis in student work, especially collaborative projects, relatively few have investigated the use of wikis<br />
for constructing simple glossary entries created by students. In our study each student was allocated a particular<br />
subject related term. The students were instructed to construct a wiki entry describing what the term meant, citing<br />
at least two appropriate references. The study was evaluated by a variety methods including quantitative analysis<br />
of the Virtual Learning Environment usage and access statistics, alongside qualitative and quantitative survey<br />
data. Generally the quality of the entries was very high and students indicated that writing the wiki glossary<br />
entries helped them to understand the terminology of the new subject. Students stated that they had read a<br />
number of entries over the course of the module and some chose to reference the wiki in their final written<br />
assignment. Our research indicates that students found creating the wiki more useful than the finished resource<br />
itself.<br />
Keywords: wiki, glossary, active <strong>learning</strong>, VLE<br />
1. Background<br />
Often students find that the terminology involved in a new subject can be a barrier to <strong>learning</strong> about<br />
concepts (see for example Wandersee, 1988; Lidbury and Zhang, 2008). Theorists such as Sweller<br />
(1994) determine this as ‘cognitive load’ – the higher the amount of information the brain the must<br />
process before it can begin to construct knowledge. Glossaries have been used in many different<br />
educational contexts (Hall, 2002) to enable more effective dialogue between tutor and student.<br />
Laurillard (2002) believes that the use of glossaries can help with the students’ <strong>learning</strong> process.<br />
However traditional glossaries are passive and have less capability to promote student engagement,<br />
especially at the beginning of studying a subject.<br />
This paper outlines a small scale study in a Business School in the North of England that used wikis<br />
to encourage students to construct their own glossary at the start of a module. A wiki is a website that<br />
can be modified and contributed to by internet users (Encyclopædia Britannica, 2011). Wikis and<br />
other social networking sites are part of a host of applications that are also known as Web 2.0<br />
technologies (O'Reilly, 2005).<br />
The rationale for introducing a glossary task was to foster enquiry based <strong>learning</strong> (Khan and<br />
O’Rourke, 2005) in order to engage students early on in the 12 week Operations management<br />
module and to help the students to familiarise themselves with the terminology of this new subject.<br />
Whilst a lot of studies have looked at use of wikis in student work , especially collaborative projects,<br />
relatively few have looked at using wikis for constructing simple glossary entries. In those studies that<br />
do, such as Cubric (2010), the glossary element forms a small part of the overall research.<br />
The students taking part in this study were in their second year of study (Level 5), from two separate<br />
undergraduate business courses, namely Business Studies, and Business and Management studies.<br />
The module taken was ‘Operations Management’ (OM) which was taught in the first semester at<br />
Leeds Metropolitan University. The OM module was delivered by 5 tutors over 12 classes and was<br />
delivered by using lectures and tutorials.<br />
The assessment was a multi-media case study accessible the Virtual Learning Environment (VLE)<br />
and via an additional DVD. The case study aimed at developing the students’ employability skills and<br />
303
Andrea Gorra and Ollie Jones<br />
to help them to handle and make sense of complexity (see Jones and Kerr, 2008, for more detail on<br />
the multi-media case study).<br />
Our key research questions were as follows:<br />
Do students find terms, language in unfamiliar subject a barrier to <strong>learning</strong>?<br />
Can the process of constructing a glossary help <strong>learning</strong>?<br />
Is the outcome of a student generated glossary a useful/used resource for students?<br />
2. Study details and data collection methods<br />
All students taking the Operations Management module were allocated a particular term or concept<br />
from the subject area of Operations Management. The students were then given two weeks to<br />
research this term and construct a wiki entry describing what the term meant. They had to reference<br />
at least two appropriate sources, which could include websites, books etc.<br />
The wiki entry was a module requirement and completion was necessary in order to gain access to<br />
the assignment multi-media case study early on in the semester.<br />
Operations Management concept allocated to the students were terms such as the following: “Agile<br />
Supply Chains”, “Buffer Inventory”, “Kanban” and “Lean”. See Figure 1, below, for an example of a<br />
wiki entry written by one of the second year students.<br />
Sample wiki entry<br />
Lean production has three main components; just-in-time, total quality management and time-based<br />
management (Marcousé, 2003). First adopted by Toyota motor corporation lean production aims to minimise<br />
waste, reduces costs, increases productivity and expand margins in order to make business more efficient in its<br />
use of resources (Biz/ed n.d.)Bibliography: Marcousé et al (2003) Business Studies. 2nd ed. Hodder<br />
Arnold.Biz/ed (n.d.) Glossary description. [internet], Available from: < http://www.bized.co.uk/cgibin/glossarydb/browse.pl?glostopic=1&glosid=1346><br />
[Accessed 8th October 2010].<br />
Figure 1: An example of a wiki glossary term created by a student<br />
After the students had submitted their term descriptions in form of a wiki glossary entry via the VLE,<br />
the module tutors marked the entries as either pass or fail. Students who ‘failed’ were permitted to<br />
submit a revised entry.<br />
Following this, the wiki was available for students to use throughout the rest of the module and the<br />
approach was embedded into the overall enquiry based <strong>learning</strong> strategy. The students were able to<br />
use the wiki to aid them later in the module and were encouraged to utilise the definitions in their<br />
assignment. In addition, some of the students’ wiki glossary definitions were incorporated into the<br />
lectures.<br />
See Figure 2 for an overview of the steps involved in the creation of the wiki glossary for the<br />
Operations Management module.<br />
The wiki task attracted a good 95 percent participation rate from the 260 student strong cohort.<br />
However, this is not surprising as the creation of the allocated two terms for the wiki glossary was<br />
needed for the students to gain access to the multi-media assignment material early on in the<br />
semester.<br />
Our small scale study was evaluated using a variety methods including quantitative analysis of the<br />
VLE usage and access statistics, alongside qualitative and quantitative survey data.<br />
3. Results from the survey<br />
Empirical data was collected by using an online survey and the link to the survey was distributed to<br />
the students via their student email address. 56 out of the 260 students responded to the survey<br />
which equates to a 22% participation rate. In addition, we used statistics from the VLE to ascertain<br />
how many students accessed the finished wiki glossary.<br />
304
Tutor finds and allocates<br />
Glossary Terms<br />
Andrea Gorra and Ollie Jones<br />
Wiki -Glossary Design Story Board<br />
Students gain Access<br />
Student researches<br />
Terms from two sources<br />
Students read each others entries<br />
Students can access<br />
Assessment Resources Faculty at of later Business date & Law<br />
Figure 2: Process of creating the wiki glossary<br />
3.1 New subject terminology and preferred <strong>learning</strong> materials<br />
Upload to the VLE, include<br />
references<br />
Tutor(s) validates all entries<br />
Tutor uses selection<br />
Of terms<br />
In some lectures<br />
The survey confirmed findings from the literature in that the terminology of a new subject can be<br />
perceived by students as an initial barrier, with 70% of the 56 students either agreeing or strongly<br />
agreeing with this statement (see Figure 3, below).<br />
Figure 3: Students perceptions of new subject terminology<br />
305
Andrea Gorra and Ollie Jones<br />
However, it appears that our students would not typically access a glossary when approaching a new<br />
subject. Survey findings (see Figure 4, below) indicate that the majority of our second year students<br />
would only “sometimes” use a traditional glossary when starting a new subject.<br />
Figure 4: Students' use of 'traditional' glossaries<br />
3.2 The wiki glossary as a constructive <strong>learning</strong> experience and as a resource<br />
The majority of our students perceived that constructing the glossary had helped them to understand<br />
the subject area of operations management (see Figure 5,). In addition, 52% of our 260 students<br />
agreed with the statement “I found the wiki glossary task helpful in getting me engaged and interested<br />
in the operations management subject”, while 29% disagreed and a fifth neither agreed, nor<br />
disagreed.<br />
Figure 5: Wiki glossary aiding understanding<br />
When asked whether the wiki glossary created by fellow students helped them with the written<br />
assignment, the students’ responses were rather ambivalent. About a third of the students agreed<br />
306
Andrea Gorra and Ollie Jones<br />
with this statement, while as much as 40% disagreed and about a quarter neither agreed or disagreed<br />
(see Figure 6, below).<br />
Figure 6: Use of the wiki glossary for the OM assignment<br />
The majority of students perceived the completed wiki glossary was a useful <strong>learning</strong> resource (see<br />
Figure 7), while an even greater majority believed that ‘constructing’ the glossary has helped their<br />
<strong>learning</strong> process in this new subject area (Figure 8).<br />
Figure 7: Perceptions of the wiki glossary as a resource<br />
307
Andrea Gorra and Ollie Jones<br />
Figure 8: Has constructing the wiki glossary helped <strong>learning</strong>?<br />
Finally we asked the students’ opinion about their wiki glossary entries being used in the Operations<br />
Management lectures. While the majority of students (59%) agreed with statement “I liked the tutor’s<br />
use of student created definitions in subsequent lectures”, a small minority (18%) were less happy<br />
with this (see Figure 9, below).<br />
Figure 9: Students’ views on wiki glossary use in lectures<br />
Some qualitative comments about the wiki use in lectures helped us to understand the students’<br />
views further. Some positive comments were expressed as follows:<br />
“Showed that the input from students mattered.”<br />
“It made them easier to understand”<br />
308
Andrea Gorra and Ollie Jones<br />
“We could see that are efforts weren't going in vain. and it's always good to see your<br />
name somewhere”<br />
While some less positive ones indicated that some students would have preferred to be only<br />
presented with materials created by tutors (“I would prefer the tutors more”), while others felt the<br />
inclusion of the wiki comments did not fit into the flow of the Operations Management lecture:<br />
“because it was never clear and on point to the questions”<br />
Students are not a homogenous group and some found the use of their own wiki glossary entries<br />
embarrassing while others thought it built their confidence as the following comments show:<br />
“it was embarrassing for those who had written them and a lot of the time it broke the<br />
flow of the lecture”<br />
“It helped to build students confidence”.<br />
4. Conclusions and implications for future research<br />
The module tutors read and marked every wiki entry and the overall tutor impression was that the<br />
quality of the entries was generally high.<br />
By evaluating the empirical data, we identified some answers to our initial three research questions:<br />
1. Do students find terms, language in unfamiliar subject a barrier to <strong>learning</strong>?<br />
Our survey confirmed our initial suspicion and findings from literature review (e.g. Lidbury and Zhang,<br />
2008) in that students do perceive the language and definitions of a new subject area as a barrier to<br />
both understanding and <strong>learning</strong>.<br />
2. Can the process of constructing a glossary help <strong>learning</strong>?<br />
The results from the survey show that students found the process of construction useful as well as the<br />
resource itself. Qualitative comments acknowledged that constructing a wiki glossary engaged the<br />
students. In addition, the students perceived it as positive to be able to read other students’ definition<br />
of a key term as opposed to only having the tutor’s or text book definitions available as for example<br />
the following comment illustrates: “Because it engaged students and gave us another students<br />
perspective on the definitions“.<br />
As discussed earlier, the process of constructing a glossary by the students can be seen as an<br />
enquiry-based <strong>learning</strong> approach which expected students to do their own research on given glossary<br />
terms. Through this process of constructing their own understanding of these terms, we believe,<br />
similar to Laurillard (2002), that this process can help students’ <strong>learning</strong>.<br />
The social component of <strong>learning</strong> was less pronounced in our chosen approach, as enquiry-based<br />
<strong>learning</strong> is often carried out in small groups (see Khan and O’Rourke, 2005). However, the fact that<br />
each wiki glossary term had several visits from other students may be an indication of an added<br />
(small) social dimension to <strong>learning</strong> when using glossaries. The aim to increase student motivation<br />
through enquiry-based <strong>learning</strong> (Khan and O’Rourke, 2005) appears also to have been fulfilled as the<br />
student quote above indicates.<br />
3. Is the outcome of a student generated glossary a useful/used resource for students?<br />
Our students perceived the final glossary to be a useful resource, as the survey findings show. For<br />
example Figure 7 indicates that the majority of students agree with the statement “I found the wiki<br />
glossary a useful <strong>learning</strong> resource for the module”. However, a significant number of students did not<br />
feel that the wiki glossary entries helped them with the assessment (for detail see Figure 6).<br />
Overall it can be concluded, that the actual creation of the wiki resource, i.e. the researching and<br />
writing about a subject-specific term, was perceived as more useful as the final resource in form of a<br />
wiki itself. This ties in with some of the students commenting that they appreciated that “[..] that all<br />
definitions were correct and proof checked [by the tutors]”.<br />
309
Andrea Gorra and Ollie Jones<br />
Students appeared to read a number of entries over the course of the module, with our VLE statistics<br />
showing that on average every wiki term had seven visits from other students.<br />
There was mixed feedback regarding the use of wiki glossary entries in the lectures. Some students<br />
commented rather negatively using comments such as “it was embarrassing for those who had written<br />
them and a lot of the time it broke the flow of the lecture” or “I would prefer the tutors more”. While<br />
other students commented “It's good to see work we've done being appreciated” and “It is an<br />
acknowledgement that Student's contributions are valued”.<br />
Overall, we can conclude that the creation of wiki glossary terms had been a successful addition to<br />
the module. Students appeared to engage with the specialist terms of the subject area from an early<br />
stage of the 12 week module and appreciated having access to definitions created by their fellow<br />
students. The final glossary resource based on the collaborative writing platform of a wiki can be seen<br />
as the outcome of ‘collaborative group work’, which also has familiarised the students with this Web<br />
2.0 technology (Shailey, 2009).<br />
However, we feel that this approach will only work if there are sufficient incentives for students to<br />
create these short wiki glossaries. For example, we used the provision of early access to the multimedia<br />
assignment case study as an incentive.<br />
We are planning to use this approach in future deliveries of this Operations Management module as<br />
well as other modules. Comments about the study to the authors are very welcome.<br />
References<br />
Cubric, M. (2010) Wiki-based Process Framework for Blended Learning. Proceedings of the 2007 international<br />
symposium on Wikis, p.11-24, October 21-25, 2007, Montreal, Quebec, Canada.<br />
Encyclopædia Britannica (2011) [Internet]. Available from: [Accessed 27 June<br />
2011].<br />
Hall, R. (2002) Aligning <strong>learning</strong>, teaching and assessment using the web: an evaluation of pedagogic<br />
approaches, British Journal of Educational Technology, 33.2, pp 149-158<br />
Kerr, M., Jones, O. (2008) ‘e-invigorating operations management case assessments using a multi-media<br />
approach ’, Proceedings of the 15th Annual EurOMAConference,Tradition and Innovation in Operations<br />
Management, University of Groningen, The Netherlands, June 23 rd 2008.<br />
Khan, P., & O’Rourke, K. (2005). Understanding enquiry-based <strong>learning</strong>. in Barrett, T., MacLabhrainn, I., Fallon,<br />
H. (eds), Handbook of enquiry and problem based <strong>learning</strong>. Galway: CELT.<br />
Lidbury, B. A. & Zhang, F. (2008). Comprehension of Scientific Language as a Strategy to Enhance Learning and<br />
Engagement for Molecular Biology Students. Australian Biochemist: The Magazine of the Australian Society<br />
for Biochemistry and Molecular Biology, 39, 10-13.<br />
O'Reilly, T. (2005) What Is Web 2.0. Design Patterns and Business Models for the Next Generation of Software<br />
[Internet]. Available from: [Accessed 27 June 2011].<br />
Shailey, M. (2009) "Role of social software tools in education: a literature review", Education + Training, Vol. 51<br />
Iss: 5/6, pp.353 - 369.<br />
Sweller, J. (1994). Cognitive load theory, <strong>learning</strong> difficulty, and instructional design. Learning and Instruction, 4,<br />
295-312.<br />
Wandersee, J. H. (1988). Ways students read texts. Journal of Research in Science Teaching, 25, 69-84.<br />
310
A Qualitative Evaluation of <strong>Academic</strong> Staff’s Perceptions of<br />
Second Life as a Teaching Tool<br />
Rose Heaney and Megan Anne Arroll<br />
University of East London, School of Psychology, London, UK<br />
r.heaney@uel.ac.uk<br />
m.a.arroll@uel.ac.uk<br />
Abstract: The aim of the study described in this paper was to investigate the potential of Second Life® (SL) as a<br />
teaching tool on an undergraduate Psychology programme at University of East London (UEL). A qualitative<br />
methodology (Interpretative Phenomenological Analysis; Smith, 1996) was chosen as we wanted to allow the<br />
participants to express openly their first hand experience of using SL in their teaching at the same time as<br />
bringing the researcher’s interpretation to bear. We interviewed four lecturers, with varied prior experience of<br />
online teaching, before and after they conducted a tutorial-type session within SL (although one participant was<br />
interviewed once only - after his session). From the interview transcriptions, we identified three superordinate<br />
themes, namely ‘comfort’, ‘investment’ and ‘clear rationale’. Participants stated that the virtual and anonymous<br />
nature of SL appeared to allow students who might have had difficulty voicing their questions/concerns in a<br />
traditional, physical environment to engage actively in the session. It was also seen as a useful adjunct to the<br />
physical campus for students studying at a distance for reasons of geography, illness, bad weather and the like.<br />
However, these positives had to be offset against drawbacks such as the lack of non-verbal cues so important to<br />
staff in the physical classroom and the potential for staff embarrassment in front of students in what is a<br />
challenging and unpredictable environment. (Unlike students, staff could not hide behind their avatars.) Given the<br />
unique characteristics of SL and the not insignificant investment of both time and resources by both staff and<br />
students to take full advantage of the environment, they all emphasised the need for a clear rationale for its use.<br />
All felt it had obvious potential for activities such as role play exercises, problem based <strong>learning</strong> scenarios and<br />
student presentations but were less certain about its suitability for tutorials (the main activity experienced during<br />
the study) and other forms of group teaching. In sum, this study illustrated that with adequate training of both staff<br />
and students, clear strategies for use and a supportive and encouraging institutional environment, SL can be a<br />
beneficial addition to the teaching and <strong>learning</strong> repertoire of higher education.<br />
Keywords: second life; staff perceptions; interpretative phenomenological analysis<br />
1. Background<br />
Second Life® (SL) is a web-based, 3D virtual world, where users interact with each other via their<br />
avatars (3D representations of themselves). As one of the largest and best known virtual words with<br />
millions of users and tens of millions of square metres of virtual land, SL has been put to many uses,<br />
not least educational. As a Multi User Virtual Environment (MUVE) rather than a Massively Multiplayer<br />
Online Role-Playing Game (MMPORG) such as World of Warcraft (WoW), SL does not come with<br />
pre-defined structures and goal oriented game-play though activities of this sort may be created.<br />
Users or residents of SL create content - 98% of all content in SL is user-generated (Boellstorff, 2008)<br />
- or use resources developed by others, make choices about how and with whom to interact and<br />
whether to restrict themselves to their own (or their institution’s) private spaces or visit the many<br />
public arenas. Many universities in the UK and elsewhere have been active in SL for several years<br />
and continue to be, though recent increases in land charges and its continued lack of adoption as a<br />
mainstream <strong>learning</strong> platform cast some doubts on its use in the longer term. However, the unique<br />
ability of SL (and Virtual Worlds in general) to give users a sense of being present in an environment<br />
other than the one they are actually in, and to interact within it (Schroeder, 2008), make for a rich<br />
educational experience that can be expected to prevail in one form or the other and, as such, a fruitful<br />
area for research and exploration. Additionally, SL’s text and voice communication facilities, in<br />
combination with its 3D features, make it a viable and potentially more rewarding setting for online<br />
meetings, tutorials etc., than those in more common use.<br />
2. Teaching and <strong>learning</strong> in SL<br />
The most documented educational uses of SL are those that benefit from its immersive and<br />
visualising qualities such as: displays and exhibits; role play; simulations; historical recreations;<br />
language <strong>learning</strong>; problem based <strong>learning</strong> (Warburton, 2009). Butler and White (2008) categorise<br />
activities into three distinct types of educational models: real-time interactions such as role play<br />
exercises or teaching sessions; machinima or the viewing of virtual interactions including orientations<br />
and/or knowledge transfer; asynchronous interactions, for instance the creation of virtual objects for<br />
assessment or the response to scripted, existing objects/environments.<br />
311
Rose Heaney and Megan Anne Arroll<br />
A published example (Walker, 2009) of a real-time interaction model concerns the use of SL to teach<br />
counselling skills to distance <strong>learning</strong> students. When assessing students’ and staff’s experiences,<br />
Walker’s findings are mixed; students’ perceptions of the benefits of SL were only reported as slightly<br />
higher than those of more traditional <strong>learning</strong> environments, whereas the instructor found the<br />
environment very useful and reported that it offered potential in terms of pedagogy. This is at odds<br />
with other studies evaluating online <strong>learning</strong> environments (OLEs) (Jones and Jones 2005; Palmer<br />
and Holt; 2009), where students’ ratings were higher than staff’s. However this mixed picture of SL is<br />
typical of other such studies and only serves to underline the potential barriers to effective use of this<br />
complex and demanding environment summarised by Warburton (2009) under the headings of<br />
technical, identity, culture, collaboration and time amongst others. The present study aims to focus<br />
exclusively on academic staff’s experiences and perceptions in an attempt to uncover in greater depth<br />
both the barriers and advantages of using SL in a higher education context and thereby hope to<br />
contribute to greater pedagogical efficacy in future.<br />
3. Context of the present study<br />
The University of East London (UEL) has had a presence on SL since early 2009 in the form of two<br />
islands. The school of Psychology is housed on UEL island where it has some large lecture rooms,<br />
break-out rooms and tutorial huts outside as well as space for social interaction and information<br />
boards and notices. The second island UEL HABitat belongs to the school of Health and Bioscience<br />
and hosts a range of simulations and problem based <strong>learning</strong> resources including a laboratory for<br />
molecular biology experiments, a multi-disciplinary polyclinic with virtual patient scenarios and a crime<br />
scene house for forensic investigations as well as formal and informal meeting spaces. The HAB<br />
activities are designed in the main to be used by students on a self-directed, asynchronous basis with<br />
minimal use to date for synchronous or group <strong>learning</strong> activities. This contrasts with the use of SL by<br />
the school of Psychology as a virtual conference and teaching space to complement or replace<br />
equivalent activities and resources on the physical campus.<br />
Having commissioned a specialist SL developer to build conferencing and tutorial facilities on UEL<br />
Island, a group of staff in the school of Psychology have been running a range of activities for<br />
students from all levels of the BSc Hons Psychology programme since early 2010. The students on<br />
this programme are all part-time or full-time campus based students i.e. none are distance learners.<br />
This paper focuses on staff experience of running group tutorials in the specially designed huts<br />
outside the main building (Figure 1).<br />
Figure 1: Small tutorial hut on the UEL Island, outside the main Psychology building<br />
Tutorials on this programme are normally offered face to face and in the rare event of offering<br />
anything online, the primary mechanism would be the VLE (Blackboard) discussion forums or<br />
chatrooms. The main drivers for using SL were a perceived need to find alternatives to physical<br />
tutorial spaces which are at a premium on UEL’s compact Stratford campus and to establish the<br />
312
Rose Heaney and Megan Anne Arroll<br />
viability of SL as a meeting place in general when staff and / or students might be prevented from<br />
attendance at a campus location through health issues, transport difficulties, bad weather and the like.<br />
The SL tutorials were all optional and, apart from one which was integrated with a particular module,<br />
were on topics of general interest and advertised to students at all levels of the programme. They<br />
were held outside normal office hours either in the evening or at weekends meaning that the majority<br />
of participants (staff and students) were working remotely from the campus when they accessed<br />
them.<br />
In terms of preparation for the SL tutorials some of the students had attended face to face orientation<br />
sessions run by the developer earlier in the semester when the facilities had first been built. They also<br />
had access to self-help guides on Blackboard. Staff participants had attended the orientation sessions<br />
or already had some familiarity with SL. Overall the majority of participants in the tutorials were<br />
unfamiliar with SL, this being their first real use of it for teaching and <strong>learning</strong> purposes.<br />
According to the pre-session interviews, apart from attendance at the orientation sessions mentioned<br />
above, preparation by staff for the tutorials ranged from doing nothing specific, to linking the content<br />
with a previously delivered lecture and ensuring the students were adequately prepared. In the event<br />
attendance at the different tutorials ranged from 0 to 20+, the highest numbers being seen at the<br />
session that was linked to a specific module and where the member of staff knew which students<br />
were likely to attend.<br />
4. Methodology<br />
4.1 Epistemological and analytical framework<br />
The goal of this study was to explore the experience of teaching via SL; therefore an inductive, rather<br />
than deductive, approach was taken. This desire to uncover experiential data led to the decision to<br />
attain verbal, as opposed to numeric, data in a naturalistic setting. Thus, the approach of this study<br />
was one of phenomenology, which is concerned with meanings and the individual’s personal<br />
perception of events and experiences rather than trying to manufacture an objective statement of an<br />
event (in this case, conducting a tutorial in SL) or experiences (Giorgi, 1995). Hence, the most<br />
appropriate analytical technique under these circumstances and with this epistemological perspective<br />
was deemed Interpretative Phenomenological Analysis (IPA; Smith, 1996; Smith & Osborn, 2003) as<br />
it was able to take into account both the individuals’ experience of using SL and the researcher’s<br />
interpretations rather than simply stating the participants’ opinions of this novel tool.<br />
4.2 Participants<br />
Four full-time lecturing staff from the Psychology department of UEL were interviewed. All but one<br />
respondent participated in a pre- and post-teaching interview; the member of staff who was only<br />
involved in a post-teaching interview was asked to answer pre-teaching questions in retrospect as<br />
best he could. The sample consisted of 3 female and one male participant, all of whom were involved<br />
in undergraduate teaching duties. Three of the interviewees had some previous experience of SL but<br />
none had conducted tutorials prior to this study. One of them had experience of Adobe Connect and<br />
Skype for online tutorials with distance <strong>learning</strong> students.<br />
4.3 The interviews<br />
A study pack including an information sheet and consent form was given to each participant before<br />
the first interview in line with the ethical approval granted from the UEL Ethics Committee. To allow for<br />
flexibility within the academics’ schedules, the interviews were conducted over the telephone and<br />
recorded via a digital device. The interviews were semi-structured and consisted of a range of openended<br />
questions, including prompts that allowed further elaboration of the topic under discussion. If it<br />
was deemed that the participants had adequately answered any questions remaining on the<br />
schedule, these were omitted from the interview. The structure of the interview schedule complied<br />
with a funnelling approach (Guba & Lincoln, 1981), and started with a question regarding preparation<br />
for teaching within SL. Subsequently, participants were asked about methods of communication,<br />
experience of the environment, and their expectations at pre-teaching interview and their actual<br />
experience of their session within SL at post-teaching interview. The average duration of the<br />
interviews was 32 minutes pre-teaching, 12 minutes post-teaching and the single merged interview<br />
313
Rose Heaney and Megan Anne Arroll<br />
lasted 26 minutes. Pseudonyms were created for each participant to ensure their anonymity and full<br />
permission was obtained to use verbatim quotations in the final write-up of the study.<br />
4.4 Data analysis<br />
The process of IPA used in the study consisted of a number of stages starting with a case-by-case<br />
analysis and followed by comparison across cases. To begin with, one transcript was read thoroughly<br />
and repeatedly, which permitted the researcher to become familiar with the account, as each<br />
additional reading tended to evoke new insights. Initial coding then entailed noting down anything of<br />
interest, interpretations and making summaries of ideas. From this stage recurrent themes were<br />
extracted, with key words or phrases that captured the essence of the content acting as codes. This<br />
procedure was then repeated for each transcript. At this point the researcher attempted to identify<br />
repeated patterns emerging in the subsequent transcripts whilst still allowing additional topics to be<br />
identified (Smith & Osborn, 2003). The next stage involved looking for thematic connections, both<br />
within and across transcripts. These themes were then clustered and developed into a consolidated<br />
list of master or superordinate themes. The transcripts were reread to ensure that the themes and<br />
sub-themes could undoubtedly be recognised in the verbatim transcripts. Finally, transcript quotations<br />
were noted for each theme and a file created.<br />
5. Findings<br />
Three superordinate themes emerged from the data; comfort, investment and clear rationale. Each of<br />
these themes contain sub-themes which include both positive and negative views of using SL as an<br />
effective teaching environment and will be discussed in turn.<br />
5.1 Comfort<br />
The theme ‘comfort’ is based on the participants’ feeling and thoughts regarding the environment as a<br />
whole and how this impacts upon teaching and <strong>learning</strong>. The academics felt that the anonymity that<br />
the virtual environment provided was beneficial to students because ‘perhaps they’ll feel more<br />
comfortable in saying “I don’t understand it” because you’re not a person’ (Jane). Similarly, Greta<br />
thought that SL gave students ‘a way of asking anything they want without people knowing who they<br />
are’. This level of anonymity could be very important to students who find it difficult to vocalise<br />
problems and ask questions in real life for fear of embarrassment and could be most beneficial to<br />
those that do not have experience in presenting work, for instance students in their first year of higher<br />
education. Interviewed after his teaching session and after he had discussed it with students (this<br />
academic asked for direct feedback from his students following the SL session), Tom noted that:<br />
‘those that are… more reticent or nervous about contributing in class … were very vocal<br />
and explicit’ (Tom)<br />
Although students could benefit from the anonymity of the environment, staff on the other hand could<br />
feel too exposed. Jane in particular appeared concerned by her lack of experience:<br />
‘I do feel like it’s that I’m too young and inexperienced rather than a competent and<br />
that’s not a nice feeling I think…oh gosh, I must be really stupid because I can’t do x, y, z<br />
…’ (Jane)<br />
In addition to the psychological comfort that SL afforded students, there were practical advantages as<br />
well:<br />
‘I think it could be a useful way of… getting round … difficulties … in London, noncampus<br />
places…that there is actually some sort of social cohesion to what they’re doing.’<br />
(Tom)<br />
As Tom notes, there is often difficulty in engendering a community feeling when a university is split<br />
into separate buildings and campuses, not all of which may be accessed by all students, and<br />
therefore SL may provide a way of integrating student experience onto a common platform. Also, by<br />
developing a virtual world, scarce resources could be saved as ‘we’re not using University space’<br />
(Tom).<br />
Another area where participants could see a use for SL was its ability to engage students who could<br />
not attend traditional lectures:<br />
314
Rose Heaney and Megan Anne Arroll<br />
‘Those interested in distance <strong>learning</strong>, will probably be the ones who really get the most<br />
out of it… for people who are physically unable to get out perhaps, for whatever reason,<br />
whether it’s a physical thing or a childcare issue’ (Jane)<br />
This was not only seen as a benefit for students but also staff, especially in dealing with issues of<br />
transport and illness during the academic year:<br />
‘Programme leader’s meeting on Wednesday ….. a snowy place …. won’t be able to get<br />
in ….brilliant to have it in SL, everybody could be there.’ (Cathy)<br />
‘We thought swine flu was taking over and we’d have to hold meetings there, but then it<br />
didn’t so we didn’t. But that sort of thing would be useful.’ (Cathy)<br />
However, the interviewees also noted the limitations of SL as a teaching tool in terms of the lack of<br />
non-verbal information that is fundamental to effective teaching and <strong>learning</strong>:<br />
‘If you are teaching something for the first time on there, you would not, you wouldn’t get<br />
the visual feedback from the audience. So, for example, I had probably twenty-three<br />
students and I’d say fifteen, fifteen or sixteen were constantly responding to me…you<br />
assume from sort of radio silence that, that those individuals may not be engaged, but<br />
you would never know’ (Tom).<br />
Additionally, Greta stated that in comparison to other technologies, SL was not as effective:<br />
‘In other systems I’ve used there is a “hands-up” signal and that allows you to have larger<br />
numbers so it hasn’t got the mechanisms you need for successful teaching. As a general<br />
tool for interactions between students and lecturers online, I don’t think it’s very good.<br />
(Greta)<br />
In fact, Greta said that she would not recommend the use of SL for online tutorials. These concerns<br />
were echoed by Tom who was cautious with the proposition that SL should be used to the exclusion<br />
of other, perhaps more suitable, technologies:<br />
‘I think there’s a slight danger that people will just use it for the sake of it and that’s when<br />
you might start to go down avenues where it’s potentially not as useful as it could be.’<br />
(Tom)<br />
In sum, SL was seen to be advantageous in terms of the anonymity it lent to the students and the<br />
convenience of its virtual nature although these benefits have to be weighed against the difficulties in<br />
interaction and the danger of using it as a one-size-fits-all instrument.<br />
5.2 Investment<br />
Participants were unanimous in the belief that staff needed to prepare themselves and students well<br />
for SL activities. For example, Jane, who had not used SL in her teaching activities before the initial<br />
interview, stated:<br />
‘I will give myself plenty of time to make sure I’m there or at least know how to get there<br />
when it is time…but I’m still not very confident…I am sort of a bit nervous…I found it<br />
quite sickening moving, I’m one of these that gets motion sickness [but] it got better in<br />
the second one’ (Jane)<br />
However, for others who were more familiar with SL, this was not an issue and implies that, with<br />
training, early reticence may be overcome:<br />
‘I use SL, so I didn’t have to really get used to just interacting within the environment<br />
because I was already familiar with that’ (Greta)<br />
Tom was more concerned with ensuring that the students were able to navigate the environment as<br />
he had already attended a training session earlier in the year:<br />
‘I mean the big thing was preparing the students [as] I had no problems at all, but that’s<br />
probably a symptom of the fact that it occurred during a training session.’ (Tom)<br />
The strategy that Tom used to prepare the students included directing them to guides held on UEL<br />
Plus (Blackboard), an oral introduction to SL in his face-to-face session which included registering and<br />
setting-up an avatar, instructions on how to get to UEL Island and:<br />
315
Rose Heaney and Megan Anne Arroll<br />
‘The week before I said, “This is what I’m doing,” and I gave them, I pointed them in the,<br />
in the direction of those guides that I was talking about that were available on UEL Plus.<br />
And then I said, “If I see you I see you, if not, don’t worry’ (Tom)<br />
Although it was integrated with a specific module teaching programme, Tom offered his SL tutorial as<br />
an additional resource for the students, i.e. it was not compulsory. The issue of whether the virtual<br />
sessions should be mandatory was viewed differently by the participants; Cathy, who previously had<br />
organised a training session for staff found that only a third of the department attended that training<br />
session because ‘it’s all optional and people didn’t want to volunteer’. Jane stated that ‘if someone<br />
said we had to do it, then I’d have to do it’ but did not see the justification in using SL more<br />
extensively at present. This reluctance was viewed by Cathy as a possible impact on the student<br />
experience:<br />
‘I think they wouldn’t engage with it if they think that, some staff won’t engage with it’<br />
(Cathy)<br />
The importance of a positive first experience, the frequency of use and the need to integrate the<br />
virtual tutorials were also highlighted by the academics:<br />
‘It [first experience] has to be something that they really get a lot out of and think ‘yes,<br />
that was worth it. I just think if people are using it a lot, then they will become familiar with<br />
it and it will feel less daunting…It [SL] needs to be embedded like UEL Plus, if it was an<br />
option, they probably wouldn’t bother to go to it.’ (Jane)<br />
Therefore, from the participant narratives, it is apparent that there is a need for investment and<br />
organisation to reap the benefits of SL as a teaching tool. This can be achieved by holding training<br />
sessions for staff and students so that everyone can feel confident in the environment and able to<br />
appreciate the advantages of the environment for teaching and <strong>learning</strong>. This may require a paradigm<br />
shift in the mindset of mainstream higher education users but is by no means unreasonable to<br />
achieve; as Cathy points out ‘we’re in same position with SL as I used to be with the VLE’.<br />
5.3 Clear rationale<br />
Participants were divided on SL’s suitability as an effective environment for tutorials and group<br />
teaching generally although all agreed that it could be beneficial for role play exercises, problembased<br />
scenarios, student presentations and conferences:<br />
‘Things like counselling courses where it may give them a way of trying out counselling<br />
sessions in a completely safe, anonymous environment and [it is] useful for practice-type<br />
sessions, simulations, but not for lectures/seminars’ (Greta)<br />
UEL’s School of Health and Bioscience uses the environment for simulations and the interviewees<br />
could see SL as very beneficial for practice sessions with virtual patients and, as above, hypothesised<br />
that counselling trainees could interact through avatars before they were allocated to real world<br />
placements. This type of ‘practice’ was also believed to be of value for student presentations:<br />
‘I think that is a nice idea because I think the anonymity, again, is great and really, it<br />
gives them a very safe environment’ (Greta)<br />
Regarding traditional ‘chalk-and-talk’ lectures where large audiences may contain anti-social<br />
elements, Cathy shared her experience of audience behaviour while attending large talks in SL:<br />
‘I have watched a large set of lectures and things in SL before I got involved in this, I<br />
didn’t see any sort of bad behaviour or anybody rioting or anything’ (Cathy)<br />
Cathy also stated that for staff, a virtual environment would assist in some of the more difficult subject<br />
matter:<br />
‘Statistics is really difficult to teach in big lectures and that’s what I used to do and that’s<br />
what people are used to doing and they talk to 250 people on the t-test and it was very,<br />
very difficult then.’ (Cathy)<br />
Small group discussions and seminars were also seen as possibilities within SL by Cathy:<br />
‘In our conference centre, we’ve actually got a breakout room so that’s certainly possible<br />
to do that, after a lecture, you could ask so many people to go into this, so many, and<br />
you go off and feedback, that’s entirely possible.’ (Cathy)<br />
316
Rose Heaney and Megan Anne Arroll<br />
From the respondents’ experience of using SL for a tutorial, it appears that there is uncertainty if this<br />
type of teaching is the most suited to the environment. Similarly, half of the respondents believed that<br />
large-scale lectures would be effective in SL , although the lack of visual feedback from the students<br />
appears problematic in terms of gauging understanding. Consensus was found for more practical<br />
applications within SL , such as simulations and problem-based <strong>learning</strong>.<br />
6. Discussion<br />
This study has presented a qualitative analysis of the perceived barriers and advantages of using a<br />
virtual world technology, SL, as a tool in higher education from the perspective of academic staff.<br />
Previous research has shown that students report higher levels of satisfaction with teaching and<br />
<strong>learning</strong> in online <strong>learning</strong> environments than staff (Jones and Jones, 2005; Palmer and Holt, 2009)<br />
and that there may be numerous barriers to the effective use of SL in education (Warburton, 2008).<br />
The findings of this study have shown that the sense of comfort derived from being anonymous can<br />
act as both a benefit to students but also as an obstacle to teaching staff in the lack of non-verbal<br />
cues though some of this could be resolved by fuller exploitation of the environment i.e. use of<br />
gestures such as nodding etc. The participants’ narratives also illustrated the need for a clear<br />
rationale for the use of SL expressing the belief that problem-based, practical sessions may be more<br />
suited to this virtual environment than tutorials. There was also the assertion by some that SL might<br />
be an effective medium for large, traditional ‘chalk-and-talk’-type lectures to mitigate anti-social<br />
behaviour amongst other things, though this contradicted other views of not making use of SL<br />
mandatory.<br />
The need for investment, both in terms of time and funds, came through very strongly and accords<br />
with Butler and White (2008) who note that the return on investment that educational establishments<br />
gain from using SL is primarily dependent on making this type of technological innovation an<br />
institutional priority. In the present study, participants highlighted the need for investment in staff and<br />
student training, as without this front-loaded investment staff would not feel confident in using SL,<br />
which in turn would impact on student engagement and, subsequently, their <strong>learning</strong> experience.<br />
7. In conclusion<br />
The participants in this study perceived both advantages and problems associated with using SL as a<br />
teaching tool. Some of the limitations noted had more to do with staff’s inexperience than the<br />
limitations of the environment per se. However, whether their criticisms of SL were valid or not the<br />
overall sense was that SL is a beneficial addition to the <strong>learning</strong> environment provided staff feel<br />
comfortable, confident and well-supported in the technology and have sound reasons for using it.<br />
Participants could appreciate the more obvious applications for SL such as role play exercises or<br />
virtual patient interactions but also saw a place for it in traditional teaching such as the online tutorials<br />
featured in this study. In terms of moving forward with pedagogy, this study points towards the need<br />
to embed technology such as SL into module programmes from the outset with specific training for<br />
both staff and students. As to the lack of non-verbal cues frequently cited as a drawback in a SL<br />
tutorial or other type of SL teaching situation, recent initiatives such as that at University of Southern<br />
California to embed technologies such as Microsoft Kinect in SL herald much richer forms of feedback<br />
as SL users have a more natural means of conveying their real emotions and gestures to their<br />
avatars. Finally, this study suggests that when the positive aspects of the three super-ordinate themes<br />
identified are present a worthwhile SL educational experience is achievable whatever form that might<br />
take. In other words it would appear that the type of activity itself is less important than the conditions<br />
under which it takes place.<br />
Acknowledgements<br />
This study was funded by a UEL Teaching Fellowship awarded to Rose Heaney.<br />
References<br />
Boellstorff, T. (2008) Coming of Age in Second Life: An Anthropologist Explores the Virtually Human, Princeton,<br />
Princeton University Press.<br />
Butler, D. and White, J. (2008) “A slice of Second Life: <strong>Academic</strong>s, support staff and students navigating a<br />
changing landscape” [online], Proceedings ascilite Melbourne,<br />
http://www.ascilite.org.au/conferences/melbourne08/procs/butler-d.pdf.<br />
Giorgi, A. (1995) “Phenomenological psychology”, in J.A. Smith, R. Hare, & L. Van Langenhove (Eds.),<br />
Rethinking psychology, London, Sage.<br />
Guba, E. and Lincoln, Y. (1981) Effective evaluation, San Francisco, Josey Bass.<br />
317
Rose Heaney and Megan Anne Arroll<br />
Jones, G. H. and Jones, B. H. (2005) “A comparison of teacher and student attitudes concerning use and<br />
effectiveness of web-based course management software”, Educational Technology & Society, Vol 8, No 2,<br />
pp 125-135 [online] http://www.ifets.info/journals/8_2/12.pdf<br />
Palmer, S. and Holt, D. (2009) “Staff and student perceptions of an online <strong>learning</strong> environment: Difference and<br />
development”, Australasian Journal of Educational Technology, Vol 24, No 3, pp 366-381.<br />
Schroeder, R. (2008) “Defining virtual worlds and virtual environments”, Journal of Virtual Worlds Research, Vol<br />
1, No 1, [online] http://journals.tdl.org/jvwr/article/view/294/248.<br />
Smith, J. A. (1996) “Beyond the divide between cognition and discourse: using interpretative phenomenological<br />
analysis in health psychology”, Psychology and Health, Vol 11, pp 261-271.<br />
Smith, J. A. & Osborn, M. (2003) “Interpretative phenomenological analysis”, in J.A. Smith (Ed.), Qualitative<br />
psychology: a practical guide to research methods, London, Sage.<br />
University of Southern California (2011) “Using Kinect and OpenNI to Embody an Avatar in Second Life: Gesture<br />
& Emotion Transference”. [online]<br />
http://ict.usc.edu/projects/gesture_emotion_transference_using_microsoft_kinect_and_second_life_avatars/<br />
Walker, V. L. (2009) “3D virtual <strong>learning</strong> in counselor education: Using Second Life in counselor skill<br />
development”, Journal of Virtual Worlds Research, Vol 2, No 1, [online]<br />
https://journals.tdl.org/jvwr/article/view/423/463.<br />
Warburton, S. (2008) “Six barriers to innovation in <strong>learning</strong> and teaching in MUVEs”, Liquid Learning, [online]<br />
http://warburton.typepad.com/liquid<strong>learning</strong>/2008/07/six-barriers-to.html.<br />
Warburton, S. (2009) Second Life in higher education: Assessing the potential for and the barriers to deploying<br />
virtual worlds in <strong>learning</strong> and teaching, British Journal of Educational Technology, Vol 40, No 3, pp 414-426.<br />
318
Introducing and Using Electronic Voting Systems in a<br />
Large Scale Project With Undergraduate Students:<br />
Reflecting on the Challenges and Successes<br />
Amanda Jefferies<br />
School of Computer Science, University of Hertfordshire, Hatfield, UK<br />
a.l.jefferies@herts.ac.uk<br />
Abstract: Electronic Voting Systems (EVS) have become a popular medium for encouraging student<br />
engagement in class-based activities and for managing swift feedback in formative and summative assessments.<br />
Since their early days of popularity and introduction some five or more years ago, the author’s UK based<br />
University has been successful in refining strategies for their use across individual academic Schools and<br />
Departments, as previously reported at ECEL (e.g. Lorimer and Hilliard, 2008). The focus of this paper is a<br />
reflection on the introduction of EVS with 300 first year undergraduate students in the School of Computer<br />
Science, within the context of a wider ‘change’ project in teaching and <strong>learning</strong> affecting the whole institution. The<br />
author examines what lessons can be learnt following this rapid scaling up of EVS activity both at a local level<br />
and more widely across an HE institution and in reflecting on the successes and challenges of this experience<br />
provides key indicators for success and useful support for others considering using EVS. The paper first<br />
considers the landscape of EVS use within the UK and then the specific introduction of EVS at her own<br />
institution, before exploring the issues in her own academic School around the latest phase of their introduction<br />
as part of an institution–wide project to review measures to support assessment and feedback.<br />
Keywords: electronic voting systems, change management, assessment, feedback, staff training, challenges,<br />
successes<br />
1. Introduction to EVS at the University of Hertfordshire<br />
The University of Hertfordshire has supported the use of electronic voting systems (EVS) for some<br />
years since they originally started appearing as a piece of electronic technology to support teaching<br />
and <strong>learning</strong> around 2003 in the UK. Sometimes known as ‘clickers’, the technology grew in popularity<br />
in the U.S. where EVS had been used in a more routine manner as part of regular classroom activities<br />
in HE as well as across the broader education sector. There are a number of competing EVS<br />
systems available commercially but they all typically comprise four elements for working effectively;<br />
an electronic hand-set per student or group of students, a digital receiver to capture responses from<br />
the individual handsets, lecture presentation software into which questions can be embedded,<br />
typically based around the use of Powerpoint and with the facility to collate and present the<br />
responses in a variety of ways, and the associated hardware to present the questions i.e. a computer<br />
and digital projector.<br />
In our own early days of piloting them, sets of student handsets were available for lecturers to reserve<br />
for classroom use from the central <strong>learning</strong> support services. This requirement to pre-order the EVS<br />
was seen as rather unwieldy since time was required at the start and end of a class to first of all hand<br />
out the handsets and then collect them in again afterwards. This was not an insignificant commitment<br />
when there might be a class of over 100 students in an hour long lecture. Additionally the software<br />
and receiver were also required to be pre-ordered for each class. The developing practice of EVS and<br />
the design of questions for their in-class use was initially focussed on a small group of <strong>learning</strong> and<br />
teaching technology ‘enthusiasts’ (Moore, 1982) working across the different schools of the<br />
university. From this early interest in using the EVS for multiple choice questions (MCQ) and in-class<br />
quizzes, their use was taken up in local pockets of interest by those who saw an opportunity to<br />
increase interactivity in the classroom,(JISC 2005). This opportunity to engage more closely with<br />
student <strong>learning</strong> and focus the in-class activities around MCQs was particularly developed at the<br />
University of Hertfordshire by those teaching in the vocational health related subjects and with smaller<br />
class sizes, (Lorimer and Hilliard 2007). At the same time the pedagogy and practice of the use of<br />
EVS was being researched and reported from a variety of scenarios across UK based HE institutions<br />
(Draper and Brown 2004;Kennedy and Cutts 2005).<br />
Our institution’s initial success might therefore be described as having been built up through a<br />
successful ‘cottage industry’, where local EVS champions crafted their expertise with groups of<br />
students and developed a series of action research projects in the process. The resulting interest in<br />
the pedagogy into the use of EVS and its potential for use in social constructivist approaches to<br />
319
Amanda Jefferies<br />
<strong>learning</strong> was further developed in line with research outputs both from the University of Hertfordshire<br />
(Lorimer and Hilliard, 2007; Russell, 2008) and further afield, for example: Nichol (Nichol, 2007) at the<br />
University of Strathclyde, and Masikunas (Masikunas, Panayiotidis et al. 2007). All these researchers<br />
have indicated an increase in student engagement with the content of their <strong>learning</strong> when EVS were<br />
used regularly. The EVS technology was refined over the past five years and their use was no longer<br />
seen as only requiring a series of knowledge-based MCQs, where the expected answers are<br />
designed to be purely testing a student’s knowledge in terms of Bloom’s taxonomy of <strong>learning</strong> (Bloom<br />
1953).<br />
In 2008 the University’s School of Life Sciences piloted the use of EVS with a complete cohort of<br />
undergraduates who were given a personal handset and retained it for use across their<br />
undergraduate careers until they graduated. As a result of this scheme there was a further increase in<br />
the training of academics and a necessary development of study materials to promote interactivity<br />
and engagement. This in turn led to an archive of local material and expertise which demonstrated<br />
some of the wide variety of ways in which EVS were being used creatively in the classroom by<br />
academics across the curriculum. The following examples indicate situations where EVS use became<br />
commonplace within the wider Faculty; group discussion and decision-making; ethical debates with a<br />
‘before’ and ‘after’ poll; conditional branching, where students follow through the logical<br />
consequences of their initial answer to a question to its conclusion in a case study, (Lorimer and<br />
Hilliard 2009) as well as revision quizzes to test previously taught material.<br />
In response to this increased enthusiasm among academics and students in Life Sciences where the<br />
use of EVS had grown over six years to become a regular feature of the student classroom, a crossuniversity<br />
project was drawn up to introduce the use of EVS across a much broader group of the<br />
university’s campus-based students from September 2010. The aim of this ambitious project was to<br />
promote a culture change in the use of <strong>learning</strong> technologies by providing an impetus for the use of<br />
inter alia EVS by an extended set of academic schools and to increase student engagement in line<br />
with the University’s own stated agenda for improving the student experience.<br />
The project was designed and grounded in line with much of the previous work of the University’s<br />
Learning and Teaching Institute in the principles for good assessment practice and feedback (Nichol<br />
and Macfarlane-Dick, 2006) and underpinned by the seven principles of ‘Good Teaching and<br />
Learning’ (Chickering and Gamson, 1999).The use of EVS had already become widespread in other<br />
institutions, (see above e.g. Draper and Brown 2004, Kennedy and Cutts 2005) but their take-up was<br />
not always valued beyond the increased student engagement in classes nor did EVS necessarily lead<br />
to an overall reduction in costs for the host School. Student satisfaction and their enjoyment of EVS<br />
had been identified as a useful marker of success, (Lorimer and Hilliard, 2008); however, the broader<br />
aims of this project wanted to delve into the nature of student engagement with EVS and see if their<br />
use could lead to measurable savings overall. These savings were envisaged as being primarily time<br />
savings through a reduction in the assessment burden for staff, reducing their marking load or<br />
replacing an examination with regular EVS tests. For some programmes engaged in the wider project<br />
there was an expectation of higher student retention rates (Nichol, 2007).<br />
Following an invitation to individual Schools from across the university to bid for funding, the project<br />
proceeded with the provision of personal EVS handsets to nearly 4,000 students across nine<br />
academic departments for use throughout the second semester of the academic year for a minimum<br />
period of at least 6 months. Many Schools intended at this stage that undergraduate students should<br />
expect to retain their handsets throughout their university career. In practical terms this ambitious<br />
project required that a number of the academics in participating Schools would have to both undergo<br />
training in the use of EVS and to reflect personally on any necessary pedagogical changes to their<br />
practice to be made, once students were provided with a personal EVS handset. The overall project<br />
also required a necessary commitment to the scaling up of teaching and <strong>learning</strong> support, to solve the<br />
practicalities of organising the handset distribution to individual students, as well as considering the<br />
provision that needed to be made for the increased number of disabled students, who were now<br />
participants in the project.<br />
2. The local situation: Introducing EVS across a wider user group of<br />
academics<br />
The situation in the School of Computer Science with regard to the use of EVS essentially mirrored<br />
practice across the wider university. There was a small group of enthusiasts who had been reviewing<br />
320
Amanda Jefferies<br />
their pedagogy and developing their own practice continually since the original opportunity for the<br />
provision of handsets. During the previous year a small-scale ‘change management’ project, called<br />
‘Piloting Innovative Practice’ (PIP) was developed in the School in line with Change Academy<br />
principles (Anderson et al, 2008; Doolan, 2010; Saward and Anderson, 2010). One mini-PIP project<br />
explored the potential of EVS in a non-traditional use for peer assessment and feedback for student<br />
presentations, as a new way of managing student engagement in class. This PIP was extremely<br />
successful in terms of engaging students and cutting staff time in administration of mark allocation<br />
and collection. It was subsequently demonstrated to other academics in the School, with the intention<br />
of gaining greater buy-in for the School’s participation in the wider EVS project planned for Autumn<br />
2010. The scene was now set for developing the use of EVS more widely across the School in the<br />
next academic year.<br />
The University of Hertfordshire operates on a semester based system and throughout the first<br />
semester 2010-2011 between October and December 2010, there was much planning required for<br />
the introduction of EVS to a cohort in excess of 4,000 students. The School of Computer Science<br />
planned to introduce EVS with first year undergraduate students on the BSc Computer Science and<br />
Information Technology programmes, having previously worked with Masters students and final year<br />
undergraduates. Each student registered on the programme would be given a handset, which it was<br />
intended they would be able to use through the rest of their undergraduate programme. The<br />
importance of the anonymity of the handset user has been widely discussed in the literature, (e.g.<br />
Banks, 2006) but it was also important to know which handset belonged to which student so that they<br />
could be used regularly in summative testing. The process therefore required each student to have<br />
their handset registered to them and a central database was designed and linked to the managed<br />
<strong>learning</strong> environment (MLE) to accommodate this instead of individual schools keeping their own<br />
records. This also facilitated the movement of students between schools where their studies<br />
overlapped from one to another, for example students who majored in Business might also be<br />
studying Computer Science but should only receive a single handset from the university.<br />
<strong>Academic</strong>s teaching on the programme who were typically very experienced with teaching first year<br />
undergraduates were offered training and additional support for re-designing their materials as<br />
necessary, as part of their participation in the project. The central university project team offered<br />
seminars and developed online ‘how to’ videos of, for example, downloading student lists and how to<br />
register student handsets. The chosen provider of the EVS technology at the University of<br />
Hertfordshire was Turningpoint; this was based on the prior experience of handsets and software<br />
used by the School of Life Sciences and the Radiography Department and from the comparison of<br />
user experiences recorded since the early days of using locally bought sets of EVS at the university.<br />
The various reporting facilities of the software allowed student scores to be gathered (as they were<br />
linked to their personal handset) and thus the use of EVS for summative testing as well as for<br />
formative work.<br />
2.1 A methodology for capturing the staff and student responses to the EVS<br />
The author’s role within the School was to both lead the project and to provide support for colleagues<br />
on a day-by-day basis as they incorporated more EVS-type activities into their teaching. The<br />
methodology for assessing and evaluating the introduction of the EVS to the cohort has used both<br />
qualitative and quantitative methods and some of the work in assessing the student experience is still<br />
ongoing at the time of writing. In addition the author provides her own reflections on the process and<br />
the outcomes of the project in the conclusion below.<br />
A set of student-focussed questions with a Likert scale for answers was used by Computer Science to<br />
enquire into the student experience. The questions were incorporated into a presentation and used by<br />
staff at the end of a lecture to gather student opinions about using EVS (n= 50). Additionally, because<br />
the case for EVS use with students has been discussed widely in the literature the author was<br />
interested in the challenges that their introduction might bring to established teachers and so<br />
individual face to face interviews were held with a cross-section of colleagues in the School, those<br />
with prior experience of using EVS and those with none. Their comments are reported below.<br />
3. Successes from the EVS project in the School of Computer Science<br />
In this section the author considers four areas where the introduction of EVS could be judged a<br />
success, in terms of student and staff engagement with both the process and the outcomes and the<br />
321
Amanda Jefferies<br />
way in which the outputs from the project have informed ongoing enquiry into assessment and<br />
feedback approaches.<br />
3.1 Student engagement<br />
It has been widely reported by inter alia Lorimer and Hilliard, (2008, 2009), Nichol (2007) that<br />
increasing interactivity in the classroom promotes a greater student engagement with their <strong>learning</strong>.<br />
The outcome from the student side of the use of EVS has been positive. As technically-literate<br />
students they rather predictably found the EVS ‘easy to learn’ and ‘easy to use’. They particularly<br />
liked the immediacy of the feedback when EVS were used in class (80%) and would recommend their<br />
use for students studying the same module the following year and in other modules. They also widely<br />
agreed (74%) that ‘using EVS helped my understanding of the module’. They did not all agree (54%)<br />
that the use of ‘Weekly EVS questions were the only reason for attending lectures’, but this could be<br />
attributed to the variety of styles of use by academics across modules. A larger group responding to<br />
the same questions in the Business School (n=200) had similar responses in terms of student use of<br />
EVS. This can be deemed a success for the local use of EVS. although the results could have been<br />
predicted from the many other national studies.<br />
3.2 Staff engagement<br />
Five undergraduate module teams committed to using EVS for some of their teaching from February<br />
to April 2011. Prior experience of their use was limited but some academics embraced the new<br />
technology with great enthusiasm, fired up by the examples and opportunities shown through the<br />
training sessions.<br />
“Seeing different examples of experienced colleagues’ use of the EVS inspired me to try<br />
other ways to use them myself.” (Module tutor)<br />
Other colleagues who had previously used EVS were keen to experiment with their use in large<br />
groups and seminars particularly now when the handsets were permanently linked to individuals and<br />
they did not need handing out at the start of a class and their ownership recorded for that<br />
assessment.<br />
“Being able to automatically link each handset with the student’s personal ID really<br />
reduced the time taken for inputting and checking marks.” (Module tutor)<br />
Particularly popular were the use of ‘surprise quizzes’, dropped in randomly to the teaching sessions<br />
but with a small summative assessment element, to ensure regular attendance and high engagement.<br />
These were used by one module team with great success as the gathering in of marks was immediate<br />
and removed much of the burden of previous testing with MCQ style questions, marked by hand.<br />
The ‘drop- quizzes’ proved to be a successful method for encouraging regular student<br />
attendance and engagement throughout the module.” (Module leader)<br />
Other teams explored the use of student discussion with EVS, before a group was required to give<br />
their joint answer, as a means of encouraging the students to articulate their understanding and to<br />
explain the solution to a problem to others.<br />
3.3 Developing the User database<br />
Much of the success of the EVS project and its embedding into the culture of the teaching of this<br />
cohort of first year undergraduates could not have been achieved without the initial design of the user<br />
database and its subsequent use for generating separate attendance lists of students registered for<br />
each module. The process of registering each handset to its student owner once and for all removed<br />
the time-consuming handing out and collection of handsets on a per class basis. Each handset has a<br />
unique barcode identifier and when the handsets were issued a barcode reader was used to register<br />
first the handset and then the student’s ID card number. This was recorded on the university’s MLE<br />
and provided an entry which could be searched either by student ID or by handset ID. In the case of a<br />
lost handset being retrieved, any lecturer in the university could access the database and arrange to<br />
notify the student of the return of the handset. Furthermore if students had their handsets piled onto a<br />
table during a discussion, it was easy to identify which handset belonged to which student. The<br />
student registration lists were downloadable from the central MLE and customisable for each module.<br />
322
3.4 Building on previous work<br />
Amanda Jefferies<br />
The success of the project at a local level was also due to the leadership of the project at an<br />
institutional level. This had built on the success of the introduction of the university’s MLE (Jefferies<br />
and Oliver, 2003) and its Blended Learning Unit (Bullen et al, 2009) which between 2005 and 2010<br />
had encouraged broader use of technologies to enhance <strong>learning</strong> and teaching practice across the<br />
university and to develop associated pedagogies, to enhance the student experience. Accordingly,<br />
lessons learnt from prior use of EVS across the university were shared, with the more experienced<br />
users from the School of Life Sciences and the Department of Radiography exploring in seminars and<br />
workshops how they had developed their own ‘best practice’ guidelines in the use of EVS to build<br />
constructive formative assessments which served to encourage interactivity in class and greater<br />
student engagement with their <strong>learning</strong> materials. This enthusiasm was infectious and encouraged<br />
colleagues to take up the challenge.<br />
4. Challenges of the EVS project<br />
While some of the immediate successes of the project have been outlined above, it is however worth<br />
noting the challenges which were faced during the swift introduction of EVS across the nine<br />
participating Schools and in Computer Science in particular. These have been categorised in terms of<br />
the hardware and software issues, and staff time and ongoing training.<br />
4.1 Hardware and software updating<br />
Introducing new IT can traditionally be a challenging time for both project managers and IT users. The<br />
challenges might include the human factors issues faced by the users who are asked to incorporate<br />
new ways of working with IT and in this case to design new ways to present their teaching materials.<br />
Alternatively, issues may arise as a consequence of the timing of the upgrading of the IT hardware<br />
and software and the availability of technical support in a large organisation, where major tasks and<br />
associated planning of their staffing might be scheduled on a three to six month or annual advance<br />
planning basis. In relation to this project the technical challenges related firstly to the timescales of the<br />
EVS introduction and to a previously unanticipated need to install new EVS receivers on some of the<br />
classroom hardware, and secondly to software issues where there was an incompatibility issue with a<br />
previous version of the software, already installed on certain classroom based PCs. An upgrade of the<br />
software from the previous academic year when the Computer Science CABLE project took place<br />
meant that new software had to be downloaded by the academics. This was not in itself a problem as<br />
this was a simple web download onto staff machines, but the problem arose as a new digital receiver<br />
was required and the earlier versions of the software which were already loaded onto some<br />
classroom and staff machines were then found to be incompatible with this new receiver. The<br />
complexity of the project’s roll out of EVS to academic staff was compounded with the nine Schools<br />
being based across two separate but linked campuses, which had different provision for software<br />
support.<br />
The newer campus commissioned and opened originally in 2003 had an integrated software upgrade<br />
system whereby selected software products could be upgraded centrally for each classroom on the<br />
campus overnight. The older campus whose IT systems were historically supported at faculty level<br />
did not at the time offer a centralised software upgrade facility and the necessary upgrading of the<br />
Turningpoint software along with the installation of tamper proof hardware receivers had to be<br />
undertaken on a per room basis. The scheduling of this significant task was soon realised to be too<br />
great to be undertaken during the normal teaching timetable throughout the semester when rooms<br />
were rarely out of use for more than an hour at a time. The temporary solution for the Computer<br />
Science academics based on this campus was to use a dedicated laptop on which was installed the<br />
recent software upgrade which was compatible with the new receivers purchased and to take this<br />
along to their classes with the portable receiver. This then required rather more ‘kit’ to be taken to<br />
classrooms instead of the preferred option of using a memory stick with their ready prepared<br />
materials installed. This challenge will have been removed by September 2011 because a schedule of<br />
hardware and software revision and upgrading has been put into place for the summer months when<br />
the classrooms are less heavily scheduled, in preparation for the new academic year.<br />
4.2 Staff time<br />
<strong>Academic</strong> staff readily took part where possible in the training sessions offered prior to the start of the<br />
semester. Some of those newer to the lecturing environment willingly and enthusiastically prepared<br />
323
Amanda Jefferies<br />
their teaching materials taking the opportunity to incorporate regular EVS sessions throughout the<br />
semester.<br />
For those with a particularly heavy teaching load which included the cohort of 300 students, through<br />
February to April, there was much less time and inclination to change tried and tested ways of<br />
delivering familiar material to incorporate the newer technology and introduce greater interactivity than<br />
they were already. In some cases it was the smaller tutorial sessions where the use of EVS proved<br />
most successful and where students shared the opportunity to discuss their answers collectively<br />
before giving a group response.<br />
This challenge will be approached by offering more local support for reconsidering the pedagogical<br />
issues related to social constructivist approaches of using EVS and associated course re-design<br />
during the quieter non-teaching periods of the summer. The module teams will thus be encouraged to<br />
plan their regular use of EVS with greater elapsed time for planning than was available in the 2010 -<br />
2011 academic year. Further training opportunities are being offered at both entry and advanced<br />
levels to promote the maintenance of the necessary skills.<br />
5. The way forward: Plans for 2011-2012<br />
The old proverb states that ‘Rome was not built in a day’, neither can such a major change as<br />
introducing EVS across 4,000 students with the resultant impact on pedagogical approaches be fully<br />
achieved within a single academic semester. The lessons learnt in the School of Computer Science<br />
from introducing 300 students and lecturing staff to using EVS include those issues which arose<br />
internally and those which arose through external decision-making.<br />
The enthusiasm of the core staff who have been using EVS regularly has been matched by the<br />
student engagement and enthusiasm for using the handsets. Plans have now been made to offer<br />
ongoing training so that this enthusiasm will encourage others to re-design and remodel their module<br />
delivery. It has become clear that local support is vital for the ongoing success of using EVS<br />
otherwise the use of this or any other newly introduced technology is likely to subside over time until<br />
the users include only those keen and already committed users. Building on the lessons learnt since<br />
the introduction of our university’s MLE from 2001 (Thornton et al,2004), it is therefore planned to<br />
have a network of local EVS champions to support academic colleagues with technical issues and<br />
through the pedagogical redesign of course delivery. Over the next year this will help to embed a<br />
changing culture in the School which will encourage the increased use of EVS, after which each<br />
succeeding cohort of undergraduate students will be offered their own free handset on entry to the<br />
university. There has been a programme of continuing classroom support via our Equalities Unit for<br />
those students whose disabilities mean they do not have the fast reaction motor skills for using EVS<br />
in tests.<br />
6. Reflections and conclusions<br />
The university’s project to make EVS available to a far wider group of staff and students than<br />
previously envisaged has already (June 2011) had some notable successes. Students have<br />
expressed their enthusiasm as users of the EVS, where this has been measured across the<br />
university. Students appreciate knowing their results and receiving feedback immediately through the<br />
use of EVS, with the opportunity to learn from their mistakes and have the correct answers explained<br />
instantly. This outcome has been in line with past research into student experiences of their<br />
introduction and use. Further savings are foreseen at this institution in terms of redesigning<br />
summative assessments for the forthcoming academic year so that EVS can be used to save more<br />
academic time currently taken up with repetitive marking of scripts. At the time of writing not all<br />
results are available but a significant increase in first year retention rates seems likely.<br />
As far as the pedagogy of using EVS is concerned this has been shifting for some time away from a<br />
culture where the traditional lecture dominates as a teaching medium and towards a more socially<br />
constructivist view of gaining knowledge. This process can not happen overnight nor even in a single<br />
academic year but the introduction of the EVS has offered further evidence of greater student<br />
engagement in their <strong>learning</strong> and of a changing academic culture that embraces this.<br />
324
Acknowledgements:<br />
Amanda Jefferies<br />
The author acknowledges the generous support of colleagues across the University of Hertfordshire<br />
in promoting EVS and sharing their experiences.<br />
References<br />
Anderson, I., P. Bullen, et al. (2008). “CABLE: an approach to embedding blended <strong>learning</strong> in the curricula and<br />
across the institution”. Reflecting Education 4: 30-41.<br />
Bloom, B. S. (1956). Taxonomy of Educational Objectives, Handbook 1: The Cognitive Domain New York, David<br />
McKay Co Inc.<br />
Bullen,P., Russell, M., Jefferies, A. (2009) "Harnessing technology, transforming higher education for the Future"<br />
Higher Education Academy Conference, University of Manchester,<br />
Chickering, A. W. and Z. F. Gamson (1999). "Development and Adaptations of the Seven Principles for Good<br />
Practice in Undergraduate Education." New Directions for Teaching and Learning 1999 Vol 80 pp 75-81.<br />
Doolan, M. (2010). "Pedagogical and Innovative Practice: Using technologies to Support Assessment and<br />
Assessment Feedback in Real-Time" Proceedings of 9th European Conference on E-Learning, Oporto,<br />
Portugal, ACI.<br />
Draper, S. W. and M. I. Brown (2004). "Increasing interactivity in lectures using an electronic voting system."<br />
Journal of Computer Assisted Learning Vol 20 Issue 2 pp 81-94.<br />
Jefferies, A., and A. Oliver, (2002) "Easing through the Pain Barrier – introducing a campus-wide MLE for staff<br />
and students" 3 rd ILTHE conference, Heriot-Watt University, July 2002<br />
JISC (2004). "Effective Practice with E-Learning " pubd HEFCE at<br />
http://www.jisc.ac.uk/e<strong>learning</strong>_pedagogy.html<br />
Kennedy, G. E. and Q. I. Cutts (2005). "The association between students' use of an electronic voting system<br />
and their <strong>learning</strong> outcomes." Journal of Computer Assisted Learning Vol 21 Issue 4 pp 260-268.<br />
Lorimer, J. and A. Hilliard (2007). "Net gen or not gen? Student and staff evaluations of the use of<br />
Podcasts/AudioFiles and an Electronic Voting System (EVS) in a Blended Learning module". 6 th European<br />
Conference on E-Learning, Copenhagen Business School, ACI.<br />
Lorimer, J. and A. Hilliard (2008). "What is your response? It's time to get personal". 7 th European Conference on<br />
E-Learning, Aghia Napa, Cyprus, ACI.<br />
Lorimer, J. and A. Hilliard (2009). "Use of an Electronic Voting System (EVS) to facilitate Teaching and<br />
Assessment of a Decision Making Skills in Undergraduate Radiography Education".8 th European<br />
Conference for E-Learning, Bari, Italy, ACI<br />
Masikunas, G., P. Panayiotidis, et al. (2007). "The use of electronic voting systems in lectures within business<br />
and marketing: a case study of their impact on student <strong>learning</strong>." ALT-J Research in Learning Technology<br />
15 Issue 1 pp 3-20.<br />
Nichol, D. (2007). "Laying a Foundation for Lifelong Learning: Case Studies of E-assessment in Large First Year<br />
Classes." British Journal of Educational Technology 38 Issue pp 668-678.<br />
Nichol, D. l. and D. Macfarlane-Dick (2006). "Formative assessment and self-regulated <strong>learning</strong>: a model and<br />
seven principles of good feedback practice." Studies in Higher Education 31 Issue 2 pp 199-218.<br />
Russell, M.B.(2008) "Using an electronic voting system to enhance <strong>learning</strong> and teaching," Engineering<br />
Education 3, Issue 2<br />
Saward, G. and I. Anderson (2010). “Macario Rock, Paper, Scissors: Overcoming Barriers to Changing Staff<br />
Practice in Technology Enhanced Learning.” 9 th European Conference on E-Learning, Oporto, Portugal,<br />
ACI.<br />
Thornton, M., Jefferies, A. et al. (2004). “How was it for you? An evaluation of Student Learning Experience<br />
Following the Introduction of an MLE in one English University” BERA, UMIST, Manchester.<br />
325
A Methodology for Incorporating Usability and<br />
Accessibility Evaluations in Higher Education<br />
Anne Jelfs and Chetz Colwell<br />
Institute of Educational Technology, Open University, Milton Keynes, UK<br />
a.e.jelfs@open.ac.uk<br />
c.colwell@open.ac.uk<br />
Abstract: Digital academic resources are rapidly growing in number and range of content, including visual as<br />
well as text based materials. To access these resources in a timely and effective way necessitates an easy to<br />
use and easy to learn environment. We consider in this paper the needs and requirements of disabled as well as<br />
non-disabled students who use these resources for study-related activities. We are currently involved in<br />
evidence-based research at the Open University on the uptake of usability, accessibility and learnability in the<br />
resource development cycle. To gain feedback from students we use a number of qualitative research methods<br />
primarily through interviews and observation studies and in our evaluations we unite disabled and non-disabled<br />
student user experience for final reports to faculties and developers The aim is to develop best practice with<br />
guidelines and information to support the institution in its development of web-based provision. At the Open<br />
University we have nearly 12,000 students with a declared disability, so ensuring the accessibility of course<br />
materials and websites is of primary concern as a large proportion of our provision is mediated online. However,<br />
we also need to retain the usability of resources because of the large non-disabled student population (approx<br />
180,000 post graduate and undergraduate students). Therefore we collaborate together to ensure that<br />
differences in usability and accessibility feedback are acknowledged and potential conflicts are part of the<br />
evidence we use when reporting to Module Teams and developers. What we are able to demonstrate in our<br />
paper is the ability to include multiple forms of evaluation including ‘expert’ walk throughs and the use of different<br />
assistive technologies, and end user evaluations. We report on further developments of the methodology<br />
developed by Colwell and Jelfs (2005; 2007) which is thought to be unique in ensuring that potential conflicts<br />
between usability and accessibility are minimised. In our paper we will present recent Open University<br />
developments to illustrate our methodology.<br />
Keywords: accessibility, usability, evaluation<br />
1. Introduction<br />
There has been increased use and development of information technology as a tool to disseminate<br />
teaching and <strong>learning</strong> across the world. The Internet has become a part of many people’s daily lives,<br />
which in turn has led to the development of educational materials for presentation on the ‘web’ and<br />
increasingly courses and universities are integrating online materials in their face-to-face teaching<br />
(Crowther, Keller & Waddoups 2004; Kelly, Phipps & Swift 2004; HEFCE 2010). For many students<br />
and teaching staff their first experience of eLearning is through their institution’s virtual <strong>learning</strong><br />
environment (VLE) which provides online teaching materials, university communities, assessment and<br />
other university specific activities. It is often seen that web-based or university based intranet<br />
resources can be one way to increase student access to materials and potentially to widen access to<br />
a new and increasingly diverse student cohort. More recent studies have shown that technology has<br />
become an ubiquitous and integral part of students’ lives (Jefferies & Hyde, 2009). However, this<br />
potentially changes the emphasis from paper-based course materials and student support to online<br />
provision of these key features of a university.<br />
In this paper we discuss our work at the Open University (OU) which is Europe’s largest educational<br />
establishment, delivering mainly distance <strong>learning</strong> modules. The OU has provided distance education<br />
since the early 1970s and has a strong commitment to eLearning. The OU currently has 180,000<br />
active students, of which over 11,000 declare a disability (~ 5.5%). Approximately one-half of all<br />
disabled students receive some form of support from the university to enable them to participate in<br />
their studies, for example providing audio files instead of text or software support through the<br />
Disabled Student Allowance. The OU has also made extensive investment in eLearning particularly<br />
since the late1990s for all students and there is a current major development programme for our VLE.<br />
The OU also has a commitment to widening access to higher education, to providing high-quality,<br />
interactive educational materials that meet students’ needs and operating within the mission of<br />
‘openness to all’. The OU is committed to making its online educational content and student services<br />
accessible to disabled students and usable by all; a considerable challenge given the size of the OU<br />
student population (Cooper, Colwell & Jelfs 2007). Although we are reporting here on our work in<br />
326
Anne Jelfs and Chetz Colwell<br />
distance education we realise that all universities face similar issues. That is providing good electronic<br />
support to staff and students and therefore evaluating how students are working with module<br />
materials and online provision is paramount.<br />
2. Evaluating internet based educational resources<br />
As a relatively new form of educational delivery there is no clear definition of how education resources<br />
presented electronically can to be evaluated. To feed into the development of our framework we<br />
looked at current practices and the accepted methods of data collection for evaluating electronic<br />
materials. Our methods are based on valid evaluation practices, but our findings inform our<br />
framework.<br />
A number of recommendations relating to the evaluation of interactive systems have been established<br />
by researchers (see Hix & Hartson, 1993; Nielsen 1994; Newman & Lamming, 1995; Joynes, 2000;<br />
Sheard & Markham 2005; Light, 2006). For example, the notion that participants should be recruited<br />
from the target population of users and that participants should be observed as they complete a set of<br />
pre-planned representative tasks (van Schaik, 1999). In fact, there is an assumption that a<br />
comprehensive evaluation of a web-based environment needs to consider both the technical and the<br />
pedagogical aspects of the system (Sheard & Markham 2005).<br />
A major challenge for designers and human computer interaction researchers is to develop software<br />
tools able to engage novice learners and to support their <strong>learning</strong> even at a distance, because poorly<br />
designed interface which makes students feel lost, confused, or frustrated will hinder effective<br />
<strong>learning</strong> and information retention (Ardito, Costabile, DeMarsico et al 2006). Ihamäki & Vilpola (2004)<br />
say that if the interface of a VLE is not usable, the user’s focus on the actual content is diminished,<br />
because using the VLE requires a considerable amount of concentration. Good usability, on the other<br />
hand, allows the user to focus on the content thus improving <strong>learning</strong> results and making websites<br />
accessible has growing importance as the Intranet becomes the main delivery channel for eLearning<br />
resources (Phipps & Kelly 2006).<br />
Often educational evaluations focus on <strong>learning</strong> outcomes, however with the growth in online<br />
provisions, our research focuses on making the system easy to use and to improve access to high<br />
quality education. In the next section we consider the recognised forms of usability and accessibility<br />
evaluations before moving to the methods we have adopted in our joint work.<br />
One of the methods at the forefront of software evaluations is verbal protocol which is also know as<br />
think aloud protocol This is where a participant is asked to speak about what they are doing and why.<br />
The think aloud protocol requires the participant to complete a set of tasks using a prototype website<br />
and to ‘think aloud’. To say everything they are thinking while they complete the tasks (Turnbow,<br />
Kasianovitz et al 2005). The participant needs to maintain a commentary about what they are doing<br />
and why they are taking a particular path. This allows the researcher to have some understanding of<br />
the choices the participant has made.<br />
It is also valuable if the participants are observed and video recorded so that the participants’ actions<br />
can be used in further data analysis (Hughes & Parkes 2003). Usually there are two observers where<br />
one observer notes everything the participant says and if and how the participant completes the task.<br />
Another observer directs the participant and answers any questions asked. This type of user test<br />
provides essential real-time feedback on potential problems in the design and organization of a<br />
website. It is useful because it gives the researcher immediate information that may be forgotten by<br />
the participant at the end of the evaluation session. An approach we adopt is to have chunks of tasks<br />
so the students are only talking aloud for short periods within the longer session.<br />
When conducting think aloud sessions Dumas (2001) considers there to be three levels of interaction.<br />
Level 1 of verbalization expects the person conducting the research to sit behind and out of view of<br />
the participants and not to speak to the participant unless the person does not follow the instructions.<br />
If participants stop talking they are prompted to keep on talking. This is usually used in problem<br />
solving tasks. Level 2 is the manipulation of non-verbal information such as geometric shapes and<br />
Level 3 participants are asked to verbalise but also to explain why they are thinking or doing<br />
something. It is at Level 3 we usually work, but have on occasions tried Level 1, but our work is<br />
primarily not problem solving, but understanding how we can improve on the current provision.<br />
327
Anne Jelfs and Chetz Colwell<br />
To try and overcome any problems associated with talking aloud Wright and Monk (1991) told users<br />
to think of themselves as ‘co-evaluators’ of the system. Wright and Monk’s ‘relaxed’ approach allows<br />
the evaluator to ask questions of the participant, which maximizes the possibility of identifying and<br />
correctly interpreting the problems experienced by the participant. Hertzum, Hansen & Andersen<br />
(2009) found that relaxed think aloud makes participants search more comprehensively and in a<br />
different way to the classic think aloud approach.<br />
Another accepted form of evaluation of electronic provision is heuristic evaluation. Heuristic<br />
evaluation is conducted by studying a user interface and then forming estimation about what is good<br />
and bad about it. The evaluation is conducted according to a certain set of rules, such as those listed<br />
in typical guidelines documents (Nielsen 1993). According to Nielsen heuristic evaluation and user<br />
testing should be alternated, because these two usability practices have been shown to find fairly<br />
distinct sets of usability problems. Some evaluators think that heuristic evaluation is valuable as a<br />
testing method because it can easily be conducted with the assistance and equipment generally<br />
available on a college or university campus (Miller-Cochran & Rodrigo 2006). This is important when<br />
there is little or no budget put aside for user testing as many education providers do not allow for the<br />
money needed for usability testing which according to Nielson & Norman (2000) is very short sighted.<br />
We have found that heuristic evaluation is useful when we conduct testing in an ‘expert’ role where<br />
we give initial feedback to developers before bringing in any students for our studies.<br />
3. Usability and accessibility combined<br />
In our work we work together to achieve an understanding of the needs for all groups of students and<br />
to produce reports to the developers without conflicting feedback. As Kelly, Phipps and Swift (2004)<br />
say it might appear desirable to simply include usability alongside accessibility but to do so there is a<br />
need to be aware of potential conflicts due to usability preferences that conflict with accessibility<br />
guidelines. Our design of the evaluation means that such conflicts are minimised and in most cases<br />
we have no differing views.<br />
Accessibility represents an important step toward independence for individuals with disabilities and it<br />
also guarantees broader access for all users of the web (Foley 2003). His claim is that accessible web<br />
design creates pages that are often easier to read, easier to navigate and faster to download. It is<br />
acknowledged by some researchers that little or no consideration is given to the real participants in<br />
the <strong>learning</strong> experience, such as disabled learners (DeMarsico, Kimani, Mirabella et al 2006). These<br />
researchers see disabled students as a rich source of guidance on how to address accessibility<br />
issues in the development and usage of e-<strong>learning</strong> material and systems. Whereas they see usability<br />
as a separate, but related concept, and that usability and accessibility experts should work side by<br />
side. However, DeMarsico et al in their publication are not conducting this type of testing.<br />
There is a lot of consideration of the value of working together for usability and accessibility mainly<br />
because it is seen as ‘Sites that are developed to meet accessibility standards are generally, but not<br />
always more usable in their practical application. This may be attributable to the similarity between the<br />
concepts shared by both accessibility and usability, such as the inclusion of appropriately contrasting<br />
colour schemes and the exclusion of non-browser compliant elements such as JavaScript on web<br />
pages’ (Yates 2005 p182). He goes on to say that websites that conform to accessibility and usability<br />
standards ‘will support the creation of web environments which perform their function more effectively<br />
and provide an enhanced experience for the use: thus accessibility and usability standards are central<br />
to good design methodologies.’ (Yates 2005 p182). A different view is that of Byerley and Chambers<br />
(2002) who found that often accessibility does not equate to usability where adherence to the law can<br />
make the website less usable for others.<br />
Although others have worked with disabled and non-disabled users to assess the accessibility of<br />
websites these have been relatively small numbers (Harrison & Petrie 2006). Harrison and Petrie do<br />
however say that ‘Many guidelines focus specifically on either accessibility or usability rather than<br />
evaluating both simultaneously, making them less efficient.’ (p 241). This is something we endorse<br />
and is why we have been conducting these types of evaluations over a number of years and<br />
numerous websites, both educational and commercial and feel in a position to discuss our findings.<br />
Harrison and Petrie go on to say that ‘The fact that participants with disabilities find more problems<br />
reinforces the concept that they make the best test participants.’ (Harrison & Petrie, 2006 p 245).<br />
Although we acknowledge that disabled participants do make good participants, where we differ is<br />
328
Anne Jelfs and Chetz Colwell<br />
that when working together with all types of participants we realise there is a different level of<br />
engagement by them.<br />
4. Putting the study into practice - the framework<br />
Our way of working together is informed in part by the needs of the students and the conformance to<br />
standards such as WAI (2010); BS 8878 (2010); and QAA (2010). A substantial part of this work is<br />
conducted through our ‘expert’ walkthroughs and heuristic evaluations. The third part of our<br />
framework is the user input to the new developing websites. Within the framework we work together,<br />
but separately at the same points within the evaluation:<br />
1. At the wireframe stage we will work independently reviewing the proposed sites and then come<br />
together to report back to the developers. At this stage the Accessibility standards will also be<br />
considered. We will review the wireframes from different student viewpoints.<br />
2. At the ‘click through’ stage we will again work independently and report back individually to our<br />
developer colleagues.<br />
3. At the next stage which may not be a fully functioning site we will bring in students for our<br />
observational studies. This will usually be 5 disabled students and 5 non-disabled students.<br />
It could be argued that observational studies, particularly under laboratory conditions, are not the<br />
same conditions as those the student works in at home. However, to gauge some understanding of<br />
student use and relevant feedback on the web resources, we have found these types of studies<br />
invaluable in our resource development. We acknowledge their limitations, but value the insight they<br />
provide. In this section we discuss how we have attempted to alleviate as many of the problems as<br />
possible.<br />
We have found that student recruitment can often be problematic, especially when trying to identify<br />
people with specific accessibility needs for our studies. Usually students are recruited via notices on<br />
module or university based websites or through existing contacts. In conducting the studies we also<br />
have to make additional allowances, such as how far the disabled toilet facilities are and specific<br />
specialist requirements such as software e.g. JAWS and Dragon Naturally Speaking. There is also a<br />
need to recruit interpreters for deaf students who do not lip read and for one evaluator to be in the<br />
same room facing the student for those students who do lip-read in order to communicate.<br />
The way we work is that students are invited to attend one of a number of sessions for an<br />
observational study which is carried out in the Jennie Lee Research Labs in the Institute of<br />
Educational Technology (IET) at the Open University. This facility has one specifically allocated<br />
Accessibility Lab which contains additional support materials such as an adjustable desk, various<br />
keyboards, and devices such as trackballs. There are also two other Usability Labs, a Games<br />
Research lab, an observation room and reception room. The cameras capture human–computer<br />
interaction, through simultaneous ‘views’ of: the user, the computer they are working at and<br />
interactions with the application on the screen. The resulting video enables the evaluator to<br />
concentrate on the interaction as it occurs and to have a recorded observation for future analysis. A<br />
number of scenarios are set up for the students to complete during the observational study to<br />
simulate some of the activities that students would be completing in their interaction with the virtual<br />
<strong>learning</strong> environment, such as wikis, forums and assessment. The aim is to work for about 45 minutes<br />
before a break and all participants are free to leave at any point in the evaluation.<br />
At the start of the evaluation session, participants are informed of the purpose of the evaluation, about<br />
the cameras and what would happen during the session. It is explained to participants that they will be<br />
asked to think aloud when they use the website during the task completion. We also ask students to<br />
sign a consent form allowing us to record their activities and permission to use their findings in<br />
publications. We usually make a small payment plus travel expenses in acknowledgment of their<br />
participation. The use of the laboratory facilities can be daunting to participants at first so we try and<br />
create a relaxed environment and offer tea, coffee and biscuits at the start of the sessions, during any<br />
breaks and if necessary we also offer lunch.<br />
329
Anne Jelfs and Chetz Colwell<br />
We work together taking the role of observer and facilitator alternatively depending on whether the<br />
person is disabled when Chetz takes the role of facilitator, or non-disabled where Anne is the<br />
facilitator. This method almost replicates the approach suggested by Craven and Booth (2005)<br />
‘Ideally the session should include a facilitator and an observer.The facilitator sits next to<br />
the participant and directs the testing while the observer watches the process remotely<br />
either through a one-way mirror, or by using a video to record the session which the<br />
observer watches in another room. This method enables the observer to concentrate fully<br />
on what the participant is doing and saying, rather than on the actual running of the<br />
experiment.’ (pp 187-188).<br />
While participants complete the tasks, they are observed and prompted as appropriate. We use<br />
probing questions to uncover reasons for particular actions and perceptions of what the participant<br />
expects to find. Tamler (2001) suggests that a good example of a question is ‘when you were<br />
selecting that file what were you expecting’ rather than ‘why did you select that file’ which makes the<br />
user feel that they need to justify their behaviour. We also give some feedback as the student<br />
progresses because they are working as co-evaluators and not purely in isolation so we might say<br />
something like ‘this is just the sort of thing we need to hear or your information will be very useful to<br />
the developers.<br />
Notes are taken by the observer concerning critical incidents, participants’ comments and relevant<br />
actions. Participants are not given a time limit to complete each task. Finally we use a pre-prepared<br />
interview schedule at the end of the session to find out about their opinions and preferences<br />
concerning the websites and to allow them to give any other views they might have. The interview can<br />
also be seen as a de-brief where we might explain further about the work we are conducting.<br />
4. Analysis and report<br />
We do task based analysis – how users tackled the tasks and where the major difficulties arose. The<br />
observer’s notes are invaluable in guiding the analysis as one hour of video tape has been reported to<br />
take at least 5 hours of analysis, or even a day or more (Preece et al 1994, p620). From both the<br />
observer’s notes and the videos we are then able to write a joint report for the developers.<br />
5. Conclusion<br />
This method of working together has proved invaluable to the developers as it give them a report<br />
where they can apply our recommendations knowing that we have discussed our findings and we are<br />
recommending the optimum solution for all users.<br />
If points remain which are inaccessible then these are highlighted, so that statements can be made on<br />
the module website. It is imperative that disabled students are made aware of any potential areas that<br />
are inaccessible for particular types of student or supporting software.<br />
References<br />
Ardito, C., Costabile, M. De Marsico, R. Lanzilotti, S.,Levialdi, T., Roselli, V. Rossano (2006) An Approach to<br />
Usability Evaluation of e-Learning Applications.<br />
http://www.di.uniba.it/~ivu/papers/2006/UAIS2006_Arditoetal.pdf (accessed 09.06.11)<br />
BS 8878: Web accessibility. Code of Practice<br />
http://shop.bsigroup.com/en/ProductDetail/?pid=000000000030180388 (accessed 09.06.11)<br />
Byerley, S. and Chambers, M.B. (2002) Accessibility and usability of web-based library databases for non-visual<br />
users. Library Hi Tech vol 20: 2, pp169-178<br />
Colwell, C., Jelfs, A. & Mallett, E. (2005) Initial requirements of deaf students for video: lessons learned from an<br />
evaluation of a digital video application. Learning, Media and Technology, Vol. 30, No. 2, July, 201–217<br />
Cooper, M., Colwell, C. and Jelfs, A. (2007) ‘Embedding accessibility and usability: considerations for e-<strong>learning</strong><br />
research and development projects', ALT-J, Vol 15, No. 3, 231 – 245<br />
Crowther, M.S., Keller, C.C. and Waddoups G.L. (2004) Improving the quality and effectiveness of computermediated<br />
instruction through usability evaluations. British Journal of Educational Technology Vol 35: 3 pp<br />
289–303<br />
Hertzum, M. Hansen, K.D. and Andersen, H. (2009) Scrutinising usability evaluation: does thinking aloud affect<br />
behaviour and mental workload? Behaviour & Information Technology 28:2 pp 165-181<br />
HEFCE (2010) Student perspectives on technology – demand, perceptions and training needs. Report to HEFCE<br />
by NUS<br />
Hix, D. and Hartson, H.R. (1993) Developing user interfaces: Ensuring usability through product and process<br />
Wiley & sons, New York<br />
330
Anne Jelfs and Chetz Colwell<br />
Hughes & Parkes (2003) Trends in the use of verbal protocol analysis in software engineering research<br />
Behaviour and Information Technology. Vol 22: 2 pp127-140<br />
Ihamäki, H and Vilpola, I. (2004) Usability of a Virtual Learning Environment Concerning Safety at work.<br />
http://www.ejel.org (accessed 09.06.11)<br />
Jefferies, A. and Hyde, R. (2009) Listening to the Learners’ Voices in HE: how do students reflect on their use of<br />
technology for <strong>learning</strong>? Electronic Journal of e-<strong>learning</strong>, Vol 7, 2 pp119-126<br />
Joynes, G. (2000) An evaluation model for supporting higher education lecturers in the integration of new<br />
<strong>learning</strong> technologies, Educational Technology and Society, 3(4) 56–65.<br />
Kelly, B., Phipps, L. and Swift, E. (2004) Developing A Holistic Approach For E-Learning Accessibility, Canadian<br />
Journal of Learning and Technology, Vol. 30, Issue 3.<br />
Light, A. (2006) Adding method to meaning: a technique for exploring peoples’ experience with technology.<br />
Behaviour and Information Technology 25:2 pp175-187<br />
Newman, W. and Lamming, M. (1995) Interactive System Design Addison-Wesley Educational Publishers inc.<br />
Nielsen, J. (1994) Heuristic evaluation, in: J. Nielsen & R. L. Mack (eds) Usability inspection methods (New York,<br />
Wiley), pp 25–62.<br />
Preece, J. (1994) Human Computer Interaction Addison-Wesley Educational Publishers inc.<br />
Quality Assurance Agency for Higher Education (2010) Code of Practice – Section 3 Disabilities<br />
http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section3/Section3Disabilities2010.pdf<br />
(accessed 18.03.11)<br />
Sheard, J. & Markham, S. (2005) Web-based <strong>learning</strong> environments: developing a framework for evaluation,<br />
Assessment & Evaluation in Higher Education 30(4), 353–368<br />
Tamler, H. M. (2001). High-tech vs. high-touch: Some limits of automation in diagnostic usability testing. In HT<br />
Consulting, http://www.htamler.com/papers/techtouch/ (accessed 09.06.11)<br />
Turnbow, D., Kasianovitz, K. and Snyder, L. (2005) Usability testing for web redesign: a UCLA case study. OCLC<br />
Systems & Services, Vol. 21 Iss: 3, pp.226 - 234<br />
Van Schaik, P. (1999) Involving users in the specification of functionality using scenarios and model-based<br />
evaluations. Behaviour and Information Technology Vol 18, 6 pp 455-466<br />
Web Accessibility Initiative (2010) www.w3.org/WAI (accessed 18.03.11)<br />
Wright, P. C. & Monk, A. F. (1991) A cost effective evaluation method for use by designers, International Journal<br />
of Human-Computer Studies, 35, 891–912.<br />
Yates, R. (2005) ‘Web site accessibility and usability: towards more functional sites for all’ Campus-Wide<br />
Information Systems Vol 22, No. 4 pp180-188<br />
331
The Virtual Learning Environment - Directions for<br />
Development in Secondary Education<br />
John Jessel<br />
Goldsmiths, University of London, UK<br />
j.jessel@gold.ac.uk<br />
Abstract: While the use virtual <strong>learning</strong> environments (VLEs) in universities and colleges is now relatively<br />
established, this is not the case in many schools today. This paper examines the particular demands made of<br />
VLEs in secondary education in the UK and how effective and innovative use inside and outside the classroom<br />
could be promoted. Work currently being carried out with teachers in five schools as part of a two-year EUfunded<br />
project [1] is reported. Surveys to establish the perceived needs of teachers and senior managers were<br />
followed up by curriculum-led workshop sessions totalling six days with up to four teachers in each school<br />
working with another teacher who is highly conversant with VLE technology. Qualitative data gathered through<br />
field notes, observations, focus groups, recorded interviews and on-site documents were analysed in terms of the<br />
uptake and use of the technology and the quality of <strong>learning</strong> activities developed. It was found that innovation in<br />
the school setting is a potentially complex process; schools have a variety of responsibilities and demands that<br />
must be concurrently accommodated. With regard to the quality of <strong>learning</strong> activity and student engagement, a<br />
theoretical framework is introduced within which technology-use can be mapped. In relation to this and the use of<br />
VLE technology, the scope for <strong>learning</strong> as participation was explored. Principles arising from monitoring the<br />
sessions included teacher ownership of development; embedding the use of the technology to existing <strong>learning</strong><br />
objectives or teaching and <strong>learning</strong> resources; levels of personalisation; external comprehensibility in view of the<br />
availability of a VLE to a variety of users. It is also argued that development of the use of the VLE in the school<br />
context could lead to both curriculum enhancement and curriculum transformation. [1] The work carried out in the<br />
UK along with partner institutions in Spain and Italy forms part of a two-year project ‘Teaching to Teach with<br />
Technology’ funded by the EU Leonardo Da Vinci Life Long Learning Program.<br />
Keywords: VLE, <strong>learning</strong> perspectives, professional development, secondary education<br />
1. Introduction<br />
The need for the effective integration of technology in the UK school curriculum has been a recurrent<br />
concern of different governments over recent years. In a strategy paper published in 2005 the belief<br />
was stated that future development of ICT in education could ‘transform teaching and <strong>learning</strong> and<br />
help improve outcomes for children and young people, through shared ideas, more exciting lessons<br />
and online help for professionals’ (DfES, 2005: 4). Although there has since been a change of<br />
government and the current Schools White Paper (DfE, 2010) does not directly specify a role for<br />
technology, the pervasiveness and availability of technology made available through initiatives<br />
following the 2005 Strategy raises key questions about the part technology can continue to play.<br />
As part of an expressed need for a common digital infrastructure to support transformation and<br />
reform, the 2005 Strategy encouraged the use of virtual <strong>learning</strong> environments (VLEs). The<br />
importance of VLEs arises from their connectivity; as computer-based systems they offer facilities<br />
such as e-mail, bulletin boards and newsgroups together with ways of storing and presenting course<br />
materials. In many respects, and in contrast to the internet of the last decade, they allow exchanges<br />
that are many to many or ‘interactive’ rather the than one-to-many characteristic of a ‘broadcast’<br />
(Selwyn, 2008). VLEs are typically closed to an institution and in that sense regarded as relatively<br />
secure and different rights can be given to users such as pupils, teachers and more remote potential<br />
users such as parents or carers. Well-known systems in the UK university sector are Blackboard and<br />
Moodle while systems such as Frog, Fronter and SIMS Learning Gateway are available in many<br />
schools. Although there are differences between each system most have the facilities noted above<br />
and for the purposes of this article VLEs will be treated generically.<br />
Much current use of VLEs in schools is administrative, with systems for recording and accessing data<br />
on student attainment. VLEs typically allow hand-in and storage of assignments for marking and<br />
feedback. Teachers can devise online tasks that are often marked automatically. While these are<br />
important aspects of use, those more central to <strong>learning</strong> are relatively unexplored.<br />
Despite major investment, the use of VLEs in schools is relatively patchy; they may work effectively in<br />
one setting but this does not guarantee they will work well in another. While the use of VLEs in<br />
universities and colleges is relatively established (UCISA, 2008; 2010), this is not the case in many<br />
332
John Jessel<br />
schools today (Ofsted, 2009). The current work explores the issues at a time when VLEs are<br />
expected to play a major role in <strong>learning</strong> and when ways of realising the potential for <strong>learning</strong> through<br />
use of VLEs in schools are relatively unexplored. The focus in this paper is on the development of<br />
resources and methods at a pre-trial stage by teachers in five London secondary schools. It considers<br />
some possibilities VLE technologies can offer and reports on ideas developed by teachers and some<br />
issues involved in the take-up of innovation.<br />
2. Perspectives on <strong>learning</strong><br />
Regardless of innovation, a central concern within education is the potential technology offers for<br />
<strong>learning</strong>. The relationship between technology and theoretical perspectives on <strong>learning</strong> is one that is<br />
continually under review, particularly as new technologies emerge. Over the last five decades ideas<br />
linking <strong>learning</strong> to technology have ranged from an associationist perspective where <strong>learning</strong> is<br />
viewed in terms of observable behaviours that could be shaped by progressive reinforcement through<br />
a tightly defined sequence of objectives that could be presented to the learner. A <strong>learning</strong> programme<br />
could be administered with the aid of a teaching machine where elements of information could be<br />
presented, questions asked and immediate feedback to responses given. From this perspective the<br />
brain was a ‘black box’; other than the effects of reinforcement what went on inside was of no interest.<br />
In contrast to viewing the mind as a black box and focusing on the programming of observable<br />
behaviours, a cognitive perspective is concerned with inner mental activities. The focus is on<br />
processes such as thinking and reasoning and representation in memory. According to constructivist<br />
theory <strong>learning</strong> is an active process. We form mental representations that are based upon our<br />
experience of the world around us and form the basis of our knowledge or understandings.<br />
Constructivist pedagogy is based on creating an environment in which learners can interact and<br />
providing activities that encourage experimentation and discovery (Jonassen, 1994). The use of<br />
technology in accordance with a constructivist framework would include the provision an environment<br />
where <strong>learning</strong> can take place through experimentation and interaction, as might be achieved with the<br />
use of computer simulations or the control of a screen ‘turtle’ using a language such as Logo.<br />
More recently in educational theory there has been a shift of focus towards work that highlights<br />
<strong>learning</strong> as a social activity rather than a cognitive and individual one (Crook, 2001). An important<br />
contribution is Vygotsky’s (1978) emphasis on the social context of <strong>learning</strong>. In a ‘social constructivist’<br />
approach learners can be encouraged by those more experienced in an area to reflect on and<br />
articulate the principles involved in activities in which they are engaged. The role of language is<br />
regarded as important and through such ‘scaffolding’ the idea is that learners would be able to<br />
achieve understandings that they would not have done if unsupported in this way.<br />
Although there are different views on what counts as social constructivism, a general feature is that<br />
meaningful knowledge, as opposed to something that already exists and waiting to be passed on, is<br />
socially and culturally constructed: meanings are created through human interaction and with the<br />
environment (Kukla, 2000). However, knowledge can also be seen to emerge as it is ‘constructed and<br />
reconstructed between participants in specific situated activities, using the cultural artefacts at their<br />
disposal, as they work towards the collaborative achievement of a goal’ (Wells, 1999: 140). This has<br />
been made explicit through Bakhtin’s (1981) notion of dialogism where interaction is viewed on a<br />
more symmetrical basis.<br />
The place of dialogue within <strong>learning</strong> is also consistent with the concept of a community of practice<br />
(Lave and Wenger, 1991; Wenger, 1998). A community of practice occurs when people jointly engage<br />
socially in a common enterprise or activity, and through this learn together. Rather than being an<br />
individual pursuit or academic exercise largely confined to the classroom, <strong>learning</strong> is ‘situated’ within<br />
the framework provided by the community of practice through which participation can take place.<br />
Here, participation is central to <strong>learning</strong>; this may be relatively peripheral at first but through<br />
observation and practice learners take on the identity of the community and become more central<br />
(Wenger 1998). This more ‘situated’ form of <strong>learning</strong> can, of course, occur without the aid of digital<br />
technology. However, with the emergence of VLEs or Web 2.0 and there is further scope for social<br />
participation.<br />
333
John Jessel<br />
2.1 An outline framework for <strong>learning</strong> and technology-use<br />
A framework that summarises the main <strong>learning</strong> perspectives is shown in Figure 1. It was designed to<br />
provide a convenient reference and basis for conceptualising, articulating and developing a discourse<br />
amongst those interested in promoting effective <strong>learning</strong> using VLE technologies.<br />
Figure 1: A framework for mapping technology-use according to <strong>learning</strong> perspective<br />
Rather than regarding different technologies as having intrinsic properties in relation to the kind of<br />
<strong>learning</strong> they may support the framework places emphasis on how a technology is used. Digital<br />
technologies can be mapped into the space delineated by the two axes, but only according to how<br />
they are used. In other words, according to use, a given technology could take up one of many<br />
different positions, or spread over a more inclusive area.<br />
Learning paradigms arranged along the horizontal axis begin with those aligned with an associative<br />
perspective. The remainder of the axis, however, is representative of a cognitive or constructivist<br />
perspective and acknowledges an increasing level of initiative and creativity on behalf of the learner.<br />
Although the kinds of <strong>learning</strong> are reminiscent to those identified by Bloom (1956) the perspective is<br />
primarily cognitive. The social dimension is marked out on a vertical axis. In its most basic form, social<br />
involvement in <strong>learning</strong> is characterised by a relatively didactic transmission of information as shown<br />
at the lowest point of the axis. Moving up the axis is the kind of interaction characteristic of scaffolding<br />
within a social constructivist approach where a teacher or more experienced peer engages within an<br />
activity so that learners are helped to develop their own understandings. Further up the axis social<br />
engagement becomes more dialogic and participatory in nature. Participatory approaches can entail<br />
an authentic quality (Brown et al., 1989). This has not been represented on the axes shown and could<br />
be represented on a third orthogonal axis.<br />
2.2 Encouraging participation through digital technologies<br />
Although digital technologies could support <strong>learning</strong> in the ways outlined it does not automatically<br />
follow that this will occur. While many adult learners may be self-motivated and work independently,<br />
this is not always the case with younger learners and to some extent might account for the noted<br />
334
John Jessel<br />
differences in VLE use in the school and higher and further education sectors. An important factor<br />
could be the role of the learner. For example, to what extent do new technologies allow learners to be<br />
self-directed, to be a resource for their own <strong>learning</strong>, to learn from peer feedback and apply their<br />
<strong>learning</strong> to real-world problems? These andragogic principles (Knowles, 1977) have hitherto been<br />
primarily associated with adult learners but could be reviewed in the context of digital technologies. A<br />
further issue, then, is what strategies could lead to the effective use of technology in <strong>learning</strong> by<br />
younger learners and how can these be established in the school environment?<br />
3. Method<br />
Participants were recruited through professional contact in schools that work in initial teacher training<br />
partnership with the university department coordinating the UK part of the Project. Altogether five<br />
London inner-city secondary schools with pupils from a range of backgrounds became involved.<br />
Three core areas of the UK National Curriculum, namely mathematics, science and English were<br />
focused on. In addition, Personal, Social and Health Education (PSHE), now a compulsory curriculum<br />
component, was explored by one of the schools.<br />
The teachers who participated were subject specialists in English, mathematics and science and were<br />
either in their early or mid career. Senior management staff from the participating schools were also<br />
interviewed. The workshops were run on site with up to four teachers in each school working with one<br />
of the two teachers from other educational institutions who were highly conversant with VLE<br />
technology.<br />
A number of principles underlay the workshop design. Firstly, work was curriculum led: any innovative<br />
methods would sub-serve existing curriculum objectives, the intention being to include the use of the<br />
technology as part of mainstream teaching activity. For similar reasons this remit included firm links<br />
with assessment. The decision to hold workshops in situ in terms of subject department was to<br />
facilitate the development of the use of technology in a way that is less likely to be isolated from the<br />
curriculum or other <strong>learning</strong> resources. Another important element of the workshop provision was to<br />
encourage use of the VLE that promoted <strong>learning</strong> through social participation as well as cognitive<br />
interaction. In addition to supporting any needed discrete technology skills, the idea was to allow the<br />
teachers to draw on their own expertise as subject specialists and use the technology as creators in<br />
the development of curriculum-specific methods and resources.<br />
The theoretical perspectives outlined above were introduced briefly with the aid of the diagram similar<br />
to that shown in Figure 1. This marked out the broader <strong>learning</strong> possibilities and understandings of<br />
the development of technology systems and associated pedagogy. Identification of <strong>learning</strong> goals and<br />
assessment opportunities were also included in the sessions as were considerations of the role of the<br />
student, the teacher and the technology. Field trials prototyping resources and methods with a subset<br />
of teachers and students were planned in conjunction with the workshops and intra- and interinstitutional<br />
dissemination and methods of sustainability discussed.<br />
Qualitative data gathered through field notes, observations, focus groups, audio recordings of<br />
workshop sessions and recorded interviews and on-site documents were analysed in terms of the<br />
uptake and sustained use of the technology and the quality of <strong>learning</strong> activities developed.<br />
4. Findings<br />
4.1 Innovation in the school setting<br />
With regard to recruitment of participating schools and teachers, it was found that introducing<br />
innovation in the school setting is a potentially complex process; schools have a variety of<br />
responsibilities and demands that must be concurrently accommodated. Points of entry for innovative<br />
practice that were voiced by senior personnel and teachers ranged from those driven by national or<br />
local policy, by impending crises, pressing social or pastoral needs and changes of staff. In other<br />
words a programme of development does not occur in isolation and often needs to be timed in<br />
relation to other demands, circumstances or initiatives; it needs to be negotiated and flexible. Even<br />
with assured financial resources needed for cover, flexibility was needed in arranging sessions in view<br />
of unforeseen demands that were not unusual in the schools concerned.<br />
An initial small-scale survey was conducted to get an indication in-class and out-of-class use of<br />
different ICT facilities and the perceived needs of teachers. With regard to the VLE, from the 20<br />
335
John Jessel<br />
teachers who agreed to participate, reported out-of-class rates of usage were proportionately higher<br />
(10 using the VLE at least once or twice a week) than reported in-class rates (only 4 using the VLE at<br />
least once or twice a week), interview data suggesting that out-of-class use was largely for<br />
administrative purposes. Although there was a lower rate of in-class usage (16 using the VLE less<br />
than once or twice a month which included 10 never having used it) this did not appear to be because<br />
of lack of interest; responses to a further questionnaire revealed that the technology was regarded as<br />
innovative and that all of the teachers surveyed were interested in finding out more about its use.<br />
4.2 Workshop sessions<br />
Applications of VLE technologies in teaching mathematics, science, English and PSHE were<br />
explored. With mathematics, opportunities for students to develop problem-solving skills were<br />
addressed through the online dimension taking account of a need for inner city students to develop<br />
their extended writing skills. The idea was to use the technology to create project-based lessons that<br />
fitted within both the GCSE and KS5 Curriculum. Work with science teachers focused on developing<br />
resources and a structure for the VLE with a focus on BTEC Science. In relation to the PSHE, the<br />
focus was on a core of underachieving girls and trying to use the technology available to engage both<br />
pupils and parents to increase engagement and communication. The aim was to use the VLE to<br />
communicate with these pupils in a less formal way in an attempt to remove some of the barriers to<br />
their <strong>learning</strong>. It will also be used to engage with and help parents, many of whom do not have good<br />
memories from their own educational experience. The idea was to set up a chat room for engagement<br />
between students. Older Y10/Y11 girls would be encouraged to take a mentoring role for younger<br />
girls. The possibility was that a forum would be a natural place to start for asking for help. Finding<br />
ways of getting girls to support each other was regarded as crucial. However virtual peer mentoring<br />
was not seen as the sole means of communication; ‘real-world physical mentoring would tie up with<br />
virtual mentoring’. It was thought that a sense of community could be developed through building links<br />
between the year groups.<br />
Two schools were involved in developing the use of the VLE in the English curriculum. The<br />
technology also included handheld audio and video devices within the Level 1 Functional English<br />
Qualification, Reading, Speaking and Listening, with Year 8 (12-13 year old) students. The work,<br />
done in relation to an existing scheme of work for a BBC School Report initiative giving students a<br />
chance to make news reports for a real audience, will now be described in more detail.<br />
The aim was for pupils to use radio or television for their reports. In addition to developing an<br />
understanding of the media they were to use, attention was given to what constituted a news story,<br />
how to go about finding a story and how stories could be put together to form a final sequence. This<br />
would involve editorial meetings and use of the VLE for an online discussion forum.<br />
As with the other schools, the technology-use framework was introduced at the start of the workshop<br />
sessions. This was found to be largely self-explanatory and it was apparent where the different modes<br />
of use of the VLE could be mapped or located within the social and cognitive space. The framework<br />
also had a discourse-setting effect and allowed discussion to move beyond the more prevalent use of<br />
the VLE as a dissemination instrument for tests and factual material. In particular, with regard to the<br />
quality of <strong>learning</strong> activity and student engagement, the framework encouraged account to be taken of<br />
types of <strong>learning</strong> paradigm and the social and collaborative dimension afforded by the connectivity<br />
available through the VLE. However, the move towards the more participatory approach to <strong>learning</strong><br />
did not occur immediately.<br />
In common with the majority of London schools, the VLE used was Fronter. In response to being<br />
asked about his initial understanding of Fronter one teacher replied ‘I think it is a virtual <strong>learning</strong><br />
environment. It’s modelled like a virtual school where there are different rooms and corridors and<br />
pupils can go into rooms and teachers will be in certain rooms. Just like a classroom, there are<br />
resources in there.’ At this stage the potential for <strong>learning</strong> was in line the more prevalent uses of VLEs<br />
mentioned earlier: ‘I think Fronter is a really good place to store resources so any child who’s got<br />
access to a computer at home can carry on with their work’ ‘Children are allowed to upload work that<br />
they’ve done so they can hand in work for the teacher to mark.’ ‘You can set work, upload resources<br />
and set tasks in Fronter.’<br />
In regard to the potential of the VLE for exploring participation in <strong>learning</strong> in the development of a<br />
news story, the question was asked ‘Why bother with Fronter?’ Although the scope for participation<br />
336
John Jessel<br />
was not immediately apparent, with continued discussion and working through ideas the<br />
communicative and participative potential began to emerge: ‘Just like a real classroom students and<br />
teachers interact in that room like they would in a real one’. On further discussion particular qualities<br />
of the virtual room were explored: ‘If you build into a scheme of work the idea that sometimes the<br />
teams are discussing live verbally, sometimes they are discussing through an internet forum and<br />
there is some kind of opportunity for them to evaluate and compare the differences in the way they<br />
use language within those two different media’. This led to a teaching strategy that could be explored<br />
with pupils in classroom equipped with computer terminals: ‘It’s a simple rule that’s established; when<br />
you discuss online you don’t speak to each other.’<br />
Later on the discussion had moved on to address aspects of pupils collaboration: ‘This is where the<br />
Fronter thing could come in as a sharing forum … so you could have two areas with Fronter. You<br />
could have, like, a forum discussion room where you generally discuss within your team, ideas, and<br />
you could have like a public forum where you could put the whole class to share. So you could have a<br />
shared forum and a team forum, and they’d have to also make decisions about what is appropriate for<br />
each forum and why some discussions would stay within the team and some things go beyond the<br />
team and can be shared with others, and what would be really nice is if they had to make those<br />
decisions themselves about which forum would be appropriate. This is where I begin to see where<br />
Fronter is a really useful way to do it.’<br />
Through having the opportunity to discuss and explore with minimal prompting from the technologyuse<br />
framework the vision had moved from ‘Why are we bothering to use Fronter?’ to ‘being able to<br />
develop a really interactive use of Fronter’.<br />
4.3 General principles concerning the use of VLE technologies<br />
Analysis of the observational data revealed a number of general principles that emerged across all of<br />
five schools that appeared to play a key role in the development of resources and methods of working<br />
with the VLE and associated technologies. These are now summarised.<br />
Embedding - This led on from the principle that the work should be curriculum led. Through<br />
negotiation, existing areas of the curriculum within which to new practices involving technology could<br />
be identified. This ‘linking-in’ to existing <strong>learning</strong> objectives also extended to existing teaching<br />
materials or <strong>learning</strong> resources used within the departments. Embedding was not only in terms of<br />
content but also in terms of structure and style of presentation.<br />
Student engagement - This was expressed in terms such as ‘kids have to take a position and have to<br />
argue’. This emerged in relation to the news editing forums mentioned above. It was also recognised<br />
that this is seldom a straightforward process and considerable discussion centred around topics such<br />
as encouraging pupils to analyse video material rather than simply access it.<br />
Enrichment - It was recognised that easy access to a range of resources in different modalities<br />
through the VLE allowed reading around the topic. Alerts to resources elsewhere such as TV<br />
documentaries to enrich were also built in. Blogs were regarded as enriching and encouraged reading<br />
around a topic.<br />
Time and location specific - The VLEs allowed small items of information such as blogs or newsfeeds<br />
to be connected to each page and could be changed when the page was revisited. The need was<br />
expressed to keep these items up to date and to make them time location specific (e.g., ‘The last<br />
lunchtime club this week is on Thursday - don’t miss it!’) Levels of personalisation were regarded as<br />
important and suggestions for these varied in terms of the institution, the learner group and the<br />
individual. ‘If you have a news feed then make sure it is something you put in – not just a link to any<br />
website so that pupils see it as something specifically for them.’<br />
Externally comprehensible - The availability of access to the VLE by a variety of users means a selfexplanatory<br />
context has to be provided because it may be used in different contexts within the school<br />
and outside the school by, for example, pupils, teachers, senior management, parents and carers.<br />
Ease of use / ergonomics - Many suggestions for this arose. These included avoiding ‘front-end’<br />
writing; ‘teachers and pupils do not want to read it, just want to get on with the work’. Another<br />
suggestion was to make as few clicks as possible for students to get to where they are working:<br />
337
John Jessel<br />
‘“reduce the clicks” is my motto’. Uniformity of pathways was considered as important, colour coding<br />
often helping. When designing pages it was felt that ‘it has to be obvious to the pupils as to how it<br />
should work’ and ‘scrolling horizontally is a no no’, vertical scrolling being more manageable.<br />
Reliability - Rather than this being a matter of the technology breaking down, the concern was with<br />
planning methods of working in sufficient detail so that <strong>learning</strong> outcomes can be achieved in a<br />
dependable way. Where the methods are innovative and carry a higher level of risk, backup strategies<br />
were included for achieving the objectives.<br />
Inter-institutional comparison - Because the trainers worked across schools that were known to each<br />
other there was interest in how other institutions were approaching particular curriculum areas and<br />
solutions to common problems were shared. In one instance a teacher from one school visited the<br />
school where the trainer taught to see particular methods in action. Although this could have been a<br />
recipe for uniformity, it was felt that one institution was never a blueprint of what happened in another<br />
and so this was regarded as a form of enrichment.<br />
Ownership - The need for teachers to take ownership and negotiate and augment aims to serve their<br />
interests was considered essential at many different levels: ‘rather like a lesson plan innovation is<br />
developed within an institution’ (teacher); [Innovation is] ‘something you develop yourself and have<br />
control over rather than have it thrust upon you’.<br />
5. Summary and conclusion<br />
Although the VLE in secondary education may have the potential to enhance <strong>learning</strong>, effective<br />
uptake may not occur automatically. One reason for this is that VLE technologies make innovative<br />
demands on practice – both by learners as well as teachers. Innovation within the school setting is<br />
something that does not occur in isolation and has to be considered along with other initiatives,<br />
changes and demands occurring within that context. Innovation has its costs: it became clear that<br />
time was needed to identify and adapt suitable resource, to structure content and develop new ways<br />
of working This can also involve the kind of detail expressed in the form of the general principle that<br />
emerged from the workshops. Much of the work associated with development appears ‘invisible’ in<br />
that it is done in advance of the use of the technology. The use of the VLE has implications for the<br />
role of the teacher and the learner. The work highlighted the fact that the teacher was not solely<br />
instrumental in passing on an established body of knowledge. Teachers acted as facilitators bringing<br />
together curriculum, teaching and technical experience in a jointly creative enterprise. It is also likely<br />
that a certain amount of inertia can be involved in the adoption of new technologies as evidenced by<br />
the move away from assumptions of the VLE as a storage and administrative system towards a more<br />
participative <strong>learning</strong> medium.<br />
Insofar that a curriculum can be viewed in not only in terms of <strong>learning</strong> outcomes but also in terms of<br />
the means and wider <strong>learning</strong> context there are implications regarding the impact of VLE<br />
technologies. The work connected with writing news reports illustrates, for example, how, on the one<br />
hand, technology can be used to enhance existing curriculum objectives, while, on the other,<br />
technology can be used to transform an existing curriculum and introduce new challenges.<br />
References<br />
Bakhtin, M.M. (1981) The Dialogic Imagination: Four Essays by M.M. Bahktin, Ed. M. Holquist, Austin, University<br />
of Texas Press.<br />
Bloom, B.S. (1956) Taxonomy of Educational Objectives, Handbook 1: The Cognitive Domain, New York, David<br />
McKay Co. Inc.<br />
Brown, J.S., Collins, A., and Duguid, P. (1989) “Situated Cognition and the Culture of Learning, Educational<br />
Researcher, Vol. 18, pp 32-42.<br />
Crook, C. (2001) “The Social Character of Knowing and Learning: Implications of Cultural Psychology for<br />
Educational Technology”, Technology, Pedagogy and Education, Vol. 10, No. 1, pp 19-36.<br />
DfES (2005) “Harnessing Technology – Transforming Learning and Children’s Services”, [online], Department for<br />
Education and Skills, www.dscf.gov.uk/publications/e-strategy.<br />
DfE (2010) “The Importance of Teaching: Schools White Paper”, [online], Department for Education,<br />
www.education.gov.uk/b0068570/the-importance-of-teaching/.<br />
Jonassen, D.H. (1994) “Thinking Technology: Toward a Constructivist Design Model”, Educational Technology,<br />
Vol. 34, No. 4, April, pp 34-37.<br />
Knowles, M. S. (1977) The Modern Practice of Adult Education: Andragogy Versus Pedagogy (Eighth ed.), New<br />
York, Association Press.<br />
338
John Jessel<br />
Kukla, A. (2000) Social Constructivism and the Philosophy of Science, London, Routledge.<br />
Lave, J. and Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge, Cambridge<br />
University Press.<br />
Ofsted (2009) “Virtual Learning Environments: An Evaluation of their Development in a Sample of Educational<br />
Settings”, [online], Ofsted, www.ofsted.gov.uk.<br />
Selwyn, N. (2008) “Education 2.0? Designing the Web for Teaching and Learning”, Teaching and Learning<br />
Research Program, London: Institute of Education, [online], www.tlrp.org.tel.<br />
UCISA (2008) “2008 Survey of Technology Enhanced Learning in Higher Education in the UK”, [online],<br />
Universities and Colleges Information Systems Association, www.ucisa.ac.uk/publications/tel_survey.aspx.<br />
UCISA, (2010) “2010 Survey of Technology Enhanced Learning in Higher Education in the UK”, [online],<br />
Universities and Colleges Information Systems Association, www.ucisa.ac.uk/groups/ssg/surveys.aspx.<br />
Vygotsky, L. (1978) Mind in Society, Cambridge MA, Harvard University Press.<br />
Wells, G. (1999) “Language and Education: Reconceptualising Education as Dialogue”, Annual Review of<br />
Applied Linguistics, Vol. 19, pp 135-155.<br />
Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity, Cambridge, Cambridge University<br />
Press.<br />
339
Mutlimodal Teaching Through ICT Education: An e-<br />
Twinning Program as a Case Study of Intercultural<br />
Exchange<br />
Paraskevi Kanari and Georgios Potamias<br />
National Kapodistrian University of Athens, Greece<br />
viviandeesse@yahoo.fr<br />
giogeo_gr@yahoo.co.uk<br />
Abstract: Multimodality is a key factor in promoting knowledge through multidimensional aspects in an approach,<br />
determined by social environment. In the approach presented, school is a part of an accepted social context. This<br />
paper shows that development of critical visual literacy via ICT education and e-<strong>learning</strong> combined with<br />
conventional activities can be an innovative multidisciplinary approach for the development of pedagogical<br />
projects. The cultural program in particular presented, which was carried out in the island of Spetses in Greece,<br />
was integrated in this approach. The ultimate goal was to create a film transfer of a local legend by the students<br />
of Spetses’ junior high an area culturally challenged. The program was integrated to an e-twinning project (e<strong>learning</strong><br />
European exchange program) concerning revitalizing local legends through e-<strong>learning</strong>, filmmaking,<br />
teaching local History through French language and intercultural exchange with the cooperation of two European<br />
schools, one in the Greek island of Spetses and one in Chorwovie in Poland. Through this e-<strong>learning</strong> project the<br />
students carried out a different multimodal approach of French Language and Local Culture achieving the<br />
development of their creativity and imagination through cultural e-interactivity and their cognitive and social<br />
awareness as well.<br />
Keywords: multimodal approach, social semiotics, e-twinning, cinema, local history, public school<br />
1. Multimodality and cinematographic education<br />
In a time of intense semiotic, visual stimulus (Cocula & Peyroutet 1999: 10) (medias, computer,<br />
cinema) an educational system that stays attached to the “ex cathedra”(traditional frontal) method of<br />
teaching is impossible to correspond to the interests and needs of modern day students and it is<br />
inevitably boring in comparison to their every day <strong>learning</strong> experiences. The case of critical visual<br />
literacy which is defined as the ability of understanding, decoding, producing and criticizing visual<br />
messages is in our times more indispensable than ever (Kress& Van Leeuwen 1996:3; Pleios<br />
2005:238). According to Sankey (2005: 2) a widespread development of critical visual literacy is<br />
necessary in order to cultivate and enhance the ability of reading and semiologically decoding both a<br />
text and an image as a whole or separately. Schirato & Yell (1996: 209) add that in western societies,<br />
a literate person will be considered primary one that can identify, read, analyze and develop a wide<br />
variety of visual signs and interactive stimulus. Researches on imagery and its semiotic sustainability<br />
(Levie & Lentz 1982) prove that images can largely promote teaching and social based knowledge.<br />
The most important advantages of teaching through the use of images are:<br />
The images are stocked easily in long term memory, creating information as connecting parts or<br />
meanings in human memory.<br />
Images provoke a wide range of reactions like the rise and wakening of imagination. They create<br />
connections; they promote and cultivate creative thinking and abstract imagination.<br />
The readers or users of teaching material prefer illustrative material for its higher information<br />
quality.<br />
The combination of imagery and text is more effective in promoting comprehension than the text<br />
alone.<br />
The images can be useful in understanding and <strong>learning</strong> abstract meanings that can be hardly<br />
expressed with words.<br />
Images can help mentally or physically challenged students in <strong>learning</strong> efficiently.<br />
The teaching use of images is proved more effective when teaching is connected with the flexible<br />
and combined use of internet; computer enhanced virtual environments and cinema (Kress 2005:<br />
11-12).<br />
The role of the teacher is to create an educational and pedagogical environment that could encourage<br />
students’ critical thinking towards imagery (images of whatever status or quality) as the increasing<br />
number of visual stimulus make students confuse reality with the virtual reality of computer or visual<br />
340
Paraskevi Kanari and Georgios Potamias<br />
empowered artificial environments (Joly 2005:447; Smyrneos 2008:214). Therefore, the development<br />
of critical visual literacy is imperative for modern students and generally citizens.<br />
The teaching via cinema, the cinematographic education, is strongly attached to multi-modal<br />
education (Jacquinot 1977:27; Tardy 1973:27). Cinema and moving pictures, more traditional semiotic<br />
means of education than modern computer empowered visualizations, can be the source of multiple<br />
teaching activities with parallel analysis of images, text and sound. The ultimate goal in this approach<br />
is the promotion of multiple communicative and sensory interactions between students (Régis 2009:<br />
9).<br />
The cinematographic education is necessary for the understanding of modern multimodal media<br />
world. The multimodal cinematographic teaching, based on image, promotes the ability of:<br />
Using an inexhaustible source of visual material.<br />
Reading, criticizing and decoding the multiple diversity, function and temporality of visual material.<br />
Using rationally electronic equipment (computer, video, cinema) and visual means for gathering<br />
and managing critical information.<br />
Using multimedia applications in order to create multimodal contexts and interactive compositions<br />
(combination of image- text- sound).<br />
Promoting and presenting cinematographic creations in an informative sustainable way<br />
(Yakoumatou 2006:74-75; Semoglou 2005:18; The New London Group 1996).<br />
Despite the objections towards developing powerful and sophisticated criteria that can be theoretically<br />
able to promote students’ understanding of visual communication (Barthes 1977:88; Baudinet –<br />
Mondzain 1990:15), the need for promoting multimodal interactive teaching is urgent in modern day<br />
school.<br />
An effort for the creation of a school curriculum that can empower the management of critical<br />
information has been carried out in many countries. Such curriculums theoretically consider very<br />
important the cultivation of metacognitive skills, the ability of managing critically important knowledge<br />
(Greek Institute of Pedagogy, 1998). Since the development of the internet has given students access<br />
to an inexhaustible source of visual information, with the risk of confusion between valid knowledge<br />
and useless information, it is imperative to encourage the integration of a more visually aesthetic<br />
critical education in a modern curriculum (Barnett 2000:22; Gonnet 1997:17).<br />
Although it would be an exaggeration to assert the primacy of visual literacy, it is generally accepted<br />
that many students accept willingly the integration of visual material, in order to support and enrich<br />
textual and non-textual material in a holistically empowered semiotic context (Semoglou 2005:564-<br />
565).<br />
Very recently, the introduction of interactive blackboards, and generally rich visual educational means<br />
(dvd, internet videos, use of youtube or teachtube videos in the classroom) has an innovative and<br />
revolutionary impact in the heart of modern school ethos. The students are excited from the<br />
immediacy, audacity and relevance of illustrations and visual reproductions of all forms. However, the<br />
teachers should be very cautious and concerned towards not only the semiotic informative quality and<br />
pedagogical content of the imagery used but also towards their ability to use, manipulate and<br />
successfully manage the inexhaustible source of visual information (Sankey 2005:10; Van Leeuwen &<br />
Jewitt 2001:107).<br />
2. Application of the theory in an educational context<br />
In the context of multimodal cinematographic education, a cultural program was carried out in the<br />
island of Spetses in Greece during the school year 2007-2008. The ultimate goal of this program was<br />
to create a film transfer of a local legend by the students of Spetses’ junior high (students from 12 to<br />
15 years old), an area culturally and educationally challenged. Spetses is a small island of 5000<br />
inhabitants with long and rich history but with poor cultural activity, due to its isolation, especially in<br />
winter time. The authors took part in the design, the implementation and the evaluation of this<br />
program, mentoring students and recording every step of its development through the e-platforms of<br />
the e-twining program. The results of the program were presented virtually in the e-twinning e-platform<br />
and in a cultural event hosted by the municipal authorities of the town.<br />
341
Paraskevi Kanari and Georgios Potamias<br />
In the case study presented in this paper, multimodality is presented as an educational means of<br />
holistically enhancing intercultural and social awareness in school. Thus multimodality in praxis affects<br />
positively not only educational competencies of students but also the social context as a whole.<br />
The program was integrated to an e-twinning project (e-<strong>learning</strong> European exchange program)<br />
concerning the revival of local legends through e-<strong>learning</strong>, ICT education, filmmaking, teaching local<br />
History and intercultural exchange. In this case two European schools cooperated, one in the Greek<br />
island of Spetses and one in Chorwovie of Poland.<br />
The purpose of the program was discovering and revitalizing local legends of each area and<br />
exchanging creatively this cultural experience with the other school. The program was divided in three<br />
parts. In the first part, each school presented itself by writing and uploading an e-twinning portrait of<br />
their school and of themselves. In the second part the students tried to discover various local legends<br />
by asking their parents or the elders of their community. After selecting amongst the amount of<br />
legends discovered, each school presented the first part (the beginning) of one or more legends and<br />
the other school tried to imagine the end of the story. Written activities in French concerning any given<br />
legend were given to students (description of the main character, developing language enhancing<br />
activities such as crosswords with the new vocabulary etc.). The Greek legend turned into a film was<br />
entitled in French language: “La vieille dame et le paon” (“the old lady and the peacock). In the third<br />
part, the end of the story was given to the students. The presentation of schools, the comments and<br />
the e-communication between students was in French. The Polish school chose to present the<br />
legends through the use of Microsoft’s power point programme. The Greek school chose to present its<br />
legend through a short film.<br />
Through this e-<strong>learning</strong> project the students carried out a different multimodal approach of Culture<br />
and History achieving not only the development of their creativity and imagination through cultural einteractivity,<br />
but their cognitive and social awareness as well. E-<strong>learning</strong> and visual enhancing tools<br />
promoted the breaking of social and educational barriers through intercultural activities in two<br />
culturally challenged European areas<br />
2.1 General aims of the cultural e-twinning program<br />
Approaching visual literacy through cinema and its techniques<br />
The production of a short film.<br />
Opening school towards other culture and mentalities (Polish legends)<br />
Opening school towards local society.<br />
Developing of communicative skills in a common language, as “lingua franca”, the French<br />
language.<br />
Considering a foreign language (French) as an empowering tool of communication and cultural<br />
exchange.<br />
2.2 Specific purposes<br />
Getting to know Greek and French cinema.<br />
Sharpening the creativity and imagination of students with exercises and games (role playing<br />
games, brainstorming and creativity games).<br />
Exercising creative writing through writing a scenario.<br />
Finding and cultivating students’ talents and developing responsibility towards class and<br />
community.<br />
Creating a short film as a final product of the project<br />
Learning a foreign language (French) through multimodal approach.<br />
Approaching two different cultures and environments (an old miners’ region in Poland and a<br />
summer resort in Greece).<br />
342
2.3 Results<br />
Paraskevi Kanari and Georgios Potamias<br />
Students got to know unknown areas of cinema and developed their critical thinking towards<br />
cinematographic art.<br />
Students got to know legends and fairy tales of their native place.<br />
Development of the creativity and imagination of students and teachers as well.<br />
Constructive use of students’ free time and creation of new educative stimulus.<br />
Development of students’ responsibility with the assignment of specific tasks and keeping a<br />
specific time plan.<br />
Improvement of students’ communicative skills in French. Using a foreign language outside<br />
classroom, in order to fulfill communicative needs was a realistic goal.<br />
Approach of a different culture and mentality.<br />
Creation of a short film based on a local legend.<br />
Presentation of the film in local community as a meta-result of the project in the end of the school<br />
year.<br />
2.4 Final evaluation<br />
The program promoted the integration of social and cognitive challenged students in two different<br />
countries: Socially isolated students had an opportunity to work in group activities and carry out more<br />
interesting and stress-free tasks (isolation due to personal, family or even linguistic problems, e.g.<br />
children of immigrants with inadequate literacy).Through this intercultural pedagogical experience,<br />
students became more “mature” by increasing their social skills. The school, a place of academic<br />
knowledge, usually detached from social context, became an integrated part of society. The research<br />
of local history and mythology led to an active acquisition of knowledge based on the given social<br />
context. Finally through film making, students and educators revaluated their cultural inheritance by<br />
discovering the best sites and traditional abandoned constructions of their island for shooting the film,<br />
in order to choose places for the scenes of the film (old houses turned into museums, old<br />
coffeehouses). E-<strong>learning</strong> became a social power tool that provoked educational awareness and<br />
cultural interchange. The case study presented in this paper sets an example of how modern<br />
innovative multimodal educational approaches, sponsored by European intercultural programmes and<br />
combined with traditional pedagogy, can enhance cultural literacy in educationally and culturally<br />
challenged areas.<br />
Acknowledgements<br />
We would like to thank the National Scholarship Fund of Greece for its support during our research.<br />
We would also like to address our gratitude to the people of the island of Spetses for its help and<br />
support during the film making.<br />
References<br />
Barnett, R. (2000) Realizing the university in an age of super-complexity, SRHE/OU, Buckingham<br />
Barthes, R. (1977) Rhetoric of the image, Hill and Wang, New York.<br />
Baudinet –Mondzain, M.J. (1990) “Icone”, Les motions philosophiques, t.1, Presses Universitaires de France,<br />
Paris.<br />
Cocula, B. & Peyroutet, C. (1999) Sémantique de l’image, Editions Delagrave, Paris.<br />
Gonnet, G. (1997) Education et Media, Presses Universitaires de France, Paris.<br />
Greek Institute of Pedagogy (1998) Interdisciplinary Curriculum, Greek Pedagogic Institute, Athens.<br />
Jacquinot, G. (1977) Image et pédagogie : Analyse sémiologique du film à intention didactique, Presses<br />
Universitaires de France, Paris.<br />
Joly, M. (2005) A propos de la formation à la lecture de l’image et à son analyse : entre le diable et l’ineffable in<br />
Semoglou, K. (2005) Image and Child, Macedonia University Press, Athens.<br />
Kress, G. (2005) Imagination, the world of images and the new media in Semoglou, K. (2005), Image and Child,<br />
Macedonia University Press, Athens.<br />
Kress, G., Van Leeuwen, T. (1996) Reading Images. The Grammar of Visual Design, Routledge, London.<br />
Levie, W. H. and Lentz, R. (1982) “Effects of text illustrations: a review of research”, Educational Communication<br />
and Technology Journal, No. 30, 195-232, Sage Publications, New York.<br />
Pleios, G. (2005) Civilization of Image and Education: The role of Iconic Ideology, Polytropon, Athens.<br />
Régis, C. (2009) ‘’La lecture de l’image ne s’improvise pas’’, Les français dans le monde, No. 364, pp. 29-31.<br />
343
Paraskevi Kanari and Georgios Potamias<br />
Sankey, M. (2005) ‘’Considering visual literacy when designing instruction’’ [online] University of Southern<br />
Queensland from:www.usq.edu.au/users/sankey/Resources/article0602.pdf (accessed on 15/07/2006)<br />
Schirato, T. and Yell, S. (1996) Communication & Cultural Literacy: An Introduction, Allen & Unwin Pty Ltd,<br />
St.Leonard.<br />
Semoglou, K. (2005) Image and Child, Macedonia University Press, Athens.<br />
Smyrneos, A. (2008) The didactic of History, Grigoris Publications, Athens.<br />
Tardy, M. (1973) Le professeur et les images, Presses Universitaires de France, Paris.<br />
The New London Group (1996) ‘’A pedagogy of Multiliteracies. Designing Social Futures’’, [online], www.<br />
Thenewlondongroup.com (accessed on 07/07/2007)<br />
Van Leeuwen, T. & Jewitt, C. (2001) Handbook of Image Analysis, Sage, London.<br />
344
Effectiveness and Learners’ Evaluation of Combining<br />
Audio and Written Online Formative Feedback for<br />
Language Learning<br />
Rosario Kane-Iturrioz<br />
Department of English and Languages, Faculty of Business Environment and<br />
Society, University of Coventry, UK<br />
r.kane-iturrioz@coventry.ac.uk<br />
Abstract: Within the factors and drivers identified as leading assessment in higher education, feedback often<br />
emerges as ‘the most powerful single influence’ (Gibbs & Simpson, 2004) on students’ engagement and<br />
achievement. Although research on feedback is relatively new, in particular, that relating to audio feedback<br />
technology, the results indicate that such has to be aligned with the assessment criteria and <strong>learning</strong> outcomes in<br />
addition to being timely in order to respond to students’ expectations (Weaver, 2006). Furthermore, not all types<br />
of tutor comments are equally useful as Walker (2009) reports. Price et al (2010) argue that “the learner is in the<br />
best position to judge the effectiveness of feedback, but may not always recognise the benefits it provides”<br />
leading to the need for an improved student advice on how to understand and use feedback. This paper reports<br />
on the development of a multi-faceted approach to providing formative feedback on a blended e-<strong>learning</strong> module.<br />
The method combines immediate written feedback on web-based <strong>learning</strong> materials and formative online<br />
assignments with near-immediate written and audio feedback through audio eMail technology (Wimba voice<br />
eMail) regarding online oral tasks. The study was conducted over a period of four years using a multi-method<br />
approach to qualitative and quantitative data collection and analyses from a total of 100 students studying<br />
Spanish for Business at beginner’s level. The use of both written and audio feedback is welcomed by students<br />
and seems to overcome some of the problems highlighted in previous research on formative assessment<br />
regarding quality, detail and timing of feedback. Nevertheless, a few students reported technical difficulties and/or<br />
misapprehension regarding the use on campus of audio eMail technology for oral assignments. The paper also<br />
presents the results on the correlation between students’ grades and improvements on formative online<br />
assignments and quality of feedback and concludes with reflections and advice on the introduction of audio<br />
feedback in combination with other forms of formative feedback.<br />
Keywords: effectiveness; online formative feedback; language <strong>learning</strong>; audio and written feedback; students’<br />
evaluation<br />
1. Introduction<br />
The extensive review of literature on assessment performed by Gibbs & Simpson (2004)<br />
demonstrated the strong link between assessment requirements and students’ <strong>learning</strong> strategies.<br />
Other studies have highlighted the potential of information and communication technology in changing<br />
students’ approaches to <strong>learning</strong> through appropriately designed assessments, a recent example<br />
being the study by Kirkwood & Price (2008).<br />
Research into technology-enhanced assessment has reported on the positive effects on student<br />
retention, enhanced quality of feedback, flexibility in distance <strong>learning</strong>, strategies to deal with larger<br />
student numbers, objectivity in marking and more effective use of virtual <strong>learning</strong> environments.<br />
Within the factors identified as leading assessment in higher education, feedback often emerges as<br />
‘the most powerful single influence’ (Gibbs & Simpson 2004) on students’ engagement and<br />
achievement. Race (2006) postulates that in order to be effective in promoting <strong>learning</strong>, a formal<br />
feedback strategy should be timely, constructive, motivational, personal, manageable and directly<br />
related to the assessment criteria and <strong>learning</strong> outcomes.<br />
Although research on feedback is relative new, the results indicate that such has to be aligned with<br />
the assessment criteria and <strong>learning</strong> outcomes in addition to being timely in order respond to<br />
students’ expectations (Weaver, 2006). Furthermore, not all types of tutor comments are equally<br />
useful as Walker (2009) reports. Price et al (2010) argue that “the learner is in the best position to<br />
judge the effectiveness of feedback, but may not always recognise the benefits it provides”<br />
suggesting a need for improved student advice on how to understand and use feedback. Regarding<br />
the application of audio feedback technology, this is a new development in higher education and the<br />
limited number of research studies published seems to indicate a greater effectiveness over textbased<br />
feedback, leading to positive perceptions from students and increased retentions rates (Lunt &<br />
Curran, 2010).<br />
345
Rosario Kane-Iturrioz<br />
Regarding language <strong>learning</strong>, there are few studies focusing on the exploration of students’<br />
experiences of web-based formative assessments and feedback and even less dealing with the<br />
specific issues affecting beginner language learners (Furnborough and Truman, 2009).<br />
This paper reports on students’ perceptions on formative online assessment and a multi-faceted<br />
approach to providing formative feedback on a blended e-<strong>learning</strong> module. The current work draws<br />
upon some of the qualitative and quantitative data collected over a four-year period from a total of 100<br />
students studying Spanish for Business at beginner’s level. Using the annual data gathered, this study<br />
sought to answer three questions, namely,<br />
Did students perceive the formative assessments (Progress Tests) as an effective method of<br />
checking their own <strong>learning</strong>?<br />
Whether the formative feedback provided online involving text and oral formats had been useful in<br />
supporting students’ <strong>learning</strong>?<br />
Were students’ perceptions about the improvements in their language skills corroborated by the<br />
summative tests results?<br />
2. Method<br />
2.1 Participants<br />
The study was conducted over a four-year period and all participants were enrolled in a 10 credit<br />
module which focused on Spanish for Business at beginner’s level. The module lasted 10 weeks (2<br />
contact hours per week) in 2007-08, 2008-09 and 2009-10 but was changed to 20 weeks (one hour<br />
contact per week) for all the language modules within the Employability Programme during 2010-11 in<br />
order to accommodate requests from some language tutors and students.<br />
2.2 Research setting<br />
The structure of the module included an extensive e-<strong>learning</strong> component supported by a limited<br />
number of in-class introductory and revision sessions in order to provide flexibility in time, place and<br />
pace of <strong>learning</strong> to university-wide undergraduate students and to minimize feelings of anxiety about<br />
<strong>learning</strong> in what for many was an unfamiliar environment.<br />
The online materials were based on an innovative e-<strong>learning</strong> template designed and developed in<br />
2006-07 that integrated several <strong>learning</strong> technologies and incorporated interactive and reusable<br />
<strong>learning</strong> objects. All the online activities and exercises provided immediate computer feedback and<br />
allowed students multiple attempts (Kane-Iturrioz, 2009). The online workbook was re-mastered using<br />
the authoring programme SoftChalk in 2009-10 in order to incorporate a greater range of interactive<br />
activities. These materials were available to students on the University’s virtual <strong>learning</strong> environment<br />
(VLE.).<br />
2.3 Formative assessment<br />
Formative assessments called ‘Progress Tests’ were aimed at providing students with the appropriate<br />
means to check their progress. They also offered tutors the opportunity to personalise their feedback<br />
based on students’ strengths and weaknesses in order to guarantee that they had achieved through<br />
their online <strong>learning</strong> the necessary knowledge, understanding and language skill development which<br />
would facilitate a more productive engagement with the communicative activities to be performed in<br />
the revision sessions.<br />
The Progress Tests were developed in the same computer-based format as those in the two<br />
summative assessments and delivered using the assessment tool provided by the University’s VLE in<br />
order to be consistent with the module’s main <strong>learning</strong> mode. Consequently, these tests also aimed at<br />
familiarizing students with a test format that was new to the majority of them. The questions covered<br />
all four language skills and reflected real life business scenarios. Multiple attempts were allowed for<br />
both the tests and the questions within each test.<br />
Changes in the organization and content of the Progress Test were implemented throughout the four<br />
years of the study in order to accommodate students’ requests and syllabus modification. In the first<br />
year of the study, Progress Tests containing five questions were developed for Topics 1 & 2, Topics 3<br />
346
Rosario Kane-Iturrioz<br />
& 4 and Topic 5. These were initially scheduled in weeks 3, 6 and 8 as part of students’ preparation<br />
for the in-class revision sessions. In order to emphasise the formative character of these tests and<br />
encourage students to engage in checking their <strong>learning</strong>, no scores were provided, the tutor issuing<br />
personalised feedback both online and in class during the revision weeks.<br />
The 2007-08 students’ perception questionnaire revealed that the majority liked the immediacy of the<br />
computer generated feedback of the online <strong>learning</strong> materials and requested that a similar approach<br />
was taken with the Progress Tests and also to have such a test after each of the Topics studied.<br />
Consequently, from 2008-09, Progress Tests were developed for each Topic incorporating immediate<br />
computer feedback for most of the questions and including scoring. Guidelines were given online to<br />
students advising the achievement of a 70% score in each test before moving to the next Topic.<br />
2.4 Formative feedback<br />
Formative feedback was the main means of supporting the teacher’s role in promoting <strong>learning</strong><br />
(scaffolding), reflection and motivation both during class contact and online <strong>learning</strong> sessions. For the<br />
latter, an action research approach was used to develop a multi-faceted method of providing timely<br />
and effective feedback including computer generated text (Figure 1)and written and oral tutor<br />
comments via audio eMail technology.<br />
Figure 1: Example of interactive activities and feedback<br />
Formative feedback given to students for each activity in the Progress Tests evolved throughout the<br />
four years of the study in response to students’ perceptions expressed in the end of year<br />
questionnaires. A transition took place from tutor comments to a combination of computer and tutor<br />
feedback. Figure 2 shows an example of the formative feedback provided via the Progress Tests.<br />
Wimba voice tools were initially integrated into online revision activities in order to provide extra oral<br />
practice and opportunities for collaborative work among students. Students were also given the option<br />
to send their oral recordings via voice eMail in order to receive tutor formative feedback. The<br />
experience was a very positive one for the students who used this tool leading to the inclusion of oral<br />
activities in the Progress Test for the last two topics in the syllabus. Figure 3 provides an example of<br />
feedback provided via voice eMail.<br />
347
Rosario Kane-Iturrioz<br />
Figure 2: Example of progress test and question feedback<br />
Figure 3: Example of voice eMail feedback<br />
348
2.5 Data collection<br />
Rosario Kane-Iturrioz<br />
2.5.1 Survey of students’ perceptions<br />
With the approval of the University Ethics Committee, two data gathering instruments (surveys and<br />
semi-structured interviews) were used in the study. The semi-structured interviews were conducted in<br />
2008-09 with funding received from the Centre for Inter-Professional e-Learning (CIPel).<br />
The initial questionnaire included background information; native language and level of knowledge of<br />
other languages; <strong>learning</strong> preferences and use of technology and, familiarity with online <strong>learning</strong>. The<br />
end of module survey was intended to ascertain students’ perceptions of the module’s blended<br />
<strong>learning</strong> environment and contained 20-26 close-ended questions. Additional questions were inserted<br />
every year in order to accommodate students’ feedback regarding the changes made in response to<br />
the previous year’s comments. To facilitate statistical analysis most of the questions were in a 5-point<br />
Likert scale. Additionally, several open-ended questions attempted to elicit more detailed students’<br />
feedback.<br />
For the purpose of this study, questions relating to the students background information, language,<br />
computer literacy and prior online <strong>learning</strong> experience were analysed. The data from the survey’s<br />
closed-questions underwent a separate analysis from that involving the open-questions and the<br />
interviews transcripts. The results were cross tabulated and analyzed for statistical differences (p
Rosario Kane-Iturrioz<br />
Tests very useful as a self-assessment facility to check their progress. A breakdown of students’<br />
perception by year can be seen in Table 1.<br />
Table 1: Students’ perceptions on the usefulness of the progress tests<br />
Statement: I have found the Progress Tests useful to keep<br />
track of how much I have learnt online.<br />
2007-08<br />
(N=13)<br />
2008-09<br />
(N= 31)<br />
2009-10<br />
(N=28)<br />
2010-11 (N=17)<br />
Strongly agree 3 (23.08%) 12 (38.71%) 11 (39.29%) 5 (29.41%)<br />
Agree 10 (76.92%) 15 (48.39%) 15 (53.57%) 10 (58.83%)<br />
Neutral 0 (0.0%) 4 (12.90% 1 (3.57%) 1 (5.88%)<br />
Disagree 0 (0.0%) 0 (0.0%) 1 (3.57%) 1 (5.88%)<br />
Strongly disagree 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%)<br />
In 2008-09, students were also asked to select which aspect they liked most about the module’s<br />
online provision from a list of ten items. Progress Tests were the students third favourite preference<br />
(63%) after “work at my own pace” and “multiple attempts allowed”. The open question<br />
(questionnaires) and interviews (2008-09) corroborated these results and many students stated that<br />
they “would rather do the Progress Test than the voice mail or discussion”. Others liked this tool<br />
because it “...help(s) us to revise as well as prepare for the face-to-face sessions.” or, “I could see<br />
how I got to learn more really and you could always go back”. Although many students associated the<br />
Progress Test with an improvement in their independent <strong>learning</strong> skills: “..this experience was good<br />
and I have got a lot of advantage out of it, like self-discipline and time management.”; they also<br />
indicated that they had faced difficulties: “I ran out of time and so my main aim was to learn the topic<br />
material” or “there was nobody pushing me”. In one case the problem was fear of failure “Didn’t do<br />
most of them (Progress Tests) because wasn’t confident enough”.<br />
One of the aims of the Progress Test was to familiarise students with the module’s summative<br />
assessment format. A computer-based test was chosen in order to be consistent with the module’s<br />
main delivery mode. Students were asked about their perceptions regarding the computer-based<br />
summative assessment at two points during the four years of this study, 2008-09 and 2010-11. The<br />
students’ answers indicate a clear change in perceptions in the intervening year. 52% and 74%<br />
respectively “found the computer-based assessment better than the traditional assessment (including<br />
face-to-face oral tests). The breakdown of the results can be seen in Table 2.<br />
Table 2: Students’ perceptions of computer-based assessments<br />
Statement: “I have found the computer-based assessment better<br />
than the traditional assessment (including face-toface<br />
oral tests).<br />
2008-09 (N = 31) 2010-11 (N =19)<br />
Strongly agree 6 (19.35%) 5 (26.31%)<br />
Agree 10 (32.26%) 9 (47.37%)<br />
Neutral 9 (29.03% 3 (15.79%)<br />
Disagree 4 (12.90%) 2 (10.53%)<br />
Strongly disagree 2 (12.90%) 0 (0.0%)<br />
3.3 Students’ perceptions about formative feedback<br />
In 2008-09 and 2010-11, students were asked to select which aspect they preferred most about the<br />
module’s online coursework from a list of ten items. Immediate computer feedback was the 6 th<br />
favourite preference in 2008-09 and the third in 2010-11 after “work at my own pace” and “variety of<br />
activities”. Many students referred to the feedback provided for activities and exercises linking them to<br />
positive effects on enjoyment: “That is the bit I enjoy the most in the module, the whole<br />
interactiveness of it, it made it seem like fun”. Others reflected on the perceived motivational help<br />
provided by the activities feedback (mainly computer-based): “ It feels you being there with me’’. This<br />
statement seems to indicate that a comprehensive feedback is seen by students as an indication of<br />
‘caring behaviour’ on the part of the tutor.<br />
The Progress Test feedback both computer-based and tutor provided was in general perceived as<br />
highly useful (84.62%) and motivating: “What really got me motivated was getting the results straight<br />
back; to have a look through at what the mistakes were so it helps us”. A few students would have<br />
350
Rosario Kane-Iturrioz<br />
preferred all the feedback computer-based because of the perceived effect of the immediacy of a<br />
response, “with an immediate answer (I) maybe do an extra half and hour”. Other students preferred<br />
the mixture of computer-based and tutor feedback as being very appropriate to prepare them for the<br />
summative assessments.<br />
The use of Wimba voice tools for oral practice was received with mixed reactions. The students more<br />
familiar with technology seemed to welcome the use of state-of-the-art tools, “Also I like the Wimba<br />
voice mail, is quite new technology, I can record my speaking, listen it, if I don’t like it, I can record<br />
again”. Other students found this tool interesting and fun: “The Wimba… I find it really interesting; it<br />
was fun as well because there were things I have never used before”. Although appreciating the<br />
benefits of Wimba’s voice tools, approximately 50% of the students did not use this resource very<br />
often for a number of reasons. Sometimes microphones were unavailable on campus or the students<br />
did not like practising on campus with other people listening to them. A larger proportion of the<br />
students (60% to 70%) submitted the voice eMails linked to the oral activities in the Progress Tests.<br />
Further insights came from the questionnaires’ open questions in which many students stated that<br />
they only engaged in activities that had a direct relevance to the summative assessments.<br />
The audio and text feedback provided via voice mail technology was well received by those students<br />
who submitted the recordings corresponding to the oral activities. There were two aspects highly<br />
valued by students, the first was a greater effectiveness of the feedback due to the synergies<br />
achieved in the integration of audio and text commentary, “The written feedback was helpful but the<br />
voiceMail feedback help me to understand better were I went wrong with my pronunciation”. Students<br />
reported that the audio feedback had reinforced the written feedback because it was more detailed<br />
and personalised. The second benefit referred to the effective delivery of feedback comments, “The<br />
feedback for my voice eMail submissions was great as I received it straight into my mail inbox”.<br />
3.4 Language skills<br />
Students’ perceptions about the module being effective in increasing their language skills in the first<br />
two years and in the final year of the study can be seen in Figure 4. This figure shows that the mean<br />
values for Reading, Listening and Oral skills increased substantially in 2008-09 and again in 2010 in<br />
the case of Reading. The highest values recorded corresponded to Writing and these values<br />
remained high in all three years.<br />
Mean Likert values (1-5)<br />
2.5<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
Reading Listening Writing Oral<br />
Figure 4: Perceptions of improvements in language skills<br />
351<br />
2007-08<br />
2008-09<br />
2010-11
Rosario Kane-Iturrioz<br />
When compared with the mean test scores (summative) for language skills over the same three years<br />
(Figure 5), it is apparent that these show the opposite distribution to the mean Likert values for Writing<br />
in 2008-09.<br />
80<br />
70<br />
60<br />
50<br />
Mean scores 40<br />
30<br />
20<br />
10<br />
0<br />
Reading Listening Writing Oral<br />
Figure 5: Mean test scores (summative) for language skills<br />
4. Discussion<br />
The study set out to explore the effectiveness and learners’ perception of formative assessment and<br />
feedback in the context of a blended <strong>learning</strong> module where online <strong>learning</strong> was the main feature.<br />
There were three questions that the study sought to answer, dealing with students’ perceptions<br />
regarding the effectiveness of the formative assessment strategy, their evaluation of the formative<br />
feedback provided and the students’ views relating to their improvement in language skills.<br />
Regarding the first question, the results of the study clearly demonstrated that the Progress Tests<br />
were perceived as an effective strategy. These results corroborated other research findings which<br />
have reported that learners valued the availability of online formative assessments because this<br />
allowed them to take control of their <strong>learning</strong> (Lyons and Thorpe, 2009) by facilitating reflexion on the<br />
gap between their performance and the level of attainment needed (Gibbs and Simpson, 2004). Some<br />
students reported that the Progress Tests had contributed to the development of better independent<br />
<strong>learning</strong> skills involving self-discipline which appeared to indicate that they were on their way to<br />
develop a better self-regulation of their <strong>learning</strong> (Nicol and Mcfarlane-Dick, 2006). Additionally,<br />
students stated in the 2008-09 survey that the Progress Tests were the third most popular element of<br />
the online provision only surpassed by the convenience of working at their own pace and that multiple<br />
attempts were allowed (for both the online exercises and the Progress Test). The latter result confirms<br />
that of Felix (2001) who reported that students felt that having multiple attempts promoted language<br />
<strong>learning</strong>.<br />
The second question focused on the students’ views regarding the formative feedback received. The<br />
results indicated that the respondents associated the provision of an immediate and comprehensive<br />
feedback with an increase in their motivation and also in their sense of efficacy. Similar findings were<br />
reported by Segarra and Zapata (2008) and Furnborough and Truman (2009).<br />
Computer-based feedback was included in the majority of the Progress Tests’ questions from the<br />
second year of the study onwards in response to students’ requests. In the case of some learners, the<br />
352<br />
2007-08<br />
2008-09<br />
2010-11
Rosario Kane-Iturrioz<br />
instant feedback led to spending “many unanticipated hours in front of the computer in pursuit of the<br />
perfect score” (Segarra and Zapata, 2008). A few of the Progress Tests questions retained tutorbased<br />
feedback in order to provide students with semi-structured activities that were too complex for<br />
automatic computer feedback and required detailed explanation from the tutor. Students found the<br />
combination of computer and tutor feedback very useful in subsequent years because this method<br />
integrated the benefits of timely guidance with constructive feedback and information on how to<br />
improve their performance (Felix, 2003).<br />
The voice tools were integrated into the module’s homepage (VLE) and considered easy to use by<br />
both students and the tutor. They were used by 40% to 50% of the students because it provided<br />
opportunities to practice their oral skills through an online environment and they could also keep their<br />
audio files for future reference by downloading them onto their computers or mobile phones. For the<br />
tutor, the recorded audio files provided opportunities for comprehensive feedback. From a practical<br />
perspective, it also facilitated internal and external moderation of students’ summative oral<br />
assessments.<br />
However, voice tools for oral practice were not universally welcomed. Approximately one third of the<br />
students each year revealed reluctance about using computers to practice their oral skills. The<br />
consistent feedback from these students was that they preferred to practice oral conversation in a<br />
class-based setting with ‘real’ people. A further drawback mentioned by some students was that they<br />
did not like being overheard by other students when using voice tools on-campus. These perceptions<br />
showing a dislike by some students to computers as a medium for language <strong>learning</strong> and the<br />
discomfort felt whilst recording on campus were also key features in a study of learner drop-out<br />
reported by Stracke (2007). A larger proportion of the students only used the voice eMail tool for the<br />
oral activities contained in the Progress Tests as preparation for the summative assessment.<br />
In an earlier paper regarding learners’ experiences and perceptions of blended <strong>learning</strong> for languages<br />
(Kane-Iturrioz, 2010), it was reported that the students’ <strong>learning</strong> experience was still mainly<br />
determined by a traditional class-based model that did not foster independent <strong>learning</strong> skills. In the<br />
current study, students referred to their difficulties in adjusting to a regular weekly online workload<br />
very different from the <strong>learning</strong> pattern used in other modules where most of their study was linked to<br />
assessments and exams. This corroborates the results of Murday et al. (2008) who also report<br />
students’ problems regarding ‘self-regulation’ and its effect on timely study.<br />
The final research question sought to determine whether the summative test corroborated students’<br />
perceptions regarding the effectiveness of the module in improving their language skills. The results<br />
reported indicate that students’ perceptions of such improvements in 2008-09 seem to mirror the only<br />
change introduced in that year, that is, the reorganization of the Progress Tests. Those in 2010-11<br />
could be attributed to the enhanced feedback provided in both materials and Progress Tests and also<br />
to the change in the module’s schedule which gave students more time to develop their skills. Which<br />
of these changes contributed more to the students’ perceptions cannot be ascertained from the<br />
available data. The comparison of the trend in the mean scores and perception values seems to<br />
indicate that they closely resemble each other in their incremental direction.<br />
A minority of students expressed doubts regarding the appropriateness of an e-blended approach to<br />
language <strong>learning</strong>. However, one of these students who had little confidence in using computers<br />
stated, “it was incredibly hard ...but overall I did enjoy it and, you know, I have learnt a lot more<br />
Spanish that I thought I would do”. Some students expressed the opinion that “.. for a language<br />
module I think online is not the way”. Nevertheless, one of these found the module useful in helping<br />
him to become a better independent learner, “I think that I know where I stand as an independent<br />
learner, which really helps”.<br />
5. Conclusion<br />
Students highly appreciated the module’s assessment strategy designed to provide regular<br />
opportunities for reflection on their online <strong>learning</strong>. The range of formative feedback offered was also<br />
welcomed by students because of the combination of timeless provision of immediate computerbased<br />
comments/hints and detailed commentary from the tutor. The first type of feedback encouraged<br />
many students to engage with the online materials and also improved motivation through a better<br />
understanding of the language and the test requirements. The second type, helped students to<br />
appreciate their strengths and weaknesses and facilitated a deeper knowledge of the subject.<br />
353
Rosario Kane-Iturrioz<br />
Although the voice tools provided were not used by approximately one third of the students, those<br />
who did access them reported that the audio and written feedback received became part of their<br />
<strong>learning</strong> process. The verbal comments recorded by the tutor complemented and clarified the written<br />
part of the feedback. Voice mail technology made the feedback easily accessible via students’<br />
personal inboxes and the tutor’s comments were received soon after the assessment enabling them<br />
to analyse their performance quickly.<br />
From the tutor’s perspective, voice mail technology does not need complex settings and instructions.<br />
The beneficial effect reported by students was achieved through the tutor dedicating a few minutes to<br />
type and record the feedback as soon as the submitted voice mail messages were received.<br />
Research has shown that students adapt their <strong>learning</strong> strategies to the summative assessment<br />
requirements (Felix, 2001; Gibbs and Simpson, 2004; Kirkwood and Price, 2008). The results in this<br />
study appear to validate the idea that students’ <strong>learning</strong> can be shaped by regular, short formative<br />
assessments leading to a steady focus on study, perception of improved language skills development<br />
and the attainment of better grades.<br />
References<br />
Felix, U (2001) “The web’s potential for language <strong>learning</strong>: the student’s perspective”. ReCALL, Vol 13, No. 1, pp<br />
47-58.<br />
Felix, U. (2003) “Teaching language online: Deconstructing the myths”. Australian Journal of Educational<br />
Technology, Vol 19, No. 1, pp 118-138.<br />
Furnborough, C. Truman, M. (2009) “Adult beginner distance language learner perceptions and use of<br />
assignment feedback”. Distance Education, Vol 30, No. 3, pp 399-418.<br />
Gibbs, G., and Simpson, C. (2004) “Conditions under which assessment supports students’ <strong>learning</strong>”. Learning<br />
and Teaching in Higher Education, Vol 1, No.1, pp 3–31.<br />
Kane-Iturrioz, R. (2009) “Enhancing Blended Language Learning: Development of a new eLearning Template”,<br />
[online], EUROCALL Review 15, March-September. Special Issue, pp 36-55, http://www.eurocalllanguages.org/news/newsletter/15/index.html<br />
Kane-Iturrioz, R. (2010) “Effective Blended Learning for Languages: Learners’ Experiences and Perceptions”.<br />
Proceedings to the Third International Conference on Innovation in Learning for the Future 2010; e-<br />
Learning, Istanbul, May, 10-14, 2010. TC Istanbul Kultur University Publication No: 125, pp 340- 350.<br />
Kirkwood, A. and Price, L. (2008) “Assessment and student <strong>learning</strong>: a fundamental relationship and the role of<br />
information and communication technologies”. Open Learning: The Journal of Open, Distance and e-<br />
Learning, Vol 23, No. 1, pp 5-16.<br />
Lunt, T. and Curran, J,. (2010) “Are you listening please? The advantages of electronic audio feedback compared<br />
to written feedback”. Assessment & Evaluation in Higher Education, Vol 35, No.7, pp 759-769.<br />
Lyons, H. and Thorpe, L. (2009) “Investigating students experiences of e-<strong>learning</strong> using the Diary Interview<br />
Approach”, [online], Brookes eJournal of Learning and Teaching, Vol 2, No. 4. http://bejlt.brooks.ac.uk/<br />
(accessed on 03.06.2009).<br />
Murday, K., Ushida, E. and Chenoweth, A. (2008) “Learners’ and teachers’ perspectives on language online”.<br />
Computer Assisted Language Learning, Vol 21, No. 2, pp 125-142.<br />
Nicol, D.J., and Mcfarlane-Dic, D. (2006). “Formative assessment and self-regulated <strong>learning</strong>: A model and<br />
seven principles of good feedback practice”. Studies in Higher Education, Vol 31, No. 2, pp 199-218.<br />
Price, M., Handley, K., Millar, J. and O’Donovan, B. (2010). “Feedback: all that effort, but what is the effect?”<br />
Assessment & Evaluation in Higher Education, Vol 35, No. 3, pp 277-289.<br />
Race, P. (2006) “The Lecturer’s Toolkit – A Practical Guide to Assessment”, Learning and Teaching, 3rd Edition,<br />
Routledge, London.<br />
Sagarra, N. and Zapata, G.C. (2008) “Blending classroom instruction with online homework: A study of students<br />
perceptions of computer-assisted L2 <strong>learning</strong>”. ReCALL, Vol 20, No. 2, pp :208-224.<br />
Stracke, E. (2007) “A road to understanding: A qualitative study into why learners drop out of a blended language<br />
<strong>learning</strong> (BLL) environment”. ReCALL, Vol 19, No. 1, pp 57-78.<br />
Walker, M. (2009) “An investigation into written comments on assignments: do students find them usable?”<br />
Assessment & Evaluation in Higher Education, Vol 34, No. 1, pp 67-78.<br />
Weaver, M. (2006) “Do students value feedback? Student perceptions of tutors’ written responses”. Assessment<br />
& Evaluation in Higher Education, Vol 31, No. 3, pp 379-394.<br />
354
Model of eLearning Project Evaluation<br />
Jana Kapounova, Jana Sarmanova and Marketa Dvorackova<br />
University of Ostrava, Czech Republic<br />
Jana.Kapounova@osu.cz<br />
Jana.Sarmanova@osu.cz<br />
D10576@student.osu.cz<br />
Abstract: The paper deals with the model of eLearning evaluation process, specifically with a proposal of<br />
methods and means for eLearning project evaluation. There are many approaches to eLearning projects<br />
evaluation. Let us consider eLearning as an educational project. An educational project, just like other projects, is<br />
comprised of design phases, proceeding one after another and interrelating. An eLearning project passes<br />
through several phases of its development: from planning through the development of study supports and the<br />
pilot stage, to usage of project results in instruction, namely in distance <strong>learning</strong>. eLearning project can be<br />
evaluated during each of its phase and as a whole project as well. It means for each phase of the project we<br />
have to collect reliable information to determine criteria of the phase quality. After evaluation of the phase we<br />
define feedback as a response to the results of performed analysis. In the present paper we want to demonstrate<br />
our way of dividing the eLearning project into phases like phases of the system development life cycle. We<br />
evaluate each individual phase. We have determined activities of evaluation, input information, outputs and<br />
staffing of the activities. We chose information that employs as measures needed for quality evaluation of each<br />
phase and of the project as a whole.<br />
Keywords: evaluation of eLearning, eLearning project, project life cycle, eLearning project phases, project<br />
evaluation<br />
1. Introduction<br />
The expansion of eLearning since the last century was very fast especially in universities and<br />
corporate training sector. Information and Communication Technologies (ICT) were first used for<br />
communication between students and teachers (via email), in the area of supporting study materials<br />
(study texts, PowerPoint presentations, video recordings of lectures), and finally for the controlling<br />
and management of studies (Lojda, 1999).<br />
eLearning plays a crucial role in distance education, but it is also important in supporting face-to-face<br />
studies. The most common form of study using eLearning is a combined form, sometimes called<br />
blended <strong>learning</strong>.<br />
With the rise of eLearning, many, especially managers of educational institutions, thought that the<br />
eLearning form of instruction will be cheaper. This belief of eLearning being a not expensive solution<br />
is still widespread among the public, even though some experts have warned about higher costs<br />
associated with the implementation of eLearning from the very beginning. What affects the eLearning<br />
projects then, what should we not neglect during their development and implementation? (Poulová,<br />
2006)<br />
Within the ‛eLearning Project Evaluation – the system approach’ project we intend to establish a<br />
procedure (procedures) and determine factors that can affect quality, effectiveness and economy of<br />
an eLearning project.<br />
There are many approaches to eLearning projects evaluation. Most of theoretical studies and<br />
examples of good practice deal with evaluation of study supports and ways of communication<br />
between learners and instructors (Eger, 2004). The effectiveness of eLearning is with no doubts<br />
affected by other factors as well which we will try to determine in our work (Freibergová, 2000). We<br />
presume that the best way of how to prepare the methodology for eLearning evaluation should be the<br />
system approach.<br />
2. Some approaches to the problematic<br />
The discussion of eLearning, or of its evaluation, requires us to define several terms. From a wide<br />
range of definitions that are to be found throughout the relevant bibliography, we have accepted the<br />
following.<br />
355
Jana Kapounova et al.<br />
eLearning is an educational environment which uses information and communication technologies to<br />
achieve the educational goal: it includes creation of educational objects, distribution of study content,<br />
implementation, communication between participants of the educational process and management of<br />
studies.<br />
The greatest emphasis is placed on the term ‛<strong>learning</strong>’ while the ‛e-’ is the electronic-related prefix,<br />
notifying of using ICT.<br />
The assessment and evaluation we have based on pedagogical principles. In pedagogical science<br />
there are two most often forms of evaluation/assessment (Zlámalová, 2002).<br />
Assessment of the learner’s achievements within the process of <strong>learning</strong>, in other words the<br />
comparison of his/her entry and output competencies.<br />
Evaluation of effectiveness, in terms of the effectiveness of the system including economic<br />
evaluation of the investment into the educational product.<br />
Theories of eLearning evaluation approaches are very commonly discussed. We have reviewed many<br />
theoretical studies and examples of good practice. Such, which can be used to determine factors<br />
suitable for our evaluation model, have been chosen. For the purposes of our work, we prefer the<br />
following theoretical approaches:<br />
Kirkpatrick’s Four Levels of Evaluation (Winfrey, 1999);<br />
A Framework for the Evaluation of eLearning (Huges, Attwell, 2002);<br />
The Model of assessing quality in eLearning elaborated by The Swedish National Agency of<br />
Higher Education (Åström, 2009);<br />
Managing eLearning Strategies (Khan, 2005).<br />
First we remind in brief, what is the main point of these methods, particularly in relation to our<br />
objectives – to prepare a methodology for eLearning project evaluation<br />
2.1 Kirkpatrick’s four levels of evaluation<br />
Donald Kirkpatrick’s four-level model of evaluation (quoted from Winfrey, 1999) can be applied to<br />
technology-based training as well as to more traditional forms of <strong>learning</strong>. In Kirkpatrick’s four-level<br />
model, each successive evaluation level is built on information provided by the lower level. According<br />
to this model, evaluation should always begin with level one, and then, as time and budget allows,<br />
should move sequentially through levels two, three, and four. Information from each prior level serves<br />
as a base for the next level’s evaluation.<br />
Level 1 Evaluation – Reactions<br />
Evaluation at this level measures how participants in a training program react to it. It attempts to<br />
answer questions regarding the participants’ perceptions. Did they like it? Was the material relevant to<br />
their work? This type of evaluation is often called a ‛smile sheet’. According to Kirkpatrick, every<br />
program should at least be evaluated at this level to provide for the improvement of a training<br />
program.<br />
Level 2 Evaluation – Learning<br />
Assessment at this level measures the amount of <strong>learning</strong> that has occurred due to a training<br />
program, evaluations often use tests conducted before training (pre-test) and after training (post-test).<br />
Methods range from formal to informal testing to team assessment and self-assessment.<br />
Level 3 Evaluation – Transfer<br />
Evaluation at this level measures the transfer that has occurred in the learners’ behaviour due to the<br />
training program. Evaluating at this level attempts to answer the question: Are the newly acquired<br />
skills, knowledge, or attitude being used in the everyday environment of the learner? For many<br />
trainers this level represents the truest assessment of a program’s effectiveness.<br />
Level 4 Evaluation – Results<br />
356
Jana Kapounova et al.<br />
Evaluation at this level assesses training in terms of business results, measures the success of the<br />
programme in terms that managers and executives seek: increased production, improved quality,<br />
decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return<br />
on investment. Return On Investment (ROI) is sometimes mentioned as a possible 5 th level or can be<br />
included in original 4 th level Results.<br />
2.2 A framework for the evaluation of eLearning<br />
J. Hughes and G. Attwell presented a new framework for the evaluation of eLearning (Huges, Attwell,<br />
2002).<br />
They tried to define a number of variables, which potentially impact on the effectiveness of the project<br />
and to decide what constitutes dependent, independent and irrelevant variables in a given situation.<br />
Five major clusters of variables have emerged:<br />
Individual learner’s variables:physical characteristics, <strong>learning</strong> history, learner’s attitude, learner’s<br />
motivation, familiarity with the technology;<br />
Environmental variables:immediate (physical) <strong>learning</strong> environment, institutional environment,<br />
subject environment;<br />
Contextual variables:socio-economic factors, political context, cultural background, geographic<br />
location;<br />
Technology variables:hardware, software, connectivity, media, mode of delivery;<br />
Pedagogic variables:level and nature of learner support systems, accessibility issues,<br />
methodologies, flexibility, learner autonomy, selection and recruitment, assessment and<br />
examination, accreditation and certification.<br />
2.3 The model of assessing quality in eLearning<br />
An approach to the problematic of aspects and quality evaluation criteria definitions, described by the<br />
team of the Swedish National Agency for Higher Education, is very similar to our attitude. They have<br />
determined two basic quality criteria (Åström, 2009).<br />
Functional and system approach to eLearning implementation including the following partial<br />
aspects:<br />
Material/content;<br />
Structure/virtual environment;<br />
Communication, cooperation and interactivity;<br />
Student assessment;<br />
Flexibility and adaptability;<br />
Support (student and staff);<br />
Staff qualification and experience;<br />
Vision and institutional leadership;<br />
Resource allocation;<br />
The holistic and process aspect.<br />
Internal evaluation, updating and improving of using holistic approach to the resolution of the<br />
problematic.<br />
2.4 Managing eLearning strategies<br />
The author, Badrul Khan (Khan, 2005), presents critical eLearning factors as questions that can be<br />
asked when planning, designing, evaluating and implementing eLearning modules, courses and<br />
programmes. eLearning represents a new paradigm both for learners and for other stakeholders<br />
(instructors, trainers, administrators, technical and support service staff) and the institution. The eightdimensional<br />
eLearning framework was created to comprehend all of the following issues: institutional,<br />
management, technological, pedagogical, ethical, interface design, resource support and evaluation.<br />
357
Jana Kapounova et al.<br />
Each dimension includes several sub-dimensions and each sub-dimension consists of items or issues<br />
focused on a specific aspect of an eLearning environment.<br />
Another idea very important for our approach is the following. In eLearning projects we need to<br />
understand people, process and products concerned in eLearning (called P3 continuum). People are<br />
involved in the process of creating eLearning materials and making them available.<br />
Khan divides eLearning process into two phases:<br />
Content development – planning, design, production, evaluation of eLearning content and<br />
resources;<br />
Delivery and maintenance – implementation of online course offerings, ongoing updating and<br />
monitoring of eLearning environment.<br />
From the above mentioned theoretical methods we take some items that suit our purposes – to<br />
prepare a methodology for evaluation of eLearning. In our model we intend to employ the system<br />
approach, to specify structure and to find its characteristics.<br />
3. eLearning project and its phases<br />
Let us consider eLearning as an educational project. An educational project, just like other projects, is<br />
comprised of design phases, proceeding one after another and interrelating.<br />
Now let us define the term eLearning project:<br />
eLearning project is such a project within of which an eLearning subject, or subjects forming one<br />
complex (e.g. a study programme), or just a part of a subject is planned, developed, implemented and<br />
introduced in instruction.<br />
Note: By the term “subject” we mean a school subject; we prefer the term “subject” to the word “course”.<br />
An eLearning project represents:<br />
Creation of a set of eLearning subjects or eLearning components (most often);<br />
Addition or construction of Learnimg Management System (LMS) functions, e.g. adaptive<br />
instruction option.<br />
An eLearning subject includes:<br />
Relevant content (study support, multimedia components, sources and links, tests, etc.);<br />
Study system (student registration, task completion, test results, etc.);<br />
LMS (study flow monitoring).<br />
3.1 Some models of project phases<br />
An eLearning project passes through stages of development – from planning through study supports<br />
development, testing to usage of project results in instruction. There are many ways how to split down<br />
into stages of developed works. To determine the phases of eLearning project we used some theories<br />
and models presented in bibliography, e.g.<br />
System development life cycle (SDCL)<br />
Instructional System Design (ISD)<br />
Statement of work (SOW)<br />
The individual phases of an eLearning project are determined similarly as the system development life<br />
cycle.<br />
System development life cycle is a process that is used in connection with information systems<br />
development (Cummings, 2006, SDLC in Wkipedia, 2011). Essential phases of the cycle are:<br />
System analysis – analyse the situation, project goals, end-user needs and requirements definition.<br />
358
Jana Kapounova et al.<br />
Design – design functions and operations in detail. The new system is described as a collection of<br />
modules or subsystems.<br />
Testing – unit, system and user acceptance testing are performed; the whole system is tested as well.<br />
Operations and maintenance – include changes and enhancements before the termination of the<br />
system; the system maintenance is an important aspect; new changes require system updates.<br />
Instructional System Design (Clark, 2002) is another possible approach. For our modelling we accept<br />
this very scheme. It is also described as an educational parallel to the software development process.<br />
It breaks down into five phases, an acronym of which (ADDIE) is used as an equivalent to the term<br />
ISD. The design of an educational process must contain all of the five phases, so that the result would<br />
be a product that teaches.<br />
Analysis – identify needs and constraints.<br />
Design – define <strong>learning</strong> activities, assessment and media.<br />
Development – produce, perform formative evaluation, and revise.<br />
Implementation – deliver the instruction.<br />
Evaluation – evaluate results.<br />
It is necessary to point out that a pre-design phase precedes every project. It includes a statement of<br />
work providing a description of what must be done and in what scope. The stated objectives must<br />
thus fulfil the conditions of measurability, specificity, and attainability. This concerns the preliminary<br />
project plan (Schwalbe, 2007).<br />
4. Model of eLearning project evaluation<br />
When considering the eLearning project evaluation we established<br />
phases of an eLearning project which enabled us to determine<br />
phases of evaluation, within of which we have specified:<br />
activities necessary to be carried out when evaluating and<br />
people responsible for their fulfilment, and further<br />
activities inputs and<br />
activities outputs.<br />
All of the above mentioned we have organized in a following table.<br />
Table 1: eLearning project table structure<br />
eLearning project evaluation activity who performs it (is<br />
phase<br />
phase<br />
responsible)<br />
resource analysis: HR analysis HR analysis manager (of the<br />
human (HR)<br />
...<br />
evaluation check<br />
project)<br />
activity inputs activity<br />
outputs<br />
checklist completed<br />
checklist<br />
4.1 Phases of eLearning project<br />
The individual phases are preceded by a pre-design phase containing a definition or a requirement to<br />
establish an eLearning project.<br />
0. Requirement or an idea to realize, update or extend an eLearning project; it can also be a project<br />
application. Includes objectives, project justification, suitability, effectiveness, target group<br />
specification, expected contribution, added value, risks and options to eliminate them, continuing the<br />
359
Jana Kapounova et al.<br />
project, connections (to other projects, to the institution, to educational intentions of the institution,<br />
etc.), management and organization concept, and other.<br />
After this pre-design phase being assessed and approved, the phases of the system development life<br />
cycle follow.<br />
1. Analysis of the resources, subjects contents, readiness of the institution.<br />
2. Design of author requirements, teaching and <strong>learning</strong> objectives, concept of the content,<br />
specification of the study matter, teaching structure, teaching strategies and teaching forms<br />
determination, <strong>learning</strong> management plan layout, specification of feedback mechanisms, <strong>learning</strong><br />
aids, instructional units, personal coverage and time schedule of the project, maintenance and<br />
operation plan, operation and instruction termination.<br />
3. Development involves assigning roles or tasks to developers, task definition and specification,<br />
milestones and deadlines layout, development of the educational environment graphic design,<br />
creation of instructional objects, subjects behaviour setup, individual components check up, study<br />
supports completion, subject(s) complex check up, functionality verification before the pilot operation.<br />
4. Implementation starts with experimental or pilot operation, followed by feedback adjustments and<br />
preparation for routine operation, i.e. ensuring operation conditions, subject administration, student<br />
activities evaluation. We must not leave out the project and its parts updates.<br />
5. Evaluation of the project When evaluating an eLearning project it is necessary to distinguish the<br />
evaluation of the project as a whole from the evaluation of the individual subjects. It is also essential<br />
to distinguish evaluation from the executors’ point of view and those who participate in the project<br />
implementation: learners, teachers, personnel concerned, etc.<br />
For the evaluation of the project as a whole we analyse data acquired from evaluation materials, e.g.<br />
study support evaluation reviews, but also the quality of the evaluation ‛secondary products’, for<br />
example the content and composition of the study support evaluation form. In the final evaluation the<br />
whole eLearning project construction view is also considerable, what will for example happen, if we<br />
take out one of the subjects?<br />
4.2 Evaluation phases<br />
Individual phases of an eLearning project life cycle can be assigned with evaluation phases. In these<br />
we determine:<br />
Objectives of evaluation phase;<br />
Activities necessary to be carried out and by whom, or who is responsible for their execution;<br />
Input information are to be required for each of the activities;<br />
Output information;<br />
Measures of a feedback.<br />
Table 2: Example of the eLearning project evaluation activities<br />
eLearning<br />
project phase<br />
readiness of the<br />
institution<br />
analysis<br />
curricular accord with the<br />
educational<br />
orientation of the<br />
institution<br />
technical functional<br />
hardware,<br />
software, internet<br />
evaluation phase activity who performs it<br />
(is responsible)<br />
assess the<br />
institution<br />
comparison of<br />
the content of<br />
instruction with<br />
the orientation<br />
ICT requirements<br />
with available<br />
technologies<br />
360<br />
activity<br />
inputs<br />
manager comparison<br />
of the<br />
educational<br />
technician,<br />
manager<br />
content<br />
comparison<br />
of ICT<br />
equipment<br />
activity outputs<br />
evaluation<br />
statement, e.g.<br />
completed<br />
checklist<br />
evaluation<br />
statement, e.g.<br />
completed<br />
checklist
human<br />
resources<br />
tutors, lectors,<br />
administrators,<br />
etc.<br />
Jana Kapounova et al.<br />
HR requirements<br />
with professional<br />
and time<br />
potentials<br />
manager,<br />
personnel officer<br />
eLearning evaluation phase activity who performs it<br />
project phase<br />
(is responsible)<br />
financial financial project budget accountant,<br />
sufficiency with the<br />
institution<br />
financial plan<br />
manager<br />
organizational ability/readiness interview with manager,<br />
of the institution management of management of<br />
to accept the<br />
project<br />
the institution the institution<br />
client approach e.g. awareness information Public Relation<br />
of the people materials review (PR) specialist,<br />
concerned (PR)<br />
manager<br />
4.3 Activity outputs<br />
comparative<br />
tables<br />
activity<br />
inputs<br />
comparative<br />
balance<br />
sheet<br />
e.g. in a<br />
form of a<br />
questionnair<br />
e<br />
set of<br />
information<br />
materials<br />
evaluation<br />
statement, e.g.<br />
completed<br />
checklist<br />
activity outputs<br />
evaluation<br />
statement, e.g.<br />
completed<br />
checklist<br />
evaluation<br />
statement, e.g.<br />
completed<br />
checklist<br />
evaluation<br />
statement, e.g.<br />
checklist<br />
As evaluation activity outputs we have chosen the following: questionnaire, review, checklist,<br />
evaluation report, survey record, evaluation statement, data analysis conclusions.<br />
Results of the evaluation should among others show, if the people preparing the project have<br />
considered all of the issues, how thoroughly and in what depth. To make the process of evaluation<br />
less demanding and consuming, we need to optimize the set of evaluation materials. It is possible to<br />
cumulate some of the outputs, e.g. in the above mentioned table we can merge all of the evaluation<br />
statements of the readiness of the institution to a single checklist.<br />
5. Conclusion<br />
Every eLearning project is unique. Every developer chooses his own priorities within the project.<br />
Our approach to eLearning project evaluation we based on dividing a project into phases (like phases<br />
of the system development life cycle) and evaluating these individual phases. For each of the phases<br />
we have determined activities of evaluation which are necessary to be realized, input information,<br />
outputs and personnel coverage of the activities.<br />
The determined items have been reviewed by experts in the field of eLearning at universities in Czech<br />
Republic. According to their suggestions, adjustments have been made to the set of activities and<br />
their attributes.<br />
Subsequently some tools for individual phases and sub-phases evaluation are being designed.<br />
First a study support evaluation form has been designed, an evaluation tool from the eLearning<br />
education methodology usage point of view (Mechlová, 2008), (Šarmanová, 2009). The evaluation<br />
form is a structured questionnaire consisting of twenty-six questions split into four areas:<br />
Basic characteristics of text.<br />
Encouragement of students.<br />
Planning and organizing of study activities.<br />
Feedback and evaluation.<br />
The results of this evaluation on 57 study supports have been analysed with statistical methods<br />
(Kapounová, Šarmanová, 2010).<br />
At present two other evaluation tools are being verified:<br />
An implementing of multimedia objects evaluation form;<br />
Analysis and assessment of the suitability of the project in form of a checklist for reviewing the<br />
readiness of the developer institution for implementation and execution of an eLearning project.<br />
In this paper we present our approach to systematic evaluation of eLearning projects that suits our<br />
<strong>learning</strong> environment, particularly at universities and high schools in the Czech Republic.<br />
361
Acknowledgments<br />
Jana Kapounova et al.<br />
This work was supported by the Czech Science Foundation under the contract No. 406090242.<br />
References<br />
Åström, E. (2009) E-<strong>learning</strong> quality. Swedish National Agency for Higher Education, Stockholm : Sweden.<br />
Clark, R. C., Mayer, R. E. (2002) e-Learning and the Science of Instruction: Proven Guidelines for Consumers<br />
and Designers of Multimedia Learning. San Francisco : Pfeiffer.<br />
Cummings, H. (2006) Management Information Systems for the Information Age. Toronto : McGraw-Hill Ryerson.<br />
Eger, L. (2004) E-<strong>learning</strong>, evaluace e-<strong>learning</strong>u + případová studie z projektu Comenius. Plzeň : Západočeská<br />
univerzita,<br />
Freibergová, Z. (2000) Hodnocení kvality studijních programů. Praha : CSVŠ.<br />
Huges, G, Attwell, J. (2003) A framework for the evaluation of e-<strong>learning</strong> [online], London : KnowNet,<br />
.<br />
Kapounová, J. (2007) Approaches to the evaluation of eLearning. Proceedings of the Computer Based Learning<br />
in Science conference, pp. 208–214.<br />
Kapounová, J., Šarmanová, J. (2010) The Evaluation of eLearning Study Supports. Proceedings of the Computer<br />
Based Learning in Science conference, pp. 103–109.<br />
Khan, B. (2005) Managing e-<strong>learning</strong>: design, delivery, implementation and evaluation. Information Science Pub.<br />
Lacko, B. (2009) Zásady moderního projektového řízení. [online], http://lacko.otw.cz/eseje/Co_je_projektoverizeni.doc.pdf.<br />
Lojda, J. (1999) Význam kvality v distančním vzdělávání. In Distanční vzdělávání v České republice – současnost<br />
a budoucnost. Praha: CSVŠ – NCDiV,<br />
Mechlová, E. (2008) Tvorba e-<strong>learning</strong>ových kurzů pro technické obory. Ostrava : Technical University of<br />
Ostrava,<br />
Poulová, P. et al. (2006) Efektivita e-<strong>learning</strong>ových kurzů. In Distanční vzdělávání v České republice –<br />
současnost a budoucnost. Praha: CSVŠ – NCDiV,<br />
Šarmanová, J. (2009) E-<strong>learning</strong>ové prvky pro podporu výuky odborných a technických předmětů (eLearning<br />
Elements Used for Support in Teaching of Specialities and Technical Subjects). Project report. Ostrava :<br />
Technical University of Ostrava.<br />
Schwalbe, K. (2007) Information Technology Project Management. UK : Amazon.<br />
Systems Development Life Cycle (2011) [online], http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle .<br />
Winfrey, E. C. (1999) Kirkpatrick's Four Levels of Evaluation. In Hoffman, B. (Ed.), Encyclopedia of Educational<br />
Technology. [online], http://coe.sdsu.edu/eet/Articles/k4levels/start.htm.<br />
Zlámalová, H. (2002) Principy distanční vzdělávací technologie a možnosti jejího využití v pedagogické praxi na<br />
technických vysokých školách. [online], http://virtual.cvut.cz/telel/zlamalova.html.<br />
362
Bridging the Gap – From Teacher to eTeacher<br />
Andrea Kelz<br />
University of Applied Sciences Burgenland, Campus Pinkafeld, Austria<br />
andrea.kelz@fh-burgenland.at<br />
Abstract: Virtual <strong>learning</strong> environments have become part of nearly every educational programme in higher and<br />
further education. Hence, competent teachers, tutors and trainers are a major key to success in higher education<br />
and need a thorough grounding in how to design, produce, use and evaluate e-<strong>learning</strong> and blended <strong>learning</strong><br />
resources. But how do teachers actually gain the skills and insights you need to practice e-<strong>learning</strong> and blended<br />
<strong>learning</strong> effectively and efficiently and how do they develop a critical understanding of the pedagogical issues<br />
involved in the design, development and implementation of e-<strong>learning</strong> in their professional context? This paper<br />
focuses on the support offered to acquire and extend the skills needed to “teach online” from the institutional<br />
point of view of an Austrian university of applied sciences. It is shown in what way an institution tries to support<br />
teachers to gain the know-how to “transform” their teaching methods, to support them to make effective use of<br />
<strong>learning</strong> management systems and the web as an educational resource. Finally, the paper refers to what kind of<br />
nationwide training is offered to university teachers to develop the competencies needed to teach online<br />
effectively.<br />
Keywords: e-teaching, e-qualifications, train the trainer, professional development<br />
1. Starting point<br />
In line with the principles of the treaty of Bologna to converge higher educational systems towards a<br />
more transparent system, a transformation of all diploma programmes at Campus Pinkafeld into<br />
bachelor’s and master’s degree programmes started in 2004. Since then Campus Pinkafeld has been<br />
offering part-time degree programmes using a blended <strong>learning</strong> approach, allowing for a combination<br />
of on-site and off-campus study modules. Our part-time students can pursue degrees in two core<br />
competences, either in energy and environmental management or in health management and health<br />
promotion. In addition, students without an ‘A’ level equivalent may qualify to study at our university of<br />
applied sciences if they pass the accreditation examination in the required subjects. Similar to our<br />
bachelor’s and master’s part-time courses the preparatory courses for the required subjects are also<br />
based on a blended <strong>learning</strong> approach (cf. Walitsch 2008). Generally, all students at Campus<br />
Pinkafeld enjoy the same level of academic assistance, regardless of whether they opt to study full- or<br />
part-time. The 250 full-time students use the infrastructure of the campus and its adjoining technology<br />
centre Monday through Friday while the 350 part-time students complete their programmes through<br />
blended <strong>learning</strong>, which mixes a schedule of on-campus studies on weekends with additional studies<br />
using an internet-based platform. This resulted in the overall reduction of the amount of on-site<br />
teaching hours by at least one third.<br />
The overall design of the part-time programmes at Campus Pinkafeld implies two major organisational<br />
differences. First, courses only take place at weekends and second, the amount of face-to-face<br />
teaching was reduced by about one third. Offering part-time programmes only at weekends, i. e.<br />
Friday afternoons and Saturdays, contributed decisively to a wider acceptance of the technical degree<br />
programmes in the field of energy and environmental management as they do not only enable more<br />
local people to continue to work alongside their studies but also allow people living in remote areas to<br />
participate in the degree programmes. Second, a new constructivist-based didactic framework and its<br />
corresponding teaching methods and <strong>learning</strong> strategies were established to help part-time students<br />
acquire the relevant skills and competences for their future field of work. Both objectives could be<br />
achieved by technology-aided blended <strong>learning</strong> settings as they allow for a considerable reduction of<br />
face-to-face sessions and help implement new <strong>learning</strong> scenarios which are based on open,<br />
computer-aided, learner-centred, self-directed and collaborative <strong>learning</strong> arrangements.<br />
2. Blended <strong>learning</strong> strategy in general and didactic framework in particular<br />
Within the part-time degree programmes at Campus Pinkafeld blended <strong>learning</strong> is the term used to<br />
describe <strong>learning</strong> activities based on a combination of traditional face-to-face with computer and webbased<br />
online <strong>learning</strong> settings, multiple theories of <strong>learning</strong> and pedagogic approaches and a variety<br />
of media and tools employed in face-to-face and virtual <strong>learning</strong> environments, including traditional<br />
print materials as well as all kinds of technology-based materials and activities. The constructivistbased<br />
didactic approaches and their corresponding teaching methods and <strong>learning</strong> strategies<br />
pursued at Campus Pinkafeld are supposed to successfully help the students acquire the relevant<br />
363
Andrea Kelz<br />
skills and competences for their future field of work. As most of the <strong>learning</strong> and teaching at Campus<br />
Pinkafeld was based on behaviourist and cognitive assumptions when the first degree programme in<br />
building engineering started in 1994, the pedagogy and curriculum of technical education in part-time<br />
degree programmes implied a rethinking of how we determine, structure and deliver the content of<br />
technical education and try to find ways to restructure the overall <strong>learning</strong> environment in which the<br />
<strong>learning</strong> process itself has gained in importance. As far as curriculum design is concerned, each<br />
programme is comprised of individual study modules, each consisting of a number of courses which<br />
are split into face-to-face and blended <strong>learning</strong> modules. The modules and courses are defined in<br />
terms of <strong>learning</strong> outcomes, i. e. what a student is expected to achieve in terms of knowledge,<br />
understanding, skills and competencies. The (virtual) classroom should be a place for knowledge<br />
construction and exchange of knowledge between members of the group, the teacher should be a<br />
motivator, facilitator and guider of the individual learner progress rather than a transmitter of<br />
knowledge and the student should interact with his group and learn in an autonomous way rather than<br />
perceive contents passively (Kelz 2008). A lot of tantalising challenges which needed and partly still<br />
need to be overcome.<br />
From the very beginning of the implementation of a blended <strong>learning</strong> approach at Campus Pinkafeld a<br />
strategic paper was written by an appointed blended <strong>learning</strong> coordinator and made available to all<br />
part-time teachers. The paper is divided into two main sections, a theoretical part on the overall<br />
organisational and pedagogic design of the new part-time programmes and a practical part which<br />
outlines how to actually design and implement blended <strong>learning</strong> activities. The latter focuses on<br />
student requirements, modularisation, <strong>learning</strong> and teaching strategies, practical approaches to<br />
<strong>learning</strong> and teaching, ICT and <strong>learning</strong> management systems, student assessment, course<br />
evaluation and teacher development. As we didn’t expect teachers to read the full 47-page strategic<br />
paper a much shorter teacher manual was created and provided for download. The main contents<br />
were also published as a <strong>learning</strong> module on our LMS, providing further information and links to<br />
relevant pages. The strategic paper and teacher manual were presented and discussed in teacher<br />
workshops alongside practical activities to plan and implement blended <strong>learning</strong> activities.<br />
Furthermore, teacher workshops on pedagogic and didactic issues in general and blended <strong>learning</strong><br />
scenarios in particular have been offered on a regular basis for the last seven years. These<br />
workshops are held by experienced blended <strong>learning</strong> trainers and are freely accessible to all<br />
members of the teaching staff (full-time teachers, private tutors, freelancers etc.).<br />
3. Learning management system<br />
Currently there are two teachers responsible for the implementation of the blended <strong>learning</strong> strategy<br />
pursued at Campus Pinkafeld. They spend up to half of their working hours to help teachers design<br />
and implement blended <strong>learning</strong> activities, train students and teachers on how to use the information,<br />
communication, and collaboration tools of the platform provided and give teacher workshops on how<br />
to use its authoring tools to design online <strong>learning</strong> materials and activities.<br />
The primary vehicle for the delivery of our blended <strong>learning</strong> activities is the open-source platform<br />
ILIAS which has been in use at Campus Pinkafeld since 2005. It is an open <strong>learning</strong> management<br />
system which serves as a knowledge and collaboration platform and supports a variety of uses and<br />
methods and allows to efficiently create <strong>learning</strong> and teaching materials by both teachers and<br />
students. As ILIAS is versatile, some more Austrian universities deploy it as a campus-wide blended<br />
<strong>learning</strong> system in which online modules and courses complement classes (e. g. Pedagogical<br />
University Vorarlberg and University of Applied Sciences Upper Austria). For further information<br />
please got to http://www.ilias.de.<br />
In order to fully utilize the advantages of ILIAS workshops to help lecturers use the online platform in<br />
a meaningful and efficient way are offered at the beginning of each term. Besides, all lecturers of the<br />
technical part-time degree programmes are encouraged to share resources and materials in an open<br />
manner. For this purpose online workspaces particularly designed for the lecturers of technical parttime<br />
programmes were designed which allow communication, exchange of information, and<br />
collaboration between all members of the teaching staff.<br />
4. Teacher and student training<br />
Technology itself, however, doesn’t mean more efficient teaching and improved <strong>learning</strong>. Effective<br />
blended <strong>learning</strong> settings proved to require a lot of commitment of the teaching staff and a lot more<br />
364
Andrea Kelz<br />
self-discipline and motivation of students than traditional face-to-face teaching. Moreover, not all<br />
lecturers and students are convinced of a constructivist-based blended <strong>learning</strong> framework from the<br />
start. The major cause lies in our partly still traditional educational system which emphasizes content<br />
memorization and individual student performance. Particularly older lecturers and part-time students<br />
in their thirties have hardly been exposed to web-based constructivist and collaborative teaching and<br />
<strong>learning</strong> techniques which are essential components of our blended <strong>learning</strong> approach. They haven’t<br />
been trained in constructivist and collaborative methods and so the pencil- and paper-based lecture<br />
method is still the method of their choice. From the teachers’ point of view the main reasons for<br />
concern have been that they fear loss of control in the lecture if they give more responsibility to the<br />
students for their <strong>learning</strong> progress. It takes a great deal of self-confidence in oneself and one’s<br />
students to transfer the responsibility of <strong>learning</strong> to the student. Some also fear the loss of content<br />
coverage because allowing group interaction often takes longer than giving mere lectures. There have<br />
also been concerns when it comes to assessing students’ efforts. Some teachers are unfamiliar with<br />
continuous assessment techniques or how to assess group efforts. Another major cause for concern<br />
lies in some students’ resistance to collaborative <strong>learning</strong> techniques. They also still need to become<br />
comfortable with new methods of teaching and <strong>learning</strong> and express concerns and doubts in the<br />
beginning. Particularly collaborative assignments are initially viewed critically. Students often express<br />
concerns about lacks of contribution of fellow students of the same group or complain about<br />
interpersonal conflicts they have to deal with. As they are not likely to change their attitudes to<br />
<strong>learning</strong> from one course to another they also gradually need to become acquainted with<br />
constructivist and collaborative <strong>learning</strong> techniques.<br />
As indicated above, both students and teachers feel that they are in a new <strong>learning</strong> situation. With<br />
regard to student needs the successful implementation of blended <strong>learning</strong> designs include knowing<br />
about their own <strong>learning</strong> styles and needs and helping them improve on how to learn successfully,<br />
i. e. helping them develop a repertoire of <strong>learning</strong> skills and strategies to support their development as<br />
learners in virtual <strong>learning</strong> environments. In order to enable technical part-time students to engage<br />
actively in the management and design of their own <strong>learning</strong> a blended <strong>learning</strong> course was designed<br />
to make them reflect on their <strong>learning</strong> styles and processes, help them learn how to learn, enhance<br />
their creative and innovative thinking and eventually improve on their overall performance. In this<br />
course the LMS provided as well as other open web-resources are used for individual and<br />
collaborative e-<strong>learning</strong> experiences to make them acquainted with various <strong>learning</strong> and teaching<br />
styles they will be exposed to during their studies.<br />
As part of the teachers are also not prepared for blended <strong>learning</strong> settings, particularly those who<br />
haven’t had any e-teaching or e-<strong>learning</strong> experiences themselves, they also need support. Due to the<br />
lack of first-hand experiences some of them try to teach in the same old way simply utilising some<br />
technologies and calling it blended <strong>learning</strong> or e-<strong>learning</strong>. They now have to face the challenge of<br />
having to update their knowledge and skills to be able to make appropriate use of ICT in their courses<br />
and learn how to utilise virtual <strong>learning</strong> environments in ways which best meet the needs of their<br />
students as well as the current challenges of higher education such as the move towards lifelong<br />
<strong>learning</strong> with its demand for continuous professional development.<br />
The question arises of what it actually is that the teacher needs to do to make a success of blended<br />
<strong>learning</strong> settings. What competencies and skills are required to teach in a virtual <strong>learning</strong> environment<br />
effectively and how will the teachers eventually make the change? Smith identifies and describes 51<br />
instructor competencies needed by online instructors and outlines the “key components of a training<br />
program to enable a traditional brick-and-mortar college to transition to a university that has a<br />
significant online component” (Smith 2005). He outlines a variety of competencies needed prior to the<br />
start of a course, competencies needed during the course and competencies needed after the course.<br />
To sum up roughly, the strategy for training our teachers at Campus Pinkafeld also implies that in a<br />
successful blended <strong>learning</strong> setting the teacher must have sufficient ICT, pedagogical and social<br />
skills. The particular skills required are being able to use the technology (LMS) provided, skills to<br />
design and implement online activities, organization and moderation skills, being able to integrate<br />
different teaching and <strong>learning</strong> styles, interact actively and give students constant feedback.<br />
But how are teachers supposed to modify their practice to work in this new environment? First of all,<br />
all teachers are advised to start simple and move towards the more complex. These gradual<br />
enhancements range from supporting students via e-mail, moderating online discussion forums,<br />
uploading files onto the platform, providing further materials and resources on the web, to integrating<br />
365
Andrea Kelz<br />
web-based activities into the course, developing and implementing online collaborative activities and<br />
authoring online course materials. In order to do this they need theoretical and practical training,<br />
which is provided by the campus. Teacher training workshops are conducted on a regular basis by<br />
our blended <strong>learning</strong> administrator and are supposed to help teachers update their ICT skills and<br />
improve their blended <strong>learning</strong> teaching designs, explore didactic strategies with their peers and<br />
generate new ideas and ways to innovate their teaching.<br />
Finally, for all courses taught at Campus Pinkafeld the teachers are required to provide a course<br />
outline which was designed to help them with the preparation and delivery of the courses. These<br />
course outlines do not only document the curriculum at the course level, they also support the<br />
teaching and <strong>learning</strong> process by identifying specific course objectives, contents, activities, resources,<br />
and evaluation methods. The outline also ensures that the purpose of the course is clear not only to<br />
the student but also to the teacher as it involves asking questions about student requirements, course<br />
objectives and its context within the overall curriculum. Particularly working with part-time students<br />
who mostly work alongside their studies and thus often have extensive work experience requires<br />
asking questions such as what knowledge and skills do the students already bring to the course and<br />
what do learners actually still need to learn or be able to do in order to succeed in their future work<br />
environments. In addition, it is important to think about the teaching resources already available and<br />
blended <strong>learning</strong> material and activities which still need to be developed. Brainstorming about such<br />
questions has proved to help set the priorities for course planning and delivery.<br />
Hence, prior to each course, the teachers are provided a course outline form which he/she is<br />
supposed to fill in and upload on the platform for everyone to see. The outline already provides<br />
general information on the course such as its number, title, name of the instructor and contact<br />
information, main course contents according to the curriculum, instructional method such as lecture,<br />
seminar, laboratory etc., and student workload. The instructor is required to add the following<br />
information:<br />
Short description of overall course objective/<strong>learning</strong> outcomes: statement of the knowledge, skills<br />
and competencies the students are expected to have acquired by the end of the course.<br />
List of required and recommended literature.<br />
Information on each face-to-face session as well as blended <strong>learning</strong> activities including short<br />
description of <strong>learning</strong> objectives, contents covered, <strong>learning</strong> material provided, description of<br />
required student tasks with due date and submission mode.<br />
Detailed description of assessment mode: statement of what percentage of the final grade each<br />
assignment and exam will represent.<br />
5. Student evaluation of blended <strong>learning</strong> activities<br />
All course evaluations are carried out by means of EvaSys at the end of each course. EvaSys allows<br />
for student feedback of the different types of part-time courses (lectures, seminars, laboratory<br />
exercises etc.) and their blended <strong>learning</strong> activities. For further information on EvaSys please go to<br />
http://www.electricpaper.biz/products/evasys-education.html. For each course the students fill in a<br />
course questionnaire evaluating student satisfaction in the following areas: course administration,<br />
course design, course contents, teacher, blended <strong>learning</strong> activities and overall impression of the<br />
course.<br />
With regard to blended <strong>learning</strong> modules there are three questions to be answered:<br />
There has been sufficient time to work on the blended <strong>learning</strong> assignments.<br />
The blended <strong>learning</strong> activities have been supported sufficiently by the teacher.<br />
The <strong>learning</strong> management system has been used efficiently by the teacher.<br />
The possible ratings range from ‘1’ (fully agree) to ‘5’ (completely disagree). The institutional results<br />
provided by the head the bachelor’s degree programme in energy and environmental management<br />
and the master’s degree programme in building technology and management are as follows:<br />
The tables show that there have been overall continuous improvements of the ratings over the years.<br />
The ratings of the bachelor’s degree students, however, are a bit weaker than those of the master’s<br />
degree students. One reason might be that the master’s students, most of whom completed the<br />
366
Andrea Kelz<br />
bachelor’s programme in energy and environmental management at Campus Pinkafeld, are already<br />
more familiar with our blended <strong>learning</strong> approach. The same holds true for the teachers as some of<br />
them also teach in the bachelor’s degree programme. The ratings are quite good but there still<br />
remains room for improvement, particularly when it comes to designing collaborative, web-based<br />
activities and making more effective use the <strong>learning</strong> management system in terms of creating<br />
interactive <strong>learning</strong> materials. Moreover, there will have to be some sort of qualitative assessment of<br />
blended <strong>learning</strong> materials and activities on an institutional level.<br />
Table 1: Ratings of bachelor’s degree programme in energy and environmental management<br />
Average student ratings of blended <strong>learning</strong> modules (all semesters/135 students)<br />
Winter Term Summer Term Winter Term Summer Term<br />
2008/2009 2009<br />
2009/2010<br />
2010<br />
Sufficient time to work on the<br />
blended <strong>learning</strong> assignments<br />
1.65 1.42 1.43 1.40<br />
Sufficient monitoring and<br />
support by the teacher<br />
1.68 1.51 1.47 1.40<br />
Efficient use of the <strong>learning</strong><br />
management system (ILIAS)<br />
1.76 1.60 1.54 1.50<br />
Table 2: Ratings of master’s degree programme in building technology and management<br />
Average student ratings of blended <strong>learning</strong> modules (all semesters/60 students)<br />
Sufficient time to work on the<br />
blended <strong>learning</strong> assignments<br />
Sufficient monitoring and<br />
support by the teacher<br />
Efficient use of the <strong>learning</strong><br />
management system (ILIAS)<br />
Winter Term Summer Term Winter Term Summer Term<br />
2008/2009 2009<br />
2009/2010<br />
2010<br />
1.53 1.25 1.29 1.20<br />
1.43 1.32 1.21 1.10<br />
1.54 1.35 1.35 1.30<br />
6. Nationwide training and initiatives<br />
About ten years ago a nationwide initiative was started by the Austrian government to expand the use<br />
of e-<strong>learning</strong> and blended <strong>learning</strong> at Austrian universities. The programme “New Media in Teaching<br />
at Universities and Universities of Applied Sciences” was started and provided the basis for a cooperation<br />
of all Austrian universities in an association called “Forum Neue Medien Austria” (Forum<br />
New Media Austria). This forum sees itself as a lobbyist for innovation in teaching in higher education.<br />
Their interactive information and communication platform, conferences and newsletters encourage<br />
scientific discourse and are open to everybody who is interested in e-<strong>learning</strong>. At present the network<br />
of the association encompasses 21 Austrian universities and 16 universities of applied sciences. This<br />
association has been working on general ICT policies and e-<strong>learning</strong> issues, especially issues of<br />
quality management, legal questions in e-<strong>learning</strong>, e-<strong>learning</strong> and career development and e-content<br />
development. For further information please go to http://www.fnm-austria.at. Furthermore, in 2005 the<br />
Austrian ministry of education developed a framework paper for an e-<strong>learning</strong> strategy and funded<br />
projects of universities which aimed at developing their individual institutional e-<strong>learning</strong> and eteaching<br />
strategies. Within this initiative nine Austrian universities of applied sciences developed<br />
components for implementing e-<strong>learning</strong>-strategies in a cooperative way.<br />
Meanwhile all Austrian higher education institutions provide some sort of virtual <strong>learning</strong> environments<br />
to support and facilitate <strong>learning</strong>. There are staff members assisting and advising lecturers in<br />
didactical matters as well as in the technical implementation of e-<strong>learning</strong> and blended <strong>learning</strong><br />
concepts and scenarios. They provide support and development in areas such as the didactical<br />
design and production of online <strong>learning</strong> materials, diagrams and animations, the programming of<br />
interactive <strong>learning</strong> software, the production of training videos and automated self-control questions<br />
with feedback, as well as the continual adjustment of the e-<strong>learning</strong> and blended <strong>learning</strong> courses and<br />
the <strong>learning</strong> management system in use. As far as <strong>learning</strong> management systems are concerned,<br />
only recently a web-survey has been conducted, trying to find out what platforms are used at different<br />
universities of applied sciences in Austria. The survey obtained 16 replies. The results so far provided<br />
to the responding universities/campuses show that 10 out of 16 universities of applied sciences in<br />
Austria use Moodle to support online <strong>learning</strong>. One of the reasons of course is that the Austrian<br />
ministry of education supported the installation and service of this open source <strong>learning</strong> management<br />
367
Andrea Kelz<br />
system. Most other universities taking part in the survey also use open source solutions such as ILIAS<br />
or some kind of combination of in-house and open source <strong>learning</strong> management systems.<br />
While competence building in e-<strong>learning</strong> and blended <strong>learning</strong> at universities still mostly takes place<br />
autonomously, the following universities are jointly offering a nationwide “E-Learning Certificate”: the<br />
Universities of Innsbruck, Salzburg, Linz, Graz and Klagenfurt; the University of Business<br />
Administration Vienna, the Medical University Graz, the University of Arts in Linz, the Universities of<br />
Applied Sciences of Upper Austria, Salzburg, Vorarlberg and Kufstein, the University College Tyrol,<br />
the Education Service Tyrol and the Management Centre Innsbruck. The course programme consists<br />
of three modules which are offered at beginners’ and advanced levels. Module 1 comprises media<br />
didactics in teaching at universities (use of media in teaching, installation and use of e-<strong>learning</strong><br />
courses in LMS systems and collaborative e-<strong>learning</strong>), module 2 focuses on media production (digital<br />
photography, e-<strong>learning</strong> and law, e-<strong>learning</strong> and web 2.0) and module 3 covers questions such as<br />
media organisation and e-moderation. If participants have completed all three modules at a minimum<br />
of two course providers within 18 months they can apply for an e-<strong>learning</strong> certificate. The certificate<br />
course is understood as a professional development instrument and is offered free of charge. For<br />
further information please go to http://www.zertifikat-e<strong>learning</strong>.at/?home=1.<br />
Finally, the Austrian Federal Ministry of Education, Arts and Culture defined the pedagogical and<br />
didactical use of IT (especially EPICT) as one of several innovative and integrative principles. Taking<br />
account of the principle is demanded in all areas of further education and training and supported by<br />
specific initiatives. EPICT – the European Pedagogical ICT Licence – is a comprehensive, flexible and<br />
efficient in-service training course introducing a European quality standard for the continued<br />
professional development of teachers in the pedagogical integration of information, media and<br />
communication technologies (ICT) in education. The European Pedagogical ICT Licence can be<br />
acquired after successful completion of a minimum of eight modules of which four are compulsory and<br />
four can be chosen out of ten optional modules. The obligatory modules focus on understanding the<br />
pedagogical role of ICT in teaching, acquiring basic ICT skills, searching and communicating with<br />
internet tools, creating and sharing content, and understanding the requirements of bringing<br />
innovation to schools and universities. For further information please go to http://www.epict.org/.<br />
7. Conclusion<br />
We have been witnessing a paradigm change in higher education for quite some time now and like in<br />
most other universities ICT has become integrated into all courses of our degree programmes.<br />
However, as ICT needs to be more than the technology itself and the content it carries, it is not<br />
sufficient to provide technical support and technical training to both students and teachers. As<br />
students and teachers act in both face-to-face and virtual classroom situations they have to build new<br />
<strong>learning</strong> and teaching concepts. Students need to become more autonomous learners and teachers<br />
need to develop the skills to teach blended <strong>learning</strong> courses effectively.<br />
In view of the importance of a successful long-term, campus-wide, blended <strong>learning</strong> strategy we<br />
constantly have to review and analyse the development of our own blended <strong>learning</strong> practices. The<br />
factors critical to the success of blended <strong>learning</strong> approaches are definitely sufficient training and<br />
continual support of the concerned students, teachers and administrators at all levels. Students and<br />
teachers do not necessarily mind change but being made to change and become blended <strong>learning</strong><br />
students and teachers in a new <strong>learning</strong> and teaching environment turned out to be quite a challenge.<br />
So far there has been no direct reward or motivation for members of the teaching staff to make the<br />
effort to incorporate ICT and blended <strong>learning</strong> activities into their courses. The primary incentives<br />
have been intrinsic rewards such as personal professional development in terms of teaching<br />
techniques, positive student feedback and peer recognition. The main problems which still need to be<br />
solved don’t concern the implementation of a blended <strong>learning</strong> approach in terms of ICT, changing<br />
<strong>learning</strong> environments and roles of the students and teachers, but the impact of blended <strong>learning</strong> on<br />
teacher workload. Compared to traditional teaching blended <strong>learning</strong> designs involve more time to<br />
develop activities and proper course materials. So the next question to be tackled is how the teachers’<br />
workload can be measured effectively when it comes to the preparation of online materials and<br />
blended <strong>learning</strong> activities. With many teachers there’s still a sort of unwillingness to experiment with<br />
innovation and the possibilities the use of ICT offers as they do not only lack time but also some sort<br />
of extrinsic motivation. We have to find a way to encourage more faculty members and freelancers to<br />
develop web-based activities. Financial incentives are being discussed at the moment, but so far we<br />
haven’t found any real solution to the problem.<br />
368
Andrea Kelz<br />
Within the last seven years it has proved that the use of ICT requires a continuous process of<br />
development of competencies online teachers should have and demands professional preparation<br />
and proper pedagogical and didactical training. Face-to-face teaching has remained central in our still<br />
campus-based institution and blended <strong>learning</strong> is mainly received as distance <strong>learning</strong>, enriched by<br />
ICT, supplementing on-campus degree programmes. In future even more emphasis will have to be<br />
put on the pedagogy and didactics of blended <strong>learning</strong> as some teachers unfortunately still think that it<br />
is possible to simply transfer their teaching material and activities used in the classroom to the web.<br />
Many teachers still have to learn how to cope with the role of becoming facilitators in the <strong>learning</strong><br />
process and have to realize that a <strong>learning</strong> management system is more than a digital library to store<br />
and provide <strong>learning</strong> materials. Only then they will stop to wonder why no one uses the discussion<br />
forums they provided in their courses.<br />
References<br />
Apel, Heino et al (2003) Online Lehren. Planung und Gestaltung netzbasierter Weiterbildung, Bertelsmann,<br />
Bielefeld.<br />
EvaSys – Education Survey Automation Suite (2011) [online], Electric Paper Official Website,<br />
http://www.electricpaper.biz/products/evasys-education.html.<br />
European Pedagogical ICT Licence (2011) [online], EPICT Official Website, http://www.epict.org/concept.shtml.<br />
Forum Neue Medien Austria (2011) [online], FNM Official Website, http://www.fnm-austria.at/Home/Home/.<br />
Garrison, D. Randy and Vaughan, Norman D. (2008) Blended Learning in Higher Education: Framework,<br />
Principles, and Guidelines, John Wiley & Sons, San Francisco.<br />
Gierke, Christiane et al (2003) Vom Trainer zum E-Trainer. Neue Chancen für den Trainer von morgen, GABAL,<br />
Offenbach.<br />
ILIAS Open Source e-Learning (2011) [online], ILIAS Official Website,<br />
http://www.ilias.de/docu/goto_docu_cat_580.html.<br />
Kelz, A. (2008) “E-Learning Strategies in Technical Part-Time Studies – Constructivist and Collaborative<br />
Approaches to Learning and Teaching”, 7 th European Conference on e-Learning, Volume 1, APL, Reading,<br />
pp 557-563.<br />
Reinmann-Rothmaier, Gabi (2003) Didaktische Innovation durch Blended Learning. Leitlinien anhand eines<br />
Beispiels aus der Hochschule. Huber Psychologie Praxis. Lernen mit neuen Medien, Verlag Hans Huber,<br />
Bern.<br />
Smith, T. C. (2005) “Fifty-One Competencies for Online Instruction”, [online], The Journal of Educators Online,<br />
Volume 2, Number 2, July 2005, http://www.thejeo.com/Ted%20Smith%20Final.pdf.<br />
Walitsch, M. (2008) “Successful Blended Learning Concept for Non-Traditional University Access in Austria”,<br />
Proceedings of the IADIS International Conference on e-Learning 2009, Algarve, Portugal 17-20 June 2009,<br />
pp 253-256.<br />
Zertifikat eLearning (2011) [online], Official Website, http://www.zertifikat-e<strong>learning</strong>.at/?home=1.<br />
369
Open Courses: The Next big Thing in eLearning?<br />
Kaido Kikkas 1, 2 , Mart Laanpere 2 and Hans Põldoja 2<br />
1 Estonian Information Technology College, Tallinn, Estonia<br />
2 Institute of Informatics, Tallinn University, Tallinn, Estonia<br />
kaido.kikkas@itcollege.ee<br />
mart.laanpere@tlu.ee<br />
hans.poldoja@tlu.ee<br />
Abstract: During the last 15 years, eLearning has undergone a number of changes regarding openness of the<br />
<strong>learning</strong> environment, <strong>learning</strong> resources as well as the process of teaching and <strong>learning</strong>. After the initial period,<br />
when eLearning used mostly the tools of 'ordinary' Web, the first-generation of eLearning emerged – large,<br />
mostly proprietary environments which firmly separated the chosen (students and tutors) from the 'barbarians at<br />
the gates' (the rest of the Internet) by using accounts and passwords. The tools themselves were shaped by<br />
creators, not users – due to closed source and restrictive licenses. WebCT and early Blackboard were prime<br />
examples of this generation. The second generation of eLearning rode the wave of free and open-source<br />
software, bringing along a much greater variety of tools as the environments became open for modifications (e.g.<br />
Moodle, Ilias). This generation also emerged along with the rise of open content (powered by free licenses like<br />
the Creative Commons family) which in turn established the Open Educational Resources (OER) movement,<br />
examples of which are Connexions, LeMill, MIT OpenCourseWare and others. We propose that the emergence<br />
of Personal Learning Environments combined with collaborative wikis signify the next stage of eLearning. Open<br />
in both the environment and the process, it facilitates a more flexible and also more challenging model of<br />
<strong>learning</strong>. Massive Open Online Courses (MOOC) represent the radical conception of openness in eLearning, as<br />
all MOOC courses are open for enrolment for any Internet user. We started to use Wikiversity and personal blogbased<br />
<strong>learning</strong> environments in 2008. This paper provides a summary of our experience with teaching 18 open<br />
courses in 2008-2011, with more than 560 enrolled students. We analyse the design, <strong>learning</strong> process and<br />
<strong>learning</strong> outcomes of these open courses using the knowledge building theory by Bereiter as our main frame of<br />
reference and the framework analysis of the courses.<br />
Keywords: open courses, personal <strong>learning</strong> environment, wiki, blog<br />
1. eLearning: a glance on history<br />
During the last 15 years, the mainstream practices of eLearning have undergone a number of<br />
changes with regard to openness of the <strong>learning</strong> environment, <strong>learning</strong> resources as well as the<br />
process of teaching and <strong>learning</strong>.<br />
Leinonen (2005) sees the main phases of using computers in <strong>learning</strong>. Various other authors have<br />
suggested slightly different versions of the timeline, but it seems to be rather close to the current<br />
consensus. However, one of the main roots of eLearning – an orderly, top-down approach rather than<br />
a distributed, community-based activity – is clearly visible here. This is likely the reason why the first-<br />
generation eLearning systems shared the same mindset.<br />
Thus after the initial starting period, when eLearning used mostly the tools of 'ordinary' Web, the firstgeneration<br />
eLearning emerged – large, mostly commercial and proprietary environments which firmly<br />
separated the chosen (students and tutors) from the 'barbarians at the gates' (the rest of the Internet)<br />
by using accounts and passwords. The tools themselves were shaped by creators, not users – due to<br />
closed source and restrictive licenses. The popular <strong>learning</strong> management systems (LMS) like WebCT<br />
and early Blackboard were prime examples of this generation.<br />
As Leinonen (2005) points out, these early systems inherited the mindset of the previous, computer-<br />
based training phase – the same top-down, teacher-to-student, low-interaction approach. Even<br />
though the later LMS's became more aware that <strong>learning</strong> requires social activities among the learners<br />
themselves and the learner and the teachers, these were often added as an afterthought.<br />
As first pointed out by David Wiley in 1998 (OECD 2007), the second generation of eLearning rode<br />
the wave of free and open-source software (FOSS). Wiley concluded that a lot of principles that<br />
applied to software were also applicable to other online content, resulting in the birth of the first free<br />
content license in OPL (Open Publication License) and the subsequent Open Content movement<br />
which in turn gave rise to the wider Free Culture movement as described by Lessig (2004).<br />
370
Kaido Kikkas et al.<br />
The term 'open educational resources' (OER) was born at the beginning of the 21st century, along<br />
with the establishment of Creative Commons (the organisation and their novel system of content<br />
licensing) in 2001 as well as the decision of the Massachusetts Institute of Technology (backed by the<br />
Hewlett Foundation and the Mellon Foundation; OECD 2007) to make their course materials<br />
available online. The term 'OER' was first used the following year (Johnstone 2005).<br />
On the one hand, the OER movement had notable influence on LMS's, promoting the next generation<br />
of eLearning tools where the environment was also open for modifications (e.g. Moodle, IVA, Ilias).<br />
On the other hand, it gave rise to dedicated online repositories like Connexions, LeMill, MIT<br />
OpenCourseWare and others. Still, while the technical, environmental side became more distributed<br />
and open, the methodological side widely remained top-down and restricted (e.g. even if a FOSS<br />
environment like Moodle was used, the course in it was still locked to outsiders).<br />
During the recent years, the emergence of Personal Learning Environments combined with<br />
collaborative wikis signify the next stage of eLearning. Open in both the environment and the process,<br />
it facilitates more flexible and also more challenging model of <strong>learning</strong>. As described by Fini (2009),<br />
Massive Open Online Courses (MOOC) represent the radical conception of openness in eLearning,<br />
as all MOOC courses are open for enrolment for any Internet user.<br />
2. Open courses<br />
While the most visible outcome of the Open Education movement are numerous OER repositories<br />
and OpenCourseWare initiatives we must point out that open education has a much wider meaning<br />
than just openly available educational content. According to the Cape Town Open Education<br />
Declaration (2007), “open education is not limited to just open educational resources. It also draws<br />
upon open technologies that facilitate collaborative, flexible <strong>learning</strong> and the open sharing of teaching<br />
practices that empower educators to benefit from the best ideas of their colleagues. It may also grow<br />
to include new approaches to assessment, accreditation and collaborative <strong>learning</strong>.”<br />
In recent years there has been a rising discussion about open educational practices (Ehlers, 2011).<br />
Ehlers defines open educational practices as “practices which support the (re)use and production of<br />
OER through institutional policies, promote innovative pedagogical models, and respect and empower<br />
learners as co-producers on their lifelong <strong>learning</strong> path.” Online courses with open enrollment could<br />
be seen as an example of open educational practices.<br />
The first widely known open course was Introduction to Open Education course that was organized in<br />
autumn term 2007 by David Wiley (Wiley, 2007). This course inspired several other educators to open<br />
up their courses with free enrollment. In spring term 2008 Teemu Leinonen and Hans Põldoja<br />
organized an open course about composing free and open online educational resources. This course<br />
was special because Wikiversity was used for running the course (Leinonen, Vaden and Suoranta,<br />
2009). Both of these courses were official university courses that were made open for external<br />
participants. The requirements for institutionally enrolled students and informally participating students<br />
were the same, but only institutionally enrolled students received credits for the participation.<br />
Most well-known open courses have been organized by George Siemens and Stephen Downes. Their<br />
courses include Connectivism and Connective Knowledge (CCK08, CCK09, CCK11), Personal<br />
Learning Environments Networks and Knowledge (PLENK2010), and Learning and Knowledge<br />
Analytics (LAK11). These courses had less formal requirements for participation. The participants<br />
were free to choose to what extent they will participate in the course activities. This resulted a large<br />
number of enrolled participants from which only a small part were active. About 2200 participants<br />
enrolled to CCK08 course, about 700 to CCK09 and about 1700 to PLENK2010 (Downes, 2011).<br />
Because of these large numbers the participants started to call these courses massive open online<br />
courses (MOOC) (Fini, 2009).<br />
There are other examples of open courses where the number of participants is limited. P2PU is an<br />
initiative that runs short 6 week online courses that have typically about 20 seats (P2PU, 2011). Small<br />
number of motivated participants guarantees that they will all receive feedback from the facilitator and<br />
their peers.<br />
371
Kaido Kikkas et al.<br />
However, in the Estonian context we can not speak about “massive open online courses” because the<br />
number of non-credit participants that enroll to the courses is typically rather small. Therefore we will<br />
use the term “open course” in this paper.<br />
3. The Wikiversity experience<br />
In our case we have used Wikiversity to provide open courses. Wikiversity is one of the projects under<br />
Wikimedia Foundation that also governs Wikipedia. It is somewhat less structured than Wikipedia,<br />
supporting <strong>learning</strong> communities, their <strong>learning</strong> resources and <strong>learning</strong> activities. In our courses<br />
Wikiversity is typically used for course syllabus, participant enrolment and <strong>learning</strong> resources. All the<br />
participants have their personal blogs to submit their course assignments. This kind of use of wikis<br />
and blogs is inspired by open courses organized by David Wiley (2007) and Teemu Leinonen<br />
(Leinonen, Vaden and Suoranta, 2009).<br />
We claim that in our daily teaching practice, open courses have evolved beyond the initial<br />
experimentation phase and reached the high level of scalability. From 2009 to 2011, we have run<br />
(separately from each other) more than 20 full-scale open courses in three different higher education<br />
institutions in Estonia:<br />
Social Software and Network Communities (SSNC), 4 ECTS M.Sc. course with 226 participants<br />
(total for all years).<br />
Ethics and Law in New Media (ELNM), 5 ECTS international M.Sc. course with 44 participants.<br />
Ethical, Social and Professional Aspects of Information Technology (ESPAIT), a 4 ECTS<br />
introductory-level B.Sc. course with 161 participants<br />
Standards and Specifications for eLearning Tools (SSET), 4 ECTS M.A. course with 20<br />
participants<br />
Learning Environments and Learning Networks (LELN), 3 ECTS M.A. course with 28 participants.<br />
Creating Digital Learning Resources (CDLR), 3 ECTS M.A. course with 29 participants<br />
Working with Sources, Referencing and Reference Management in ELearning (WSRRME), a 10 -<br />
week, 100% online professional development course for teachers and educational technologists,<br />
38 participants.<br />
Creating and Using Open Educational Resources (CUOER), a 10-week, 100% online course for<br />
teachers and educational technologists, 31 participants.<br />
Didactics of Informatics, a 4 ECTS M.A. course with 42 participants.<br />
Basics of Instructional Design, a 3 ECTS M.A. course with 45 participants.<br />
Active <strong>learning</strong> in Informatics Lessons, a 3 ECTS M.A. course with 26 participants.<br />
Professional Development with ePortfolio, a 12-week. 100% online professional development<br />
course for teachers, 12 participants<br />
M-Learning, 12 week long 100% online professional development course for teachers, 10<br />
participants<br />
As Wikiversity-based courses can be run in a variety of ways, the brief descriptions of two possible<br />
approaches used by us wll follow below.<br />
3.1 Approach 1: Wiki-centred course designs<br />
Each student was expected to have a blog (RSS feed for posts was required – both free services like<br />
Blogger or WordPress and standalone installations were allowed), a Skype account and an e-mail<br />
address. Also at the SSNC and ELNM courses, they also needed a wiki space (either using a free<br />
service like Wikidot.com or setting up their own) for teams of 4-5.<br />
372
Kaido Kikkas et al.<br />
A typical task set included weekly 'lecture' texts (1 or 2 per week; available from the course page at<br />
Wikiversity), a weekly blogging task at the end of the lecture text (to be completed by the next<br />
Monday noon) and a weekly Skype text chat meeting (at SSNC and ELNM; lasting typically 1-1.5<br />
hours). In addition, everyone was to post to the course forum following the weekly topic, but this task<br />
did not have fixed weekly regimen. Finally, the SSNC and ELNM had to write a wiki-based group<br />
paper and review another group's work (as an individual blog post).<br />
The SSNC and ELNM courses lacked formal examination, the ESPAIT featured an optional written<br />
exam (a 'backup' option for less active people or also an extra way to raise the grade – but it was<br />
possible to earn the highest grade without it). Thus, the SSNC and ELNM did not have the 'end rush'<br />
at all, while at the ESPAIT it was much milder than is typically seen at traditional courses.<br />
The exact graduation systems varied somewhat by course, but a typical one consisted of:<br />
40% - the wiki-based group paper (or in the case of ESPAIT, a written exam essay)<br />
35% - blogging tasks (typically compact pieces of opinion, ~1 page each)<br />
25% - forum posts (of substance – the 'me too!'-type and offtopic were typically not counted in)<br />
10% - weekly Skype chats (for consulting as well as community building).<br />
Note: the course point system featured a ~10% surplus, designed to give students some choice.<br />
3.2 Approach 2: Blog-centred course designs<br />
These courses were organized as blogging seminars with weekly topics and assignments. New<br />
assignments were given every Monday, the students had to write a blog post by the end of the week.<br />
In addition to Wikiversity, a course blog was used in all courses. Wikiversity was used for course<br />
syllabus and enrolments while weekly assignments and summaries were published in the course<br />
blog.<br />
The 10-week online courses WSRRME and CUOER included a video conference during the first and<br />
the last week of the course. Weeks 5 and 10 were left without assignments so that people could<br />
complete the assignments they had not managed to do in time. In case of Master level courses there<br />
were also 4...6 classroom meetings during the course. Besides individual blogging, the Master level<br />
courses required substantial group work. Students typically used a shared blog, a wiki or Google<br />
Docs to complete the group assignment.<br />
To emphasize openness, the <strong>learning</strong> environment consisted mostly of tools that were open source or<br />
provided with free hosting. The course blog was hosted at WordPress.com, the <strong>learning</strong> resources<br />
were published at LeMill and SlideShare and links were shared through Delicious. In the CUOER<br />
course also Skype was used for text chat. In 2010 it was decided to use Twitter instead of Skype for<br />
rapid communication. However, in Estonian context Twitter is not still actively used by the educational<br />
community. Another service that was added in 2010 was Mendeley, used to share references to<br />
research papers. The only closed tool used during the courses was FlashMeeting that was used for<br />
video conferences in the WSRRME and CUOER courses.<br />
A complicated issue in open courses is to manage the list of participants and to aggregate all their<br />
blog posts. In the first courses, Wikiversity was used for course enrolments and blog posts were<br />
aggregated using Pageflakes. This required a lot of manual work from the facilitator. That led us to<br />
design and develop the EduFeedr tool that handles course enrolments, manages assignments and<br />
aggregates all the blog posts (Põldoja, 2010). Since 2010 EduFeedr has been used for managing the<br />
courses and Wikiversity is used only for course syllabus that can be collaboratively edited.<br />
4. Discussion<br />
To start with, we have to agree with Seely Brown and Adler (2008) in that the world is becoming both<br />
flat<br />
(in that the virtual communication has eliminated many obstacles by reaching almost any place in the<br />
world) and spiky (in that the local level of productivity and innovation will determine the success of a<br />
location, and the differences will increase). Thus, it is necessary to not only supply the next<br />
373
Kaido Kikkas et al.<br />
generation of workforce but also do it dynamically and provide flexible environment to facilitate it<br />
(various authors like Benkler, Barnes, Himanen, Theobald and others also point to that direction).<br />
We used the Ritchie and Spencer (1994) framework analysis method to study the feedback gathered<br />
from the participants of open courses. The data originates from two sources - transcripts of the weekly<br />
chat sessions and the reflections on <strong>learning</strong> experiences collected from ELNM and SSNC students<br />
after the end of the course in 2009 and 2010. After familiarizing ourselves with the collected data, we<br />
identified five emerging themes. Eventually, we indexed and charted the sections of the data using<br />
EverNote. These five themes are listed below with some samples of data belonging to each theme.<br />
1. Enlightening experience: open course as a new way to learn together<br />
The first aspect clearly visible at the courses was their novel form for students. While a number of<br />
them were familiar with eLearning, almost nobody had any previous experience with open courses.<br />
This resulted in technical questions (“"Are the blogs bundled up somewhere?"), some doubts in the<br />
overall process ("Do we have to be constructive? :P") as well as problems with time management<br />
probably familiar from all e-courses (“What was the deadline? Sorry, I was in Finland last week").<br />
However, the overall feedback reflects the courses being successful. The initial awkwardness ("This<br />
course is my nightmare :)") was replaced by joy ("Loved it!” - the same person after graduation). The<br />
feedback evaluated the courses very well (e.g. the SSNC 2010 received 92 points out of 100) and<br />
also suggested good overall quality of the course ( "And as educational technologist, I now know why<br />
some courses get the quality certificate and some don't :)”, "I enjoyed this course a lot, probably one<br />
of the best this semester. Thanks, Kaido, and hope to take more of your courses next semesters.").<br />
Another point was the ability of the facilitators to provide continuous feedback ("Thanks for the<br />
immense work you did in weekly grading of tasks. It was an excellent course.").<br />
2. The culture of sharing: daring to write in public<br />
Writing in public was not very easy to adopt either - reasons were various like authorship concerns (“I<br />
firmly believe that one's intellectual property is an asset and should be addressed as one”), lack of<br />
time (“Blogging proved to be very difficult with such a hectic schedule"), organisational issues ("Are<br />
references and such necessary?", “Do you REALLY read ALL the blog posts?") and difficulties in<br />
formulating one’s opinion in writing (“The topics themselves are interesting enough and the main<br />
obstacle may be how to stop thinking and just to write something in short words.").<br />
However, several people pointed out that the publicity in fact works as a good method for quality<br />
assurance, stimulating them to produce quality output ("My friend asked me about that mysterious<br />
course which makes me write such interesting things in my blog.").<br />
3. Real-time chat as a community tool: what it gives and what it takes<br />
The weekly real-time chat sessions (we mostly used Skype in text mode) was received well by the<br />
participants ("Thanks folks! It was fun. I'll look forward to the next chat") - among the outlined features<br />
were immediateness and interactivity compared to other componentes of the course (“What are we<br />
discussing? Or isn't it such a structured conversation at all?") as well as the wide range of topics<br />
discussed (“"One of the best courses. and that’s because I expected those Wednesdays.a bit scary<br />
and thrilling at same time and so interesting!"), although there were questions about privacy ("Will we<br />
really have discussions that we won't dare to share? The course topic would imply openness") or<br />
simply the need to be active in discussion (“Will we get credit for speaking or just being present?”).<br />
4. Community gravity: nurturing the social ties within the group<br />
The community gravity which measures how strongly a user might be attracted to a community was<br />
first researched by Matsuo and Yamamoto (2009). We find the concept to be applicable also in the<br />
context of eLearning (see also Väljataga and Põldoja, 2011). We suggest that open courses do<br />
possess a good potential for strong community gravity.. Of the factors influencing the gravity, relaxed<br />
atmosphere and a modest level of formality (“"Sorry, it is easier and more interesting than I thought"),<br />
informativeness (not strictly limited to the course programme ("Thanks - I'd have never heard about<br />
Pekka Himanen without this course", but also "So, let's start with Angelina Jolie and her part in "The<br />
374
Kaido Kikkas et al.<br />
Hackers""), changes in requirements when dictated by circumstances ("if we would drop one task,<br />
which one should it be?") and a healthy dose of humour ("... Everything has fallen so silent... doo-doodoo"<br />
- "Nii vaikseks kõik on jäänud", a popular Estonian song; a remark on chat inactivity). Plus,<br />
another notion shared with the next category is the low level of hierarchy (see below).<br />
5. Facilitation: changing the role of teacher<br />
Open courses tend to have facilitators enabling the <strong>learning</strong> process rather than professors teaching.<br />
This was also witnessed at our courses where most tasks were analytical in nature and without a<br />
predetermined ‘correct’ answer. Rather than demanding the ‘party line’ in thinking, sometimes a direct<br />
(albeit well-argumented) opposition to the facilitator was graded above the like-mindedness (“I totally<br />
disagree, but that's OK :)"). This resulted in sharp remarks (“The happiest people are those who<br />
reside in infirmaries...", "Internet is the reflection of the real world - perhaps just a bit more cruel"),<br />
some really down-to-earth approaches to sensitive topics ("I just tested out Net Nanny (a web<br />
censorship software). Was still able to surf some porn and book a couple of hookers.") and a range of<br />
thought-provoking questions presented to the peers ("What can a parent do if his/her child is<br />
cyberbullied when he/she knows nothing about it?", "About today's topic, people what do you think is<br />
a hacker a modern anarchist?" etc). However, the overall reaction was again very positive ("I'm<br />
grateful for the course, it taught another angle to look at the world, introduced problems which are not<br />
met in everyday life and forced us every week to think and write about the world.").<br />
Our findings from the Wikiversity courses were the following:<br />
The open content model functions as an effective quality assurance system – students are<br />
motivated to produce quality work because of the possibility of immediate (both in-course and<br />
from outside) peer review. Other fields of Free Culture (e.g. free and open-source software, as<br />
suggested by Mockus et al 2002) tend to share the phenomenon.<br />
Extra students joining from outside should be treated as a possibility, not a liability. The main<br />
point here is to design the course in such a way that substantial increase in student numbers<br />
would increase the supervisor's workload the least possible amount. One option is to make<br />
distinction between for-credit and non-credit students. For-credit students have to complete all the<br />
assignments and will get university credits for the course. Non-credit students participate the<br />
course in various less formal ways and do not have to complete all the assignments. They will<br />
receive feedback from other participants but the facilitator will not take responsibility to give them<br />
the same amount of feedback that is given to for-credit students. This model is used by Alec<br />
Couros (2010).<br />
The model is especially suitable for the fields where rapid changes in the course materials are<br />
needed in order to follow the state of the art (new media, information technology). As evident from<br />
the history of Wikipedia, wiki as a media form is very flexible and can follow changes well. The<br />
experience suggests that adequately supervised Wikiversity courses can have attrition rates far<br />
less than conventional courses – especially if such a method is new for students. This has one<br />
condition – the structure of the course and the tasks must be clear from the beginning, making a<br />
very well laid-out and written course guide a must.<br />
Community management and also conflict solving skills are very valuable – especially in<br />
multicultural courses or when a large share of participants have other first languages. In some of<br />
our courses, the supervisor was very thankful for having previous experience as a chatroom<br />
administrator.<br />
While it is possible to facilitate large courses in a single chat (SSNC 2010 made likely a record<br />
with weekly Skype text chats involving ~140 people), it is advised to split large groups for chat. If it<br />
is impossible, stricter chat etiquette must be maintained to minimise noise, a simple helpful tip is<br />
to make the supervisor type with Caps Lock turned on (even if it is usually interpreted as shouting<br />
and therefore discouraged).<br />
For best results, the homework should contain tasks with varying response cycle (e.g. wiki-based<br />
group paper vs weekly Skype chat session – ample thinking room vs 'on the toes' situation). Also,<br />
some people enjoy writing large papers, some like short opinions or direct discussion – this is<br />
where the surplus in total points counts.<br />
375
Kaido Kikkas et al.<br />
Occasional guests participating in the weekly chat (1-2 per course) are very stimulating for the<br />
sense of community (e.g. ELNM 2009 had a lawyer and a Microsoft specialist) Creating a<br />
dedicated <strong>learning</strong> community rather than stressing a teacher-student dichotomy is crucial to<br />
success. Our experience shows that the weekly chat sessions are crucial here – these should not<br />
be neither just formal exchanges or just chit-chats, but a mixture of both with a strong accent in<br />
the 'we'-aspect, or community building. Also, these sessions, when directed skillfully, can have a<br />
large impact on students' self-study motivation and skills. A good enough proof was the end of the<br />
SSNC 2009 course, with students grumbling during the last chat session: 'What the heck will we<br />
do on Wednesday nights from now on?'.<br />
5. Conclusions<br />
In this study, we applied action research approach and framework analysis method to make sense of<br />
our experiences with open courses in 2008-2011 (more than 20 courses at three different higher<br />
education facilities in Estonia). Our analysis shows that open courses have evolved beyond the phase<br />
of experimentation by few enthusiasts and are a likely candidate to be the next major step in the<br />
evolution of eLearning.<br />
Acknowledgements<br />
This research was funded by Estonian Ministry of Education and Research targeted research grant<br />
No. 0130159s08.<br />
References<br />
Brown, S.J. and Adler, R.P. (2008) ‘Minds on Fire: Open Education, the Long Tail, and Learning 2.0.’,<br />
EDUCAUSE Review, vol 43, no 1, January/February, pp. 16-32.<br />
Couros, A., (2010) ‘Developing Personal Learning Networks for Open and Social Learning’, in Veletsianos, G.<br />
(ed.) Emerging Technologies in Distance Education. Athabasca: Athabasca University Press, pp. 109-128.<br />
Downes, S. (2011) Connectivism and Connective Knowledge, [Online], Available:<br />
http://www.huffingtonpost.com/stephen-downes/connectivism-and-connecti_b_804653.html [9 Jun 2011].<br />
Ehlers, U., (2011) ‘From Open Educational Resources to Open Educational Practices’, eLearning Papers, no 23,<br />
March, pp.1–8.<br />
Fini, A. (2009) ‘The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course<br />
Tools’, International Review of Research in Open and Distance Learning, vol 10, no 5.<br />
Johnstone, S.M. (2005) ‘Open Educational Resources Serve the World’, EDUCAUSE Quarterly, vol 28, no 3, pp.<br />
16-32.<br />
Lessig, L. (2004) Free Culture: How Big Media Uses Technology and the Law to Lock Down Culture and Control<br />
Creativity. New York: The Penguin Press.<br />
Leinonen, T. (2005). ‘(Critical) history of ICT in education – and where we are heading?’, [Online], Available:<br />
http://flosse.blogging.fi/2005/06/23/critical-history-of-ict-in-education-and-where-we-are- heading/ [4 Jun<br />
2011].<br />
Leinonen, T., Vadén, T. and Suoranta, J., (2009) ‘Learning in and with an open wiki project: Wikiversity’s<br />
potential in global capacity building’, First Monday, vol 14, no 2.<br />
Matsuo, Y. and Yamamoto, H. (2009) ‘Community gravity: measuring bidirectional effects by trust and rating on<br />
online social networks’, WWW '09 Proceedings of the 18th international conference on World Wide Web,<br />
pp. 751-760.<br />
Mockus, A., Fielding, R.T. and Herbsleb, J.D. (2002) ‘Two case studies of open source software development:<br />
Apache and Mozilla’, ACM Transactions on Software Engineering and Methodology, vol 11, no 3, pp. 309-<br />
346.<br />
OECD (2007) Giving Knowledge for Free: The Emergence of Open Educational Resources, OECD Publishing.<br />
P2PU (2011), [Online], Available: http://p2pu.org/ [9 Jun 2011].<br />
Põldoja, H., (2010) ‘EduFeedr: following and supporting learners in open blog-based courses’, Proceedings<br />
Barcelona Open ED 2010, The Seventh Annual Open Education Conference, Barcelona, pp. 399-407.<br />
The Cape Town Open Education Declaration (2007),[Online], Available:<br />
http://www.capetowndeclaration.org/read-the-declaration [9 Jun 2011].<br />
Väljataga, T. and Põldoja, H. (2011), ‘Enhancing community gravity in open online courses’, ECER 2011 Urban<br />
Education (accepted for presentation/publication).<br />
Wiley, D. (2007) Intro Open Ed Syllabus, [Online], Available:<br />
http://www.opencontent.org/wiki/index.php?title=Intro_Open_Ed_Syllabus [9 Jun 2011].<br />
376
Using a Social Networking Environment to Facilitate<br />
Transition Into Higher Education<br />
John Knight and Rebecca Rochon<br />
Bucks New University, High Wycombe, UK<br />
John.Knight@bucks.ac.uk<br />
Rebecca.Rochon@bucks.ac.uk<br />
Abstract: Transition into higher education (HE) can be challenging for incoming students. Literature identifies<br />
three main areas where students may benefit from support: social, practical and academic. This paper discusses<br />
a case study that explores the potential of a social networking environment to provide support in these areas. The<br />
Learning Development Unit (LDU) at Bucks New University launched Startonline in 2010. This online presessional<br />
environment used the social networking platform Ning to provide new students with access to nonsubject-specific<br />
academic activities (e.g., critical thinking), social networking tools and practical information. As a<br />
pilot, the aim was to observe where students focussed their attention and explore the affordances of a social<br />
networking environment for facilitating transition. Startonline ran for the month leading up to the beginning of the<br />
academic year. Around 300 students participated. Quantitative analysis of platform user data was conducted and<br />
student and staff participants were interviewed, providing useful qualitative data. Postings and replies were<br />
counted and organised into the following themes: social, practical and academic. Findings highlighted that<br />
students’ engaged intensely in social and informational aspects of the environment, but remained resolutely<br />
uninterested in generic academic activities. There was, however, considerable self-directed interest in finding<br />
subject-specific information and <strong>learning</strong> activities. Personal involvement of subject-teaching staff seemed the<br />
determining factor in this. This project underlines the usefulness of SNSs in providing powerful opportunities for<br />
students to establish social networks as they transition into higher education. Methods for engaging students<br />
academically in social networking environments are also suggested.<br />
Keywords: transition, social networking, social capital, affordances, engagement<br />
1. Introduction<br />
A review of theories related to student retention by Braxton and Hirschy (2005) suggests that while<br />
the issue of transition is complex, two models remain important in understanding: Tinto’s (1975, 1993)<br />
<strong>Academic</strong> and social integration and Astin's (1977, 1985) Theory of Involvement . Both these models<br />
suggest that the more engaged a student is with the institution, the higher likelihood of student<br />
retention. By focussing on the “known” elements, such as academic, social and practical information,<br />
it is possible for universities to contribute actively and positively during the transition phase to facilitate<br />
this engagement. This case study considers the use of a social networking site (SNS) to provide<br />
support to students in their transition into higher education.<br />
The Learning Development Unit (LDU) at Bucks New University (Bucks) has been addressing<br />
transitional issues for a number of years through face-to-face pre-sessional campus-based<br />
programmes. These programmes have been highly successful in terms of providing students with<br />
opportunities to develop academic skills and engage socially. To provide opportunities for a wider<br />
range of students, the LDU launched Startonline in 2010, a bespoke SNS based on the Ning platform<br />
which went live one month prior to induction. The environment included video, podcasts, discussion<br />
forums and web pages, including:<br />
Non-subject-specific academic activities: critical thinking; logical reasoning, writing skills<br />
Social networking tools: profiles, ‘friending’, messaging and chat<br />
Practical information ranging from student services to local entertainment<br />
In total, just over 300 people became members. Of these, 25 were staff. Student-initiated discussion<br />
activity was counted and organised by theme. A total of 332 discussion threads and replies were<br />
recorded, of which staff activity accounted for 91 (28%). This initial analysis was complemented with<br />
interviews with students as well as one staff member.<br />
2. Emphasising the social in social networking<br />
Establishing friendships and social networks has been described as key to transition (Lowe and<br />
Cooke, 2003; Maunder et al., 2010). A study of undergraduate students at Bucks noted that first-year<br />
students depend on their network of friends for everything from moral support to more general<br />
377
John Knight and Rebecca Rochon<br />
guidance, concluding that universities should do more to ensure that students have the opportunity to<br />
create these networks as soon as they start their studies (Wickens et al., 2006).<br />
SNSs are well-placed to provide students with opportunities for establishing contact with each other;<br />
they are well established as part of students’ existing communication systems (Phipps, 2007) and<br />
play an acknowledged role in the ways in which they manage and maintain their ‘social capital’<br />
(Ellison et al., 2007). Indeed, it was clear from Startonline activity and from interviewees that students<br />
were keen to use the technology to lay the groundwork for future university-based relationships.<br />
I like how it was set up so that students that were just starting could like get to know<br />
people that could be on their course or who they might be living with things like that... so<br />
that you don’t feel too scared or anything when you come in to the Uni thinking “Oh no I<br />
don’t know anyone.” (Excerpt from student interview)<br />
Statistical data demonstrated that the most popular areas were member profiles and discussion<br />
forums. Just over 60% of discussions initiated by students related to social issues, in particular finding<br />
others who were either studying on the same course or living in or near the same accommodation<br />
(see Fig. 1). This highlights that building social capital was the primary concern of users, and that<br />
there was interest in transferring this beyond the Startonline environment.<br />
Theme Threads Replies Total (threads and replies)<br />
% of total<br />
discussion<br />
activity<br />
Social 37 173 210 63<br />
Practical 7 24 31 9<br />
<strong>Academic</strong> 0 0 0 0<br />
Figure 1: Summary of student discussion forum activity (threads and replies)<br />
Using a bespoke SNS solution rather than building on students existing online profiles (e.g., in<br />
Facebook), allowed Startonline to circumvent those privacy issues identified in other studies<br />
(Ribchester, 2009; Sturgeon and Walker, 2009) and located control firmly with the students in terms of<br />
the extent to which they could choose to make links between their personal online ‘spaces’ and their<br />
more public university ‘faces’. Interestingly, there is clear evidence of students establishing initial<br />
contact in Startonline and then moving out into the private domain by exchanging profile information<br />
and ‘friending’ each other on Facebook. Similarly, there is also evidence of interest and intention of<br />
meeting within the physical University environment at an event just prior to induction<br />
3. Managing practicalities<br />
Managing the practicalities of living away from home in an unfamiliar environment and obtaining<br />
information related to the university was another focus for concern, accounting for slightly less than<br />
10% of discussion activity (see Fig. 1). Students were also pro-active in starting their own discussion<br />
threads if they needed information. Indeed, almost 20% of all student-initiated threads were requests<br />
for information. Equally, students seemed content to answer one another, or direct others to<br />
information. In cases where specialised information was needed, the Students Union or appropriate<br />
staff member provided information. Students were both interested in practical information and proactive<br />
in obtaining it:<br />
I’m disabled. So I was a bit unsure about how I was, you know, how I’d go about finding<br />
the Disabled Unit and what I could do, and what sort of help they’d give[...] I did ask a<br />
few questions I wasn’t sure about. So, I asked those myself. (Excerpt from student<br />
interview)<br />
4. Learning<br />
Student problems during first year are often related to a lack of study skills (Winterson and Russ,<br />
2008; Cook and Leckey, 1999). However, none of the three <strong>learning</strong> generic activities were engaged<br />
with by students (see Fig.1). Reasons for students’ lack of engagement in this area remain unclear,<br />
but a possible cause is suggested in the way that SNSs can be said to afford interaction and provide<br />
opportunities for <strong>learning</strong>. Where the technology was used in a way that ‘played to its strengths’ (i.e.,<br />
to connect with others and convey information) <strong>learning</strong> opportunities seemed to occur<br />
spontaneously. Where subject-related teaching staff members were available, students self-directedly<br />
sought information and engaged with materials in a way that could be usefully built on within the<br />
378
John Knight and Rebecca Rochon<br />
physical teaching environment. Students also showed interest in finding subject-specific information<br />
and <strong>learning</strong> activities. It is telling that this <strong>learning</strong>-related activity took place solely via direct<br />
messages to staff profiles, rather than in the public discussion forum, perhaps reflecting Greenhow’s<br />
(2009) assessment of the usefulness of SNSs to afford useful academic behaviour. Direct, personal<br />
involvement of subject-teaching staff seemed the determining factor in encouraging this. Staff who<br />
engaged with students directly, providing subject-related content, found the experience rewarding and<br />
were able to draw on shared Startonline experiences during initial face-to-face contact:<br />
...[it] was very useful in terms of that engagement: breaking it down rather than being just<br />
a cold lecture [...] it became something that was a little bit more vibrant, really.(Excerpt<br />
from staff interview)<br />
5. Conclusion<br />
Using SNSs has the potential to widen access to useful pre-entry information and allow students to<br />
engage easily and at a distance with other students and members of staff. Issues of privacy are easily<br />
circumvented by using bespoke solutions, allowing students to manage the degree of overlap<br />
between academic and private social networks. It is apparent that SNSs are at their most beneficial<br />
when employed for their intended purpose. Using SNSs empowers students to engage in the useful<br />
activity of developing friendships, something that they are demonstrably interested in doing.<br />
Attempting to manipulate the affordances of the platform to provide generic pre-entry <strong>learning</strong><br />
activities via online discussions was not successful. However, the social engagement of staff<br />
members encouraged students’ pro-active engagement with subject specific resources. The<br />
development of students as self-directed learners, who take responsibility for their own engagement,<br />
is a key benefit of using SNSs that could be usefully be explored further.<br />
References<br />
Alcock, J. (2009) “Using Facebook Pages to reach users: the experiences of University of Wolverhampton”,<br />
Association of Libraries and Information Professionals in the Social Sciences (ALISS) Quarterly, Vol. 4,<br />
pp.2-6.<br />
Astin, A.W. (1993) What matters most in college: Four critical years revisited, San Francisco, Jossey-Bass.<br />
Braxton, J. M. and Hirschy, A. S. (2005) “Theoretical Developments in the study of college student departure”. In<br />
Seidman, A. (Ed.). College student retention: Formula for student success, Westport, CT, Praeger Press,<br />
pp.61-87.<br />
Cook, A. and Leckey, J. (1999) “Do Expectations Meet Reality? A survey of changes in first year student<br />
opinion”, Journal of Further and Higher Education, Vol.23, No. 2, pp.157-171.<br />
Ellison, N. B., Steinfield, C., and Lampe, C. (2007) The benefits of Facebook friends: Social capital and college<br />
students use of online social network sites. Journal of Computer-Mediated Communication. Vol.12, No.4,<br />
pp.1143-1168.<br />
Greenhow, C. (2009). Social network sites and education: emerging research within CSCL, in A. Dimitrakopoulou<br />
(Ed.), CSCL 2009: Proceedings of the International Society of the Learning Sciences Computer-supported<br />
Collaborative Learning Conference, Rhodes, Greece, June 10-12.<br />
Lowe, H. and Cook, A. (2003) “Mind the Gap: Are students prepared for higher education?”, Journal of Further<br />
and Higher Education, Vol. 27, No.1, pp.53 -76.<br />
Maunder, R., Mjali, S. and Cunliffe, M. (2010) “Exploring transition in HE for first and second year undergraduate<br />
psychology students: the role of expectations, personal growth and social identity in shaping experiences”.<br />
Paper read at 2nd Annual Aim Higher West Yorkshire Symposium: What is Transition?, University of Leeds,<br />
England, 16 July 2010.<br />
Phipps, L. (2007) “Web 2.0 and social software: an introduction”, [online], JISC.<br />
http://www.jisc.ac.uk/publications/publications/web2socialsoftwarev1.aspx.<br />
Ribchester, C. (2009) “Enhancing the student experience through the use of online social networks, with<br />
particular reference to induction and employability”. Proceedings of the ‘Hitting the Ground Running’<br />
Conference at Glasgow Caledonian University, Glasgow.<br />
Sturgeon, C.M. and Walker, C. (2009) “Faculty on Facebook: Confirm or Deny?” Paper read at the Annual<br />
Instructional Technology Conference (14th, Murfreesboro, TN, Mar 29-31, 2009)<br />
Tinto, V. (1975). “Dropouts from higher education: a theoretical synthesis of recent research”, Review of<br />
Educational Research, Vol. 45, pp.89-125.<br />
Tinto, V. (1993) Leaving college: rethinking the causes and cures of student attrition, 2 nd ed, Chicago, University<br />
of Chicago Press.<br />
Wickens, E. Forbes, A. and Tribe, J. (2006) Listening, understanding and responding to leisure and tourism<br />
undergraduates. Journal of Hospitality, Leisure, Sport and Tourism Education, Vol. 5, No. 2, pp.4-13.<br />
Winterson, J. and Russ, M. (2008) “Understanding the transition from school to university in music and related<br />
subjects”. Project Report. University of Huddersfield,<br />
http://eprints.hud.ac.uk/3692/2/Understanding_the_Transition_from_School_to_University.pdf.<br />
379
Evaluation of Quality of Learning Scenarios and Their<br />
Suitability to Particular Learners’ Profiles<br />
Eugenijus Kurilovas 1, 2, 3 , Inga Zilinskiene 2 and Natalija Ignatova 4<br />
1<br />
Centre of Information Technologies in Education, Vilnius, Lithuania<br />
2<br />
Vilnius University Institute of Mathematics and Informatics, Lithuania<br />
3<br />
Vilnius Gediminas Technical University, Lithuania<br />
4<br />
Vilnius Pedagogical University, Lithuania<br />
eugenijus.kurilovas@itc.smm.lt<br />
inga.zilinskiene@gmail.com<br />
natalija.ignatova@itc.smm.lt<br />
Abstract: The aim of the paper is to investigate and present a comprehensive scientific model and several<br />
methods suitable for the expert evaluation of quality of <strong>learning</strong> scenarios. A special attention is paid to their<br />
suitability to particular learner groups (i.e., profiles). The solution of <strong>learning</strong> scenarios quality evaluation and<br />
optimisation problems could help educational institutions to select suitable <strong>learning</strong> scenarios for the particular<br />
learner profiles. The research results will be implemented in iTEC – a four-year, pan-European research and<br />
development project focused on the design of the future classroom funded by EC 7th Framework Programme.<br />
The main objectives of iTEC are to develop and refine a range of teaching and <strong>learning</strong> scenarios for the future<br />
classroom, to develop decision support criteria that facilitate the selection of scenarios that can be mainstreamed<br />
and taken to scale; and to carry out large-scale pilots in up to 1,000 classrooms in at least 12 European countries<br />
exploring both the integration of technologies and how these impact on teaching and <strong>learning</strong> practices.<br />
Suitability of several iTEC scenarios to particular learner groups is analysed in the paper. A number of multiple<br />
criteria decision analysis principles are applied to create a comprehensive quality model (criteria tree) for<br />
evaluating <strong>learning</strong> scenarios. Several optimisation methods are explored to optimise <strong>learning</strong> scenarios in<br />
conformity with particular learner’s profile. Several practical examples of iTEC <strong>learning</strong> scenarios have been<br />
evaluated against the proposed model and methods. The research results have shown that both Analytic<br />
Hierarchy Process (AHP) method to establish the weights of quality criteria and several fuzzy optimisation<br />
methods are suitable to solve <strong>learning</strong> scenarios’ multiple criteria evaluation and optimisation tasks for particular<br />
learner profiles.<br />
Keywords: <strong>learning</strong> scenarios, evaluation of quality, multiple criteria decision analysis, learner profile,<br />
optimisation methods, Analytic Hierarchy Process (AHP)<br />
1. Introduction: Evaluation of <strong>learning</strong> scenarios in iTEC project<br />
The aim of the paper is to investigate, propose, and demonstrate examples of practical application of<br />
a model and methods suitable for the expert evaluation of quality of <strong>learning</strong> scenarios (LS). A special<br />
attention is paid to LS suitability to particular learner groups (i.e., profiles).<br />
1.1 What is <strong>learning</strong> scenario?<br />
Unit of Learning (also known as ‘Learning Scenario’ (LS) – this term is commonly used in European e-<br />
Learning community, e.g., in iTEC project) is referred here as an aggregation of <strong>learning</strong> activities that<br />
take place in particular virtual <strong>learning</strong> environments (VLEs) using particular <strong>learning</strong> objects (LOs).<br />
This notion is based on Koper and Tattersall (2005) work and IMS Learning Design (IMS LD, 2003)<br />
specification. Quoting Koper and Tattersall (2005), “a 'Unit of Learning' refers to a complete, selfcontained<br />
unit of education or training, such as a course, a module, a lesson, etc”. IMS LD (2003)<br />
specification’s conceptual vocabulary clarifies that a 'unit of <strong>learning</strong>' is an abstract term used to refer<br />
to any delimited piece of education or training. It is noted that a 'unit of <strong>learning</strong>' represents more than<br />
just a collection of ordered resources to learn, it includes a variety of prescribed activities (problem<br />
solving activities, search activities, discussion activities, peer assessment activities, etc.),<br />
assessments, services and support facilities provided by teachers, trainers and other staff members.<br />
A <strong>learning</strong> design as an integral part of any unit of <strong>learning</strong> is a description of a method enabling<br />
learners to attain certain <strong>learning</strong> objectives by performing certain <strong>learning</strong> activities (LA) in a certain<br />
order in the context of a certain <strong>learning</strong> environment. According to IMS LD (2003), activities are one<br />
of the core structural elements of the '<strong>learning</strong> workflow' model for <strong>learning</strong> design. They form the link<br />
between the roles and the <strong>learning</strong> objects and services in the <strong>learning</strong> environment. The activities<br />
describe a role they have to undertake within a specified environment composed of <strong>learning</strong> objects<br />
380
Eugenijus Kurilovas et al.<br />
and services. Activities take place in a so-called 'environment', which is a structured collection of<br />
<strong>learning</strong> objects, services, and sub-environments.<br />
Therefore, we can conclude that in general case LS could consist of <strong>learning</strong> activity (or method),<br />
<strong>learning</strong> objects and <strong>learning</strong> environment referred here as services package. This kind of services<br />
package in e-Learning theory is commonly known as Virtual Learning Environment (VLE) (also known<br />
as Learning Management System, etc.).<br />
Therefore, we can divide LS into three components, namely LA, LOs and VLE.<br />
1.2 Learning scenarios in iTEC project<br />
Research results will be implemented in iTEC project. iTEC (Innovative Technologies for an Engaging<br />
Classroom) is a four-year, pan-European project focused on the design of the future classroom. The<br />
main iTEC objectives are (iTEC, 2011):<br />
To develop and refine a range of teaching and <strong>learning</strong> scenarios for the future classroom, and to<br />
develop decision support criteria (technological, pedagogical and policy-related) that facilitates the<br />
selection of scenarios that can be mainstreamed and taken to scale; and<br />
To develop specific teaching and <strong>learning</strong> activities, based on the scenarios, and test these in a<br />
pre-pilot phase, and to carry out large-scale pilots in up to 1,000 classrooms in at least 12<br />
European countries exploring both the integration of technologies and how these impact on<br />
teaching and <strong>learning</strong> practices and the engagement of a wider group of stakeholders outside the<br />
school;<br />
1.3 Learning scenarios personalization strategies<br />
The paper is aimed to evaluate quality of LS paying especial attention to their suitability to particular<br />
learners’ groups (i.e., profiles).<br />
Learning groups (styles) were developed by Honey and Mumford (1992), based upon the work of<br />
Kolb (1984), and they identified four distinct <strong>learning</strong> styles or preferences: Activist, Theorist;<br />
Pragmatist and Reflector.<br />
Activist: Activists are those people who learn by doing. Have an open-minded approach to<br />
<strong>learning</strong>, involving themselves fully and without bias in new experiences. Their preferred activities<br />
are: brainstorming, problem solving, group discussion, puzzles, competitions, and role-play.<br />
Reflector: These people learn by observing and thinking about what happened.<br />
Pragmatist: These people need to be able to see how to put the <strong>learning</strong> into practice in the real<br />
world.<br />
Theorist: These learners like to understand the theory behind the actions.<br />
As far as the format of the paper does not allow presentation of research results for all<br />
aforementioned learner profiles in the same paper, the authors aim to present quality and suitability of<br />
LS to the only learner profile, namely, activist learners.<br />
The literature review (Mulwa et. al. 2010, O' Keeffe et. al. 2006, Essalmi et. al. 2010) shows that there<br />
are a lot of personalisation parameters and when several personalisation parameters are used, the<br />
question is: what personalisation parameters are to be used for personalising each course (LS)?<br />
Essalmi et. al. (2010), discussed 4 personalisation strategies (i.e., alternatives A1, A2, A3, and A4):<br />
A1: Using all the personalisation parameters for each course<br />
A2: Using the subset of personalisation parameters which includes divergent characteristics of<br />
learner<br />
A3: Using the subset of personalisation parameters to include only the most significant ones for<br />
each course<br />
A4: Using the subset of personalisation parameters recommended by the professor responsible<br />
for the course<br />
381
Eugenijus Kurilovas et al.<br />
Essalmi et. al. (2010) approach is based on the fourth alternative, that the teacher is the expert, who<br />
decides which personalisation parameters (criteria) are most significant by selecting <strong>learning</strong> scenario<br />
for appropriate course (LS).<br />
Figure 1: Proposed framework for evaluation of suitability of <strong>learning</strong> scenario to learner<br />
In this paper, the authors play the role of experts-evaluators (lecturers) who decide which LS quality<br />
criteria are more significant for the activist learners, and which are less significant, as well as evaluate<br />
quality of each LS alterative against the proposed quality criteria model.<br />
The rest of the paper is organised as follows: methodology of the research is presented in chapter 2,<br />
literature analysis and research results are presented in chapter 3, and conclusion and<br />
recommendations are presented in chapter 4. The main chapter containing research results is divided<br />
into two separate parts – creation of LS quality evaluation model (criteria tree) and practical<br />
application of the method for evaluating LS which in its turn is divided into (a) the use of the novel<br />
method of consecutive triple application of AHP to establish the weights of quality criteria, (b)<br />
application of triangular and trapezoidal fuzzy numbers to establish the values of criteria, and (c)<br />
practical example of evaluating several real-life LS alternatives used in iTEC project.<br />
2. Methodology of the research<br />
The main problem analysed in the paper is how to establish<br />
A ‘proper’ set of LSs quality evaluation criteria that should reflect the objective scientific principles<br />
of constructing a quality model (criteria tree) for LS suitable to particular learners’ groups (i.e.,<br />
activist learners in our case), and<br />
‘Proper’ methods for evaluation of LS quality and suitability to particular learners’ profiles paying<br />
especial attention to establishment of proper weights of quality criteria by the novel method of<br />
consecutive triple application of AHP.<br />
Expert evaluation is referred here as the multiple criteria evaluation of LS alternatives aimed at<br />
selection of the best ones based on score-ranking results (Kurilovas and Dagiene 2009a,b).<br />
According to (Zavadskas and Turskis 2010), there is a wide range of multiple criteria decision making<br />
problem solution techniques, varying in complexity and possible solutions. Each method has its own<br />
strength, weaknesses and possibilities to be applied. Usually the experts have to deal with the<br />
problem of optimal decision in the multiple criteria situation where the objectives are often conflicting.<br />
In this case, an optimal decision is the one that maximises the expert’s utility.<br />
382
Eugenijus Kurilovas et al.<br />
Evaluation of quality of LS alternatives is a typical case where the criteria are conflicting, i.e., LS could<br />
be very qualitative against several criteria, and not qualitative against the other ones, and vice versa.<br />
Therefore, the authors propose to use multiple criteria decision analysis (MCDA) approach for<br />
creation of LS quality evaluation model. In order to construct a proper comprehensive scientific quality<br />
criteria system (model), the authors use the well known principles of identification of quality criteria<br />
that have been proposed by Belton and Stewart (2002) in multiple criteria decision analysis (MCDA)<br />
theory related research work. Practical application of these principles will be described below while<br />
analysing LS quality criteria system (tree).<br />
LS multiple criteria evaluation method used by the authors is referred here as the experts’ additive<br />
utility function represented by formula (1) below including LS evaluation criteria, their ratings (values)<br />
and weights. According to (Kurilovas and Serikoviene 2010b), this method is well-known in the theory<br />
of optimisation methods and is named “scalarisation method”. A possible decision here could be to<br />
transform multi-criteria task into one-criterion task obtained by adding all criteria together with their<br />
weights. Therefore, here we have the experts’ additive utility function:<br />
m<br />
∑<br />
i=<br />
1<br />
f ( X ) = a f ( X ) , a = 1,<br />
a > 0 .<br />
i<br />
i<br />
m<br />
∑<br />
i=<br />
1<br />
i<br />
i<br />
where f i (X j) is the rating (i.e., non-fuzzy value) of the criterion i for the each of the examined LS<br />
alternatives X j. The weights here should be ‘normalised’ according to the ‘normalisation’ requirement<br />
m<br />
∑<br />
i=<br />
1<br />
a = 1,<br />
a > 0 .<br />
i<br />
i<br />
According to (Zavadskas and Turskis 2010), the normalisation aims at obtaining comparable scales of<br />
criteria values. The major is the meaning of the utility function (1) the better LS meets the quality<br />
requirements in comparison with the ideal (i.e., 100%) quality (Kurilovas and Dagiene 2009ab).<br />
The complexity of the analysed problem influences the application of more complex methods for<br />
evaluating the quality of LS from the point of view of different learner groups. In this paper, a novel<br />
method of consecutive triple application of Analytic Hierarchy Process (AHP) is used to establish<br />
proper weights of LS quality criteria in the case when there are several experts evaluators. Triangular<br />
and trapezoidal fuzzy numbers methods are used to establish proper values of LS quality criteria.<br />
After that, formula (1) is used to calculate the values of additive utility functions for each of the<br />
explored LS alternatives.<br />
3. Literature analysis and research results<br />
This section is aimed to apply the aforementioned scientific approaches in order:<br />
To propose a suitable scientific model for evaluation of quality of LS paying especial attention to<br />
their suitability to particular learners’ profiles (i.e., activist learners in our case);<br />
To propose suitable scientific methods for evaluation of quality of LOs paying especial attention to<br />
the use of novel method of consecutive triple application of AHP for establishing quality criteria<br />
weights, and different fuzzy numbers methods to establish the values (ratings) of LS quality<br />
criteria; and<br />
To present the experimental evaluation results using proposed evaluation model and methods.<br />
3.1 Learning scenarios quality model<br />
Belton and Stewart (2002) have identified the following principles of identification of quality criteria<br />
that are relevant to all MCDA approaches: (1) Value relevance; (2) Understandability; (3)<br />
Measurability; (4) Non-redundancy; (5) Judgmental independence; (6) Balancing completeness and<br />
conciseness; (7) Operationality; and (8) Simplicity versus complexity.<br />
LS quality model based on these MCDA criteria identification principles is presented in the Fig. 2. The<br />
model consists of three groups of quality criteria (i.e., components of LS), namely <strong>learning</strong> objects<br />
383<br />
(1)<br />
(2)
Eugenijus Kurilovas et al.<br />
(LOs), <strong>learning</strong> activities (methods) (LAs), and virtual <strong>learning</strong> environments (VLEs). The model<br />
consists of already created LOs and VLEs quality criteria systems accordingly presented in (Kurilovas<br />
et. al. 2011) and (Kurilovas and Dagiene 2009b), and LA quality criteria system created specially for<br />
the presented research. LA quality criteria were proposed by iTEC working groups, and a number of<br />
scientific papers were additionally analysed (Shee and Wang 2008, Sun et. al. 2008, iCOPER 2011)<br />
to propose LA quality criteria. Belton and Stewart (2002) principles were applied to form the system of<br />
criteria. The authors have paid special attention to Non-redundancy, Judgemental independence,<br />
Balancing completeness and conciseness, and Simplicity Vs complexity principles to create the<br />
comprehensive criteria tree presented in Fig. 2.<br />
Figure 2: Learning scenario quality evaluation model (criteria tree)<br />
There is a clear division of all the criteria into ‘internal quality’ and ‘quality in use’ criteria here. In the<br />
model, the authors apply criteria classification principle claiming that one should evaluate <strong>learning</strong><br />
384
Eugenijus Kurilovas et al.<br />
software alternatives using two different groups / types of evaluation criteria – ‘internal quality’ and<br />
‘quality in use’ criteria. According to (Kurilovas and Serikoviene 2010b), ‘internal quality’ is a<br />
descriptive characteristic that describes the quality of <strong>learning</strong> software independently from any<br />
particular context of its use, and ‘quality in use’ is evaluative characteristic of software obtained by<br />
making a judgment based on criteria that determine the worthiness of software for a particular project<br />
or user / group (e.g., activist learners).<br />
Therefore, if we want to analyse general quality of LS, we should consider all quality criteria a priori<br />
equal, but if we want to analyse suitability of LS to particular learner profile, we should pay special<br />
attention to ‘quality in use’ criteria by establishing higher weights.<br />
3.2 Learning scenarios quality evaluation methods<br />
As it was mentioned above, in order to evaluate quality of alternatives or their suitability to particular<br />
learner profiles, one should establish the weights of quality criteria, their values, and after that to apply<br />
experts’ additive utility formula (1) to get final evaluation results.<br />
3.2.1 Use of Analytic Hierarchy Process (AHP) to establish the weights of quality criteria<br />
According to (Kurilovas et. al., 2011), the weight of the evaluation criterion reflects the experts’<br />
opinion on the criterion’s importance level in comparison with the other criteria for the particular<br />
needs.<br />
In this paper, the authors propose an original method of consecutive triple application of AHP to<br />
establish proper weights of LS quality evaluation criteria in the case when there are several experts<br />
evaluators.<br />
According to Saaty (1990), AHP is a useful method for solving complex decision-making problems<br />
involving subjective judgment. In AHP, the multi-attribute weight measurement is calculated via pairwise<br />
comparison of the relative importance of two factors (Lin 2010). The design of the questionnaire<br />
incorporates pair-wise comparisons of decision elements within the hierarchical framework. Each<br />
evaluator (three in our case) is asked to express relative importance of two criteria in the same level<br />
by a nine-point rating scale. After that, we have to collect the scores of pair-wise comparison, and<br />
form pair-wise comparison matrices for each of the K evaluators.<br />
According to Saaty (2008), the fundamental scale of absolute numbers is as follows:<br />
Table 1: Pair-wise comparison scale for AHP preferences<br />
Numerical rating Verbal judgements of preferences<br />
9 Extremely preferred<br />
8 Very strongly to extremely<br />
7 Very strongly preferred<br />
6 Strongly to very strongly<br />
5 Strongly preferred<br />
4 Moderately to strongly<br />
3 Moderately preferred<br />
2 Equally to moderately<br />
1 Equally preferred<br />
After that, we have to construct a set of pair-wise comparison matrices (size n x n) for each of the<br />
lower levels with one matrix for each element in the level immediately above by using the relative<br />
scale measurement shown in Table 1. The pair-wise comparisons are done in terms of which element<br />
dominates the other:<br />
There are n(n-1)/2 judgments required to develop the set of matrices in this step. Reciprocals are<br />
automatically assigned in each pair-wise comparison.<br />
385
Eugenijus Kurilovas et al.<br />
Then hierarchical synthesis is used to weight the eigenvectors by the weights of the criteria and the<br />
sum is taken over all weighted eigenvector entries corresponding to those in the next lower level of<br />
the hierarchy:<br />
In order to check the correctness of calculation, having made all the pair-wise comparisons the<br />
eigenvalue λ max is calculated:<br />
The method of consecutive triple application of AHP consists of application of aforementioned AHP in<br />
three consistent stages:<br />
Establishment of comparative weights of three different groups of LS quality criteria and weights a<br />
i of all quality criteria;<br />
Establishment of comparative weights of ‘internal quality’ and ‘quality in use’ criteria groups from<br />
activist learner point of view; and<br />
Establishment of final weights of quality criteria from activist learners point of view by application<br />
of AHP once again only for ‘quality in use’ criteria.<br />
3.2.2 Use of triangular and trapezoidal fuzzy numbers to establish values of quality criteria<br />
The widely used measurement criteria of the decision attributes’ quality are mainly qualitative and<br />
subjective. Decisions in this context are often expressed in natural language, and evaluators are<br />
unable to assign exact numerical values to the different criteria. Assessment can be often performed<br />
by linguistic variables: ‘bad’, ‘poor’, ‘fair’, ‘good’ and ‘excellent’. These values are imprecise and<br />
uncertain: they are commonly called ‘fuzzy values’. Integrating these different judgments to obtain a<br />
final evaluation is not evident (Kurilovas and Serikoviene 2010ab).<br />
Therefore, the authors have proposed to use fuzzy group decision making theory (Ounaies et. al.<br />
2009) to obtain final assessment measures. The fuzzy numbers are: (1) triangular fuzzy numbers, (2)<br />
trapezoidal fuzzy numbers, and (3) bell-shaped fuzzy numbers (Zhang Li Li and Cheng De Yong<br />
1992, Kurilovas et. al., 2011). In the presented paper, the authors use triangular (Fig. 3) and<br />
trapezoidal (Fig. 4) fuzzy numbers for evaluating quality of LS and their suitability to activist learner<br />
profile:<br />
Figure 3: Triangular fuzzy numbers Figure 4: Trapezoidal fuzzy numbers<br />
386
Eugenijus Kurilovas et al.<br />
According to Kurilovas et. al. (2011), in the case of using average triangular fuzzy numbers, linguistic<br />
variables conversion into non-fuzzy values of the evaluation criteria should be as follows:<br />
‘excellent’=0.850; ‘good‘=0.675; ‘fair’=0.500; ‘poor’=0.325; ‘bad’=0.150, and in the case of using<br />
average trapezoidal fuzzy numbers – ‘excellent’=1.000; ‘good‘=0.800; ‘fair’=0.500; ‘poor’=0.200;<br />
‘bad’=0.000.<br />
There are three experts-evaluators (i.e., the authors of the paper) in our case, and therefore it was<br />
necessary to calculate average values for each LS quality criterion.<br />
3.3 Practical example of evaluation of <strong>learning</strong> scenarios in iTEC project<br />
Two LS alternatives proposed by iTEC experts were chosen by the authors to demonstrate<br />
application of the aforementioned methods for evaluating LS quality and their suitability to chosen<br />
activist learner profile:<br />
LS1: “A Breath of Fresh Air” (Cycle 1 detailed scenario available at<br />
http://itec.eun.org/web/guest/scenario-library )<br />
LS2: “Online Repositories Rock” (Cycle 1 detailed scenario online at<br />
http://itec.eun.org/web/guest/scenario-library )<br />
The scenarios do not contain any LOs and there is no explanation of what kind of VLEs should be<br />
used to implement the scenarios. Use of particular LOs and VLEs is up to the decision of every<br />
country participating in iTEC, and the project experts propose a number of widgets for each scenario<br />
to enrich VLEs to be used while implementing scenarios at schools.<br />
Therefore, the authors decide to consider ‘good’ LOs only for the chosen LS. This means that all LOs<br />
quality criteria in the model presented in Fig. 2 should be evaluated 0.800 according to trapezoidal<br />
fuzzy numbers method, and 0.675 – according to triangle fuzzy numbers method.<br />
VLE Moodle was chosen by the authors as a proper environment to implement both LS. Since VLE<br />
Moodle was already evaluated by the experts in (Kurilovas and Dagiene 2010, 2009a), the authors<br />
use the same values to evaluate the VLE component of LS quality criteria.<br />
Application of the aforementioned method of consecutive triple application of AHP has shown the<br />
following results:<br />
Stage 1 (establishment of comparative weights of three different groups of LS quality criteria and<br />
weights a i of quality criteria) results have shown that the evaluators prefer LOs and LA<br />
components more in comparison with VLE component (LOs – 39.7%, LA – 39.7%, and VLE –<br />
20.6%). Weights a i for all 24 LS quality criteria were also calculated in conformity with AHP.<br />
Stage 2 (establishment of comparative weights of ‘internal quality’ and ‘quality in use’ criteria<br />
groups from activist learner point of view) results have shown that the evaluators prefer ‘quality in<br />
use’ criteria in comparison with ‘internal quality’ criteria (respectively 69.4% Vs 30.6% for LOs,<br />
72.2% Vs 27.8% for LA, and 61.1% Vs 39.8% for VLE) to analyse suitability of chosen LS to<br />
activist learners.<br />
Stage 3 (establishment of final weights of LS quality criteria from activist learners point of view by<br />
application of AHP once again only for ‘quality in use’ criteria) has re-established the weights of<br />
‘quality in use’ criteria.<br />
Final evaluation results after using utility function (1) are as follows:<br />
In general (G) case when experts do not take into account particular learner profile: 72.7%<br />
according to trapezoidal method, or 63.8% according to triangle method for LS1 Vs 68.5%<br />
according to trapezoidal method, or 61.0% according to triangle method for LS2:<br />
a ⋅ f ( X ) = ( 0,<br />
727 0,<br />
685)<br />
a ⋅ f ( X ) = ( 0,<br />
638 0,<br />
610)<br />
a<br />
iG j<br />
iG j<br />
In particular case when experts take into account particular (activist) learner (A) profile: 76.4%<br />
according to trapezoidal method, or 65.6% according to triangle method for LS1 Vs 69.7%<br />
according to trapezoidal method, or 61.8% according to triangle method for LS2:<br />
iA<br />
⋅ f ( X j ) =<br />
( 0,<br />
764 0,<br />
697)<br />
a<br />
iA<br />
⋅ f ( X j ) =<br />
( 0,<br />
656 0,<br />
618)<br />
387
Eugenijus Kurilovas et al.<br />
If we look separately at <strong>learning</strong> activity (LA) component of the scenarios, we’ll get the following<br />
results for LA comparative weights in LS:<br />
In general case when experts do not take into account particular learner profile: 28.3% according<br />
to trapezoidal method, or 25.2% according to triangle method for LA1 Vs 24.1% according to<br />
trapezoidal method, or 22.3% according to triangle method for LA2:<br />
⋅ f ( X ) = 0,<br />
283 0,<br />
241 ⋅ f ( X ) = 0,<br />
252 0,<br />
223<br />
aiG j<br />
a<br />
iA<br />
⋅ f ( X j ) =<br />
( )<br />
( 0,<br />
298 0,<br />
232)<br />
aiG j<br />
( )<br />
In particular case when experts take into account particular (activist) learner profile: 29.8%<br />
according to trapezoidal method, or 25.7% according to triangle method for LA1 Vs 23.2%<br />
according to trapezoidal method, or 21.8% according to triangle method for LA2:<br />
a<br />
iA<br />
⋅ f ( X j ) =<br />
( 0,<br />
257 0,<br />
218)<br />
In general case these results mean that, according trapezoidal fuzzy numbers method, LS1 meets<br />
72.7% quality in comparison with the ideal, LS2 – 68.5%, and using triangle fuzzy numbers method –<br />
LS1 – 63.8%, and LS2 – 61.0% respectively.<br />
In particular case (for activist learner) these results mean that according trapezoidal fuzzy numbers<br />
method, LS1 meets 76.4% quality in comparison with the ideal, LS2 – 69.7%, and using triangle fuzzy<br />
numbers method – LS1 – 65.6%, and LS2 – 61.8% respectively.<br />
The same tendency in ranking the alternatives is noticeable in evaluation of LA. It is understandable,<br />
because the experts have decided that LOs for each alternative should be evaluated equally, and in<br />
this case LA remains the most significant component of LS.<br />
The obtained results mean that LS1 is a better alternative in comparison with LS2 both in general case<br />
and in particular case of suitability to activist learner, and both for <strong>learning</strong> activity component and the<br />
whole <strong>learning</strong> scenario.<br />
4. Conclusion and recommendations<br />
The research results presented in the paper show that (1) complex application of principles of multiple<br />
criteria decision analysis for identification of quality evaluation criteria, (2) quality criteria classification<br />
principle, (3) fuzzy group decision making theory to obtain final evaluation measures, and (4) the<br />
original method of consecutive triple application of AHP to establish criteria weights:<br />
Are applicable in real life situations when schools have to decide on use of particular <strong>learning</strong><br />
scenarios for their education needs, and<br />
Could significantly improve the quality of expert evaluation of <strong>learning</strong> scenarios by noticeably<br />
reduce of the expert evaluation subjectivity level.<br />
Use of the method of consecutive triple application of AHP leads to establishing different weights of<br />
criteria for particular leaner groups, and respectively – to different LS alternatives evaluation results.<br />
Application of both triangle and trapezoidal fuzzy numbers shows similar alternatives’ quality<br />
evaluation results, i.e., the ranking of analysed alternatives have not changed while applying different<br />
fuzzy numbers methods. The experimental evaluation results show that proposed scientific<br />
approaches are quite objective, exact and simply to use for selecting qualitative LS alternatives for<br />
particular learner groups. On the other hand, proposed LS personalised quality evaluation approach is<br />
applicable for the aims of iTEC project in order to select LS suitable for activist learners. Therefore,<br />
these approaches have been recommended by the authors to be widely used by European policy<br />
makers, publishers, practitioners (teachers), and experts-evaluators both inside and outside iTEC<br />
project to evaluate quality and personalisation level of <strong>learning</strong> scenarios. Method of consecutive<br />
triple application of AHP presented in the paper is absolutely novel, and these new elements make<br />
the given work distinct from all the other earlier works in the area.<br />
5. Appendix 1<br />
The work presented in this paper is partially supported by the European Commission under the 7th<br />
Framework Programme – as part of the iTEC project (Grant Agreement Number 257566). The<br />
authors are solely responsible for the content of this paper. It does not represent the opinion of the<br />
388
Eugenijus Kurilovas et al.<br />
European Commission, and the European Commission is not responsible for any use that might be<br />
made of data appearing therein.<br />
References<br />
Belton, V., Stewart, T.J., (2002) “Multiple criteria decision analysis: an integrated approach”. Kluwer <strong>Academic</strong><br />
Publishers, 2002<br />
Essalmi, F., Jemni Ben Ayed, L., Jemni, M., Kinshuk, Graf, S. (2010) “A fully personalization strategy of E<strong>learning</strong><br />
scenarios”. Computers in Human Behavior 26(4): 581-591 (2010)<br />
iCOPER (2011). EU eContentplus Programme’s iCOPER (Interoperable Content for Performance in a<br />
Competency-Driven Society) Best Practice Network web site. D. 3.1. Annex A. Available online at<br />
http://www.icoper.org/<br />
IMS LD (2003) “IMS Learning Design Information Model. Version 1.0 Final Specification”. Available online at<br />
http://www.imsglobal.org/<strong>learning</strong>design/ldv1p0/imsld_infov1p0.html#1495449<br />
Honey, P. & Mumford, A. (1982) “Manual of Learning Styles”. London: P Honey, 1982<br />
iTEC (Innovative Technologies for an Engaging Classroom) project website (2011). Available online at<br />
http://itec.eun.org/web/guest/<br />
Kolb, D.A. (1984) “Experiential Learning: experience as the source of <strong>learning</strong> and development”. Englewood<br />
Cliffs, Prentice Hall, 1984<br />
Koper, E.J.R., and Tattersall, C. (Eds.) (2005) “Learning Design: A handbook on Modelling and delivering<br />
Networked Education and Training”. Heidelberg: Springer<br />
Kurilovas, E., Bireniene, V., Serikoviene, S. (2011) “Methodology for Evaluating Quality and Reusability of<br />
Learning Objects”. The Electronic Journal of e-Learning, Vol. 9, Issue 1, 2011, pp. 39–51. Available online<br />
at www.ejel.org<br />
Kurilovas, E., Dagiene, V. (2010) “Multiple Criteria Evaluation of Quality and Optimisation of e-Learning System<br />
Components”. The Electronic Journal of e-Learning, Vol. 8, Issue 2, 2010, pp. 141–150. Available online at<br />
www.ejel.org<br />
Kurilovas, E., Dagiene, V., (2009) “Multiple Criteria Comparative Evaluation of e-Learning Systems and<br />
Components”. Informatica. 2009, Vol. 20, No. 4, pp. 499–518<br />
Kurilovas, E., Dagiene, V. (2009) “Learning Objects and Virtual Learning Environments Technical Evaluation<br />
Criteria”. The Electronic Journal of e-Learning, Vol. 7, Issue 2, 2009, pp. 127–136. Available online at<br />
www.ejel.org<br />
Kurilovas, E., Serikoviene, S. (2010) “Application of Scientific Approaches for Evaluation of Quality of Learning<br />
Objects in eQNet Project”. In: M.D. Lytras et. al. (Eds.): WSKS 2010, Part I, CCIS 111, pp. 437–443.<br />
Springer, Heidelberg 2010<br />
Kurilovas, E., Serikoviene, S., (2010) “Learning Content and Software Evaluation and Personalisation Problems”.<br />
Informatics in Education, 2010, Vol. 9 (1), pp. 91–114<br />
Lin, Hsiu-Fen (2010) ”An application of fuzzy AHP for evaluating course website quality”. Computers & Education<br />
54 (2010), p. 877–888<br />
Mulwa, C., Lawless, S., Sharp, M., Wade, V. & Arnedillo-Sanchez, I. (2010) “Adaptive Educational Hypermedia<br />
Systems in Technology Enhanced Learning: A Literature Review”. ACM Special Interest Group for<br />
Information Technology Education Conference (SIGITE), Central Michigan University, Midland, MI,<br />
USA, October 7th-9th, 2010, pp. 1-12<br />
O' Keeffe, I., Brady, A., Conlan, O. & Wade, V. (2006) "Just-in-time Generation of Pedagogically Sound, Context<br />
Sensitive Personalized Learning Experiences", International Journal on E-Learning (IJeL), Special Issue:<br />
Learning Objects in Context," International Journal on E-Learning (IJeL), vol. 5, pp. 113-127, 2006<br />
Ounaies, H.Z., Jamoussi, Y., Ben Ghezala, H.H., (2009) “Evaluation framework based on fuzzy measured<br />
method in adaptive <strong>learning</strong> system”. Themes in Science and Technology Education, 2009, Vol. 1, Nr. 1,<br />
pp. 49–58<br />
Saaty, T. L. (1990) “How to make a decision: The analytic hierarchy process”. European Journal of Operational<br />
Research, 48(1), 9–26<br />
Saaty, T. L. (2008) “Relative Measurement and its Generalization in Decision Making: Why Pairwise<br />
Comparisons are Central in Mathematics for the Measurement of Intangible Factors - The Analytic<br />
Hierarchy/Network Process”. RACSAM (Review of the Royal Spanish Academy of Sciences, Series A,<br />
Mathematics) 102 (2), 2008, p. 251–318.<br />
Shee, Daniel Y., Wang, Yi-Shun (2008) “Multicriteria evaluation of the web-based e-<strong>learning</strong> system: a<br />
methodology based on learner statisfaction and its application”. Computers and Education 50(48), 894-905.<br />
Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y., & Yeh, D. (2008) “What drives a successful e-<strong>learning</strong>? An<br />
empirical investigation of the critical factors influencing learner satisfaction”. Computers and Education, 50,<br />
pp. 1183–1202<br />
Zavadskas, E. K., Turskis, Z., (2010) “A new additive ratio assessment (ARAS) method in multicriteria decisionmaking”.<br />
Technological and Economic Development of Economy 16(2): 159–172<br />
Zhang Li Li, Cheng De Yong, (1992) “Extent analysis and synthetic decision”. In: Support Systems for Decision<br />
and Negotiation Processes, Preprints of the /FAC/IFORS/IIASA/TIMS Workshop, Vol. 2. System Research<br />
Institute, Warsaw, pp. 633-640<br />
389
Models of eLearning: The Development of a Learner-<br />
Directed Adaptive eLearning System<br />
Stella Lee 1 , Trevor Barker 1 and Vive Kumar 2<br />
1<br />
Computer Science Department, University of Hertfordshire, Hatfield, UK<br />
2<br />
School of Computing & Information Systems, Athabasca University,<br />
Edmonton, Canada<br />
stellaylee@gmail.com<br />
t.1.barker@herts.ac.uk<br />
vive@athabascau.ca<br />
Abstract: In many eLearning contexts, materials are designed to be self-paced, with the content being available<br />
anytime, anywhere for learners to study independently. Commonly, without the presence and immediate<br />
feedback of an instructor, distance learners are left to their own devices to negotiate their <strong>learning</strong> path and to<br />
monitor their own progress. Furthermore, <strong>learning</strong> a complex topic structured in terms of various media and<br />
<strong>learning</strong> materials requires learners to make certain instructional decisions concerning what to learn and how to<br />
go about their <strong>learning</strong>. In other words, self-paced <strong>learning</strong> requires learners to self-regulate their own<br />
<strong>learning</strong>(Hadwin & Winne, 2001). Very often, learners have difficulty regulating <strong>learning</strong> in higher education when<br />
topics are complex and unfamiliar and it is not always clear to the learners if their instructional decisions are<br />
optimal.(Azevedo, Cromley, Seibert, & Tron, 2003) Research into adaptive eLearning systems has attempted to<br />
facilitate this process by providing recommendations, classifying learners into different preferred <strong>learning</strong> styles,<br />
or highlighting suggested <strong>learning</strong> paths(Brusilovsky, 1998).The aim of this research is to explore how learners<br />
can self-directed and self-regulate their online <strong>learning</strong> both in terms of domain knowledge and meta knowledge<br />
in the subject of computer science with a flexible and adaptive eLearning system. Two educational theories:<br />
experiential <strong>learning</strong> theory (ELT) and self-regulated <strong>learning</strong> (SRL) theory are used to aid learners’ in their<br />
<strong>learning</strong> paths. As a result, changes in domain-knowledge, meta-knowledge, learner experience, learner<br />
satisfaction, perceived controllability, and system usability are being measured. All in all, this paper sums up the<br />
research work being done on the initial development of the system, instructional design framework based on the<br />
two theories, experimental design plan and course material examples as well as related issues.<br />
Keywords: adaptive systems, eLearning, instructional design, <strong>learning</strong> design<br />
1. Introduction<br />
eLearning has become so prevalent in higher education and corporate training that we have seem<br />
every effort being made to reshape and digitized contents online in the past decade or so. ELearning<br />
uses the internet to deliver instructions to the learner and it may be argued that one of the most<br />
popular forms of eLearning is web-based distance <strong>learning</strong> (Koohang & Du Plessis, 2004) In many<br />
cases, these web-based <strong>learning</strong> materials are designed to be self-paced, with the content being<br />
available anytime, anywhere to allow learners to study independently. Commonly, without the<br />
presence and immediate feedback of an instructor, distance learners are left to their own devices to<br />
negotiate their <strong>learning</strong> path and to monitor their own progress. Furthermore, <strong>learning</strong> a complex topic<br />
structured in terms of various media and <strong>learning</strong> materials requires learners to make certain<br />
instructional decisions concerning what to learn and how to go about <strong>learning</strong>. In other words,<br />
eLearning requires learners to self-regulate their own <strong>learning</strong>(Hadwin & Winne, 2001). Very often,<br />
learners have difficulty regulating <strong>learning</strong> when topics are complex and unfamiliar and it is not always<br />
clear to the learners if their <strong>learning</strong> decisions are optimal(Azevedo et al., 2003). Research into<br />
adaptive eLearning systems has attempted to facilitate the <strong>learning</strong> process by providing<br />
recommendations with respect to classifying learners into preferred <strong>learning</strong> styles and by associating<br />
recommended <strong>learning</strong> paths with these <strong>learning</strong> styles(Brusilovsky, 1998). Indeed, research has<br />
shown the importance of adapting online course material to support learners with different<br />
background knowledge and skills(Brusilovsky, 1998; Weber, 1999). Another work has described a<br />
user modeling approach that is beneficial to learners who are interacting with complex <strong>learning</strong><br />
applications in an online environment(Adisen & Barker, 2007).<br />
Broadly speaking, user modeling is a technique employed to provide users with options with respect<br />
to performing tasks and interacting with systems differentially. The common variables to model<br />
include the user’s personalities, abilities, prior knowledge, preferences, performances and<br />
intentions(Barker & Adisen, 2005). User modeling can also be defined as a model of users residing<br />
inside a system or computational environment(Fischer, 2001). However, many of these models<br />
attempt to “match” students with a certain <strong>learning</strong> styles or learner characteristics that it falls short on<br />
390
Stella Lee et al.<br />
placing the initiative on the learners themselves and help them make aware of their own <strong>learning</strong> and<br />
in control of their <strong>learning</strong> paths. In fact, there has been no evidence in matching teaching to <strong>learning</strong><br />
styles as the best way to improve <strong>learning</strong>. Instead, there is an optimal way to teach chunk of content<br />
to all students(Pashler, McDaniel, Rohrer, & Bjork, 2008). In this study, our modeling approach is<br />
centred around a constructive cognitive model of <strong>learning</strong> (Liu & Matthews, 2005). This approach<br />
involves a model that helps make <strong>learning</strong> constructive while providing adaptive feedback to learners.<br />
As a result, an adaptive eLearning system has been designed that is task-based and support learners<br />
to self-direct their own <strong>learning</strong> paths by choosing the <strong>learning</strong> preferences (or <strong>learning</strong> modes as we<br />
called them) as they see fit. Additionally, contextually appropriate meta level skills (i.e. study skills)<br />
information and tools are provided along with the domain-level material. Adaptive feedback is<br />
provided at the end in the form of a self quiz to evaluate whether the learners have progress through<br />
their <strong>learning</strong> with adequate domain and meta knowledge.<br />
The domain we have chosen for this study is in computer science. Specifically, we are conducting our<br />
research study within an introductory Java programming course at a distance <strong>learning</strong> university in<br />
Canada. Two relevant educational theories: experiential <strong>learning</strong> theory (ELT) and self-regulated<br />
<strong>learning</strong> (SRL) have been selected as the underlining instructional design principles for this eLearning<br />
system. We will discuss more about these theories in Section 2. Section 3 will cover the planned<br />
research and the experimental design in details while Section 4 provides examples of the material we<br />
have redesigned to suit this <strong>learning</strong> approach. Finally we conclude with discussion and future work in<br />
Section 5.<br />
2. Background on the two educational theories used<br />
Two educational theories have emerged from authors’ experiences in designing eLearning models<br />
and reviewing relevant literature in <strong>learning</strong> preferences and online <strong>learning</strong> - the Experiential<br />
Learning Theory (ELT) and Self-Regulated Learning (SRL) theory. The fundamental ideas underlying<br />
the two theories are explained below.<br />
2.1 Experiential Learning Theory (ELT)<br />
Developed in the early 1970's by David Kolb, Experiential Learning Theory (ELT) emphasizes<br />
experience as the central focus in <strong>learning</strong>. Learning-styles theories(Coffield, Moseley, Hall, &<br />
Ecclestone, 2004) raised questions about what <strong>learning</strong> strategies we operate with and how we use<br />
<strong>learning</strong> strategies to enhance student <strong>learning</strong>. By studying these theories, one can also gain<br />
insights into what motivates learners and how to help them to understand more about their own<br />
strengths and weaknesses as learners. Kolb's model of <strong>learning</strong> styles could be represented as<br />
"flexible stable <strong>learning</strong> preferences"(Coffield et al., 2004) as ELT is not about fixed learner traits, but<br />
rather a "differential preference for <strong>learning</strong>, which changes slightly from situation to situation”. For<br />
instance, a student working in a group setting during a field trip for geo-caching might prefer to take a<br />
turn at hands-on interaction with the device even though he would normally prefer reading textual<br />
instruction at home alone. In his research, Kolb observed that some students have a definite<br />
preference for certain <strong>learning</strong> activities(Kolb, 1984). For example, one student might prefer reading<br />
lecture notes whilst another student might prefer working on an interactive simulation. “From this<br />
emerged the idea of an inventory that would identify these preferences by capturing individual<br />
<strong>learning</strong> differences.” (Kolb & Kolb, 2005)<br />
Figure 1 shows how a learner can progress through the experiential <strong>learning</strong> cycle: concrete<br />
experience (and new information acquired) is translated through reflection into concepts, which are in<br />
turn used as guides for active experimentation and the choice of new experiences. Kolb stated that a<br />
learner can begin the <strong>learning</strong> cycle at any one of the four modes, but that <strong>learning</strong> should be carried<br />
on as a continuous spiral. As a result, knowledge is constructed through the creative tension among<br />
the four modes and learners will be exposed to all aspects of <strong>learning</strong>: experiencing, reflecting,<br />
thinking and acting.<br />
Furthermore, there are four dominant <strong>learning</strong> styles that are associated with these modes:<br />
converging, diverging, assimilating and accommodating(Kolb, 1984).<br />
According to Kolb, the four <strong>learning</strong> styles are:<br />
391
Stella Lee et al.<br />
Converging (AC and AE focus) - interested in practical applications of concepts and theories. A<br />
learner who has this preference is good at problem solving, decision making and practical<br />
application of ideas.<br />
Diverging (CE and RO focus) - interested in observation and collection of a wide range of<br />
information. A learner who is strong in this preference is imaginative and aware of meanings and<br />
values. They are interested in people and are socially inclined.<br />
Assimilating (AC and RO focus) - interested in presentation and creation of theoretical models. A<br />
learner leaning toward this preference is more concerned with ideas and abstract concepts than<br />
with people.<br />
Accommodating (CE and AE focus) - interested in hands-on experiences. A learner with this<br />
preference likes doing things; carry out plans and trial-and-error method.<br />
Figure 1: Kolb's Experiential Learning Cycle<br />
Our research is grounded in individualized instruction and the intention is to make explicit to the<br />
learners why we are employing this instructional design approach. To further Kolb's take on distinctive<br />
<strong>learning</strong> preferences and to be able to provide options on <strong>learning</strong> activities accordingly, ELT has<br />
been used to design task-level content and skills in Java programming along the two dimensions of<br />
Concrete Experience-Abstract Conceptualization (CE-AC) and Reflective Observation-Active<br />
Experimentation (RO-AE). Content for the Java programming course has been redeveloped to<br />
present the course online content along the four modes of <strong>learning</strong>. The four modes of <strong>learning</strong> will be<br />
discussed in details in Section 3.<br />
2.2 Self-Regulated Learning (SRL) theory<br />
Self-Regulated Learning (SRL) fits in well with a learner-initiated eLearning model because in an<br />
individualized <strong>learning</strong> environment, learners are often left to their own devices, and a flexible<br />
eLearning system can help make them aware of their own thinking, monitoring, planning and<br />
evaluating personal progress against a standard, and motivation to learn(Boekaerts & Corno, 2005;<br />
Butler & Winne, 1995; Perry, 2006; P.H. Winne & Perry, 2000; Zimmerman, 1990). When learners<br />
adapt their approaches to <strong>learning</strong>, <strong>learning</strong> is said to be self-regulated(Winne, 1997). Self-regulated<br />
<strong>learning</strong> (SRL) states that learners not only need to regulate their performance, but also how they<br />
learn. Literature shows that self-regulation can positively affect learners’ achievement(Azevedo,<br />
2005). Self-regulated <strong>learning</strong> (SRL) is “an active, constructive process whereby learners set goals for<br />
their <strong>learning</strong> and then attempt to monitor, regulate, and control their cognition, motivation, and<br />
behavior” (Pintrich, 2000). Furthermore, Hadwin, Wozney, & Pontin (2005) revealed that ‘ordinary’<br />
collaboration (as found in traditional online forums and chats) is insufficient to transform learners’<br />
everyday “reflection” into productive “regulation”.<br />
392
Stella Lee et al.<br />
What sets self-regulated learners apart is their awareness of when they know a skill or fact and when<br />
they do not, at a meta knowledge level - i.e. they plan, set goals, organize, self-monitor, and selfevaluate<br />
thorough out their studies(Zimmerman, 1990). In addition, self-regulation works best when<br />
learners are provided with continual feedback concerning the effectiveness of their <strong>learning</strong> approach.<br />
This is something that an adaptive system can provide as it can support and reinforce self-monitoring<br />
techniques as well as self-regulated <strong>learning</strong> strategies. We mainly focus on this aspect of selfregulation<br />
in relation to the design of an eLearning system wherein students can receive feedback if<br />
they choose to do so.<br />
Our model’s SRL interface includes components that allow students to self-reflect along with<br />
components that engage them in self-regulation. On one hand of the spectrum, self-reflection could<br />
be as simple as a thinking process made self aware, the intangible 'ah ha' moment of a conceptual<br />
breakthrough; on the other hand, self-regulation could be more tangible as a system or a tutor can<br />
observe what students do after they self-reflect. For example, in debugging, a student can self-reflect<br />
on errors identified by the compiler at the end of each compile of the program being developed. The<br />
system can track/record the number of errors and warnings faced by the student at the end of each<br />
compile. The system can also classify the types of errors encountered by a single student over<br />
multiple sessions of program development. Looking at this list of errors is an act of self-reflection.<br />
However, self-regulation takes it one step further. The student may try to identify most common errors<br />
and warnings he/she faced, take notes on how he/she resolved these common errors and warnings,<br />
and refer to these notes when he/she encounters these common errors and warnings when writing<br />
another program. This "proactive self-reflection" is what we identified as self-regulation even thought<br />
at this stage, we are not planning on tracking the end results of using the study skill tools, it is the<br />
provision of these “how to” study skill guides and tools embedded as options for students to assist<br />
them in becoming better self-regulated, self-reflected learners that we are interested in.<br />
3. Program of research<br />
3.1 Participants<br />
In order to undertake this program of research, 60-80 voluntary participants who enroll in COMP 268 -<br />
Introduction to Java Programming course at a distance <strong>learning</strong> university in Canada will participate in<br />
the study. The participants are adult learners in various age groups with various programming skills.<br />
This is a 100 level (all introductory computer science courses start with the number “2” at this<br />
institution) course and there is no prerequisite. We expect participants to have at least basic<br />
computing and distanceLearning skills.<br />
3.2 Material and procedure<br />
One unit from COMP 268 - Introduction to Java Programming course consists of five <strong>learning</strong><br />
concepts has been redesigned and redeveloped to presents a series of <strong>learning</strong> activities designed<br />
according to the ELT theory - i.e. the material has been designed to be presented in four different<br />
<strong>learning</strong> modes: watching, discussing, diagramming, and trying out. These four modes correspond to<br />
ELT’s four <strong>learning</strong> dimensions as discussed in Section 2. For example, for “watching” mode, the<br />
material might be presented in a YouTube video tutorial with narration and textual explanation. A<br />
learner is free to choose any of the <strong>learning</strong> activities to begin his/her study. The system doesn’t<br />
initiate or suggest any particular <strong>learning</strong> paths. The idea is that we are giving the learner complete<br />
control with respect to which <strong>learning</strong> activity suits his/her <strong>learning</strong> preference at the time of the study<br />
while providing <strong>learning</strong> options. According to ELT, a learner can begin at any stage of the <strong>learning</strong><br />
cycle, but it is most beneficial for acquiring new knowledge if he/she would go through them all<br />
eventually to fully understand and apply this new knowledge. He/she might or might not choose to go<br />
through them all; the decision is his/her to make. If at the end of any chosen <strong>learning</strong> activity, the<br />
learner feels that he/she has a good grasp of the material, he/she can opt to skip the rest of the cycle<br />
and go to the next topic. A post-test is available at the end of the unit for learner to self-assess his/her<br />
knowledge both at the domain and meta level. In another word, learners can test how well they have<br />
learn a certain programming concepts in Java as well as whether the associated key study skills are<br />
helpful to them in regulating their own <strong>learning</strong> and deploying them as <strong>learning</strong> strategies. The posttest<br />
is a formative test and it will not affect the participants’ grade in their overall course performance.<br />
In addition, as supplementary <strong>learning</strong> material, a SRL theory based interface titled “Key Study Skills”<br />
is available on the upper right-hand corner of the webpage to assist with <strong>learning</strong> about meta<br />
393
Stella Lee et al.<br />
knowledge. Meta knowledge in terms of key study skills for studying programming have been selected<br />
as: note taking skill, reflection and discussion skill, conceptualization skill, and problem solving skill.<br />
For example, within the “Watching” activity, the key study skill we emphases is “note taking”. Learners<br />
can choose to access this supplementary material at any given time during his/her study. The design<br />
consideration is that the key study skills appear as an unobtrusive interface that sits on the upperright-hand-corner<br />
of the main content. Learners can choose to engage and learn how the “note<br />
taking” skill helps with the “watching” activity (i.e. how to take better notes while watching a tutorial<br />
video online and what are the appropriate tool to use for note taking), or he/she can just ignore it and<br />
carry on with watching the YouTube tutorial video on its own.<br />
One pre-test has been developed with 25 questions for domain-knowledge and 20 questions for<br />
meta-knowledge (study skills). In this test we are looking at two types of data:<br />
Test score for the domain-knowledge - to gauge how much a student has already know about<br />
control structures in java programming prior to this unit. This test score will be compared to the<br />
post-test domain-knowledge test score.<br />
Test score for meta-knowledge (study skills) - to gauge a student’s study skill competency. This is<br />
more of a self-report rather than an objective measurement of specific study skills. This test score<br />
will be compared to the post-test meta-knowledge test score to identify whether the study skills<br />
provided help improve students’ overall study strategies and make them aware of certain metalevel<br />
skills.<br />
Similar to the pre-test, one post test is available with 25 questions for domain knowledge and 20<br />
questions for meta-knowledge (study skills). It is designed to measure the efficacy of each <strong>learning</strong><br />
mode. Questions are generated based on the content from select mode of each <strong>learning</strong> concept. For<br />
each answer, the system will then give feedback and suggestions according to the test results, and<br />
direct the student to focus more in the <strong>learning</strong> mode in which the student is weak. The aim is to help<br />
learners to become more efficient and aware in self-monitoring their <strong>learning</strong>.<br />
It is intended that there will be an end-of-unit survey and a follow-up structured interviews with 6-10<br />
participants two weeks upon the completion of the study. The survey aims to measure learner<br />
experience, usability, user satisfaction and perceived controllability of the system. At the end of the<br />
survey, we will invite participants to take part in a 40-min structured interview using Adobe Connect.<br />
Participation in the interview is entirely voluntary and has no effect on their grades. The purpose of<br />
the interview is to discover more details about the learners’ attitudes and impression about the<br />
experiment, and give learners a chance to elaborate on the survey results. Questions will also be<br />
geared towards whether they find the system and the options provided helpful, in both the domain<br />
specific content as well as the meta-cognitive activities, i.e., study skills.<br />
3.3 Research questions<br />
This study was designed to answer the following research questions:<br />
How would the two named <strong>learning</strong> theories, ELT and SRL, be useful in the production of<br />
constructive adaptive system?<br />
How might such a system be tested and evaluated in a real context?<br />
What are the effects of such a system on:<br />
Domain-knowledge competency<br />
Meta-knowledge competency<br />
What are the effects of such a system on:<br />
Learners’ experience<br />
System usability<br />
User satisfaction<br />
Perceived controllability<br />
394
3.4 Modeling approach<br />
Stella Lee et al.<br />
The form of modeling we used is a learner-directed model. A learner initiated system as a system that<br />
places full <strong>learning</strong> control in the hands of the learner. The system does not automatically adapt or<br />
assign any differentiated content to the learner(Marzano, 1992). Simply put, the system serves a<br />
suggested guide to the learner where he/she can accept or reject the suggestions. This approach<br />
enables personal autonomy, active involvement and engagement with the content and serves as a<br />
motivational factor for online <strong>learning</strong>. For <strong>learning</strong> styles, learners will self-select from the multiple<br />
<strong>learning</strong> content representations, so there is no need for an automated system intervention. For prior<br />
knowledge and skills, the comparisons of the pre-test and post-test results will inform the system as to<br />
what to recommend and adapt at the end. If the post-test results indicate a deficiency in certain<br />
domain knowledge area, then the system will provide appropriate feedback to the students. It is not so<br />
much a specific prescription; rather, it is used as reference guide.<br />
4. Example of the course<br />
Prototype of the eLearning system has been developed in Moodle (an open source Learning<br />
Management System) providing alternative content for Unit 3 of the course for our study. Traditionally,<br />
Unit 3, similar to the rest of the course, is written in a linear, textual format with heavy reference to the<br />
textbook.<br />
Currently, after the redesigned of the unit, unit 3 composes of five <strong>learning</strong> concepts and each<br />
concept starts with the same home page interface that present the four modes of <strong>learning</strong>. The five<br />
<strong>learning</strong> concepts are: If-Else Statement; Loops; Break, Continue, and Return; Switch, Try-Catch-<br />
Finally, and Labels; and Recursion. Figure 2 below illustrates the Unit 3 layout design for the course<br />
while Figure 3 shows the home page of each <strong>learning</strong> concepts.<br />
Figure 2: Unit 3 home page as viewed by students entering into the experiment<br />
Once a student access the <strong>learning</strong> concept home page, a circular wheel interface with four<br />
quadrants will be presented to them. Each quadrant is linked to a different but comparable <strong>learning</strong><br />
activity based on Kolb’s experiential <strong>learning</strong> cycle: experiencing, reflecting, thinking and acting. For<br />
instance, the experiencing, the <strong>learning</strong> activity is presented as a series of tutorial videos on YouTube<br />
on how to program If-Statements (see figure 4).<br />
395
Stella Lee et al.<br />
Figure 3: Home page for <strong>learning</strong> concept 1 - If-Else Statement<br />
Figure 4: YouTube tutorial videos on how to program If-Statements under the “Watching” quadrant<br />
<strong>learning</strong> activity<br />
Twenty study skills --five successive skills for each of the four <strong>learning</strong> modes-- have been developed.<br />
The four key study skill areas we focus on are: note taking, communications, conceptualizing, and<br />
problem solving skills. Each study skill is paired with a tool for learners to try out and practice these<br />
skills. Figure 5 shows that the Note Taking skill is being presented with Evernote, a note-taking<br />
software. The rest of the study skills are paired with the following: communications – Moodle<br />
discussion forum; conceptualizing – FreeMap (mind mapping tool); and problem solving – BlueJ (an<br />
integrated development environment tool).<br />
396
Stella Lee et al.<br />
Figure 5: One of the key study skill – Note taking skill along with suggested tool to use<br />
5. Discussion<br />
The research reported here relates to the design of a quite significant implementation of an eLearning<br />
system in a pragmatic educational context based on constructive models of <strong>learning</strong>. The approach<br />
differs in several ways from other approaches we have used in the past. In the first place the<br />
modeling approach is intended to foster a constructive approach to <strong>learning</strong>. To this end two <strong>learning</strong><br />
theories have been employed in the design of the system. Modeling approaches we have used in the<br />
past have been to some extent instructive in that they have been used (Adisen & Barker, 2007;<br />
Adisen, Barker, & Britton, 2004; Barker, Jones, Britton, & Messer, 2002; Brusilovsky, 1998). In the<br />
authors’ opinion, eLearning 2.0 implies self-directed <strong>learning</strong> and a student taking personal<br />
responsibility for his or her <strong>learning</strong>. The question remains: how can a system based on a modeling<br />
approach foster constructive rather than instructive <strong>learning</strong>? The ways that student models have<br />
been used in the past to recommend or prescribe presentation can be seen as fostering instruction<br />
rather than construction of knowledge. Secondly in our approach the model we use relates directly to<br />
the tasks that learners are undertaking and their approaches to them, rather than to models of an<br />
individual learner. Our modeling approach does not attempt to model the characteristics or<br />
knowledge of a learner, but rather models the relationship between the task the learners are<br />
undertaking and the level of confidence, skill and general strategy they are using. The support and<br />
presentation strategy provided for learners is therefore not intended to deliver instruction differentially<br />
based on settings of a student model, but rather is intended to help the learner to construct his or her<br />
own <strong>learning</strong> path through the material in order to solve a problem or to learn new material related to<br />
a problem. The use of <strong>learning</strong> theory or cognitive style was previously used in our research to<br />
classify or categorize learners(Barker et al., 2002). In the current approach we have avoided this<br />
method and have instead modeled the tasks and the approaches to solving them that best fits the<br />
needs of an individual learner. In the next stage of the research a full-scale evaluation of the initiative<br />
is currently underway with learners at an online university in Canada. It is hoped to gain an<br />
understanding of how this approach may be of benefit to learners.<br />
References<br />
Adisen, A., & Barker, T. (2007). Supporting the diversity of the ELearning 2.0 learners: The development of a<br />
psychological student model. Paper presented at the E-Learn 2007 Conference, Quebec City, Canada.<br />
Adisen, A., Barker, T., & Britton, C. (2004). Investigating the potential of mental models in adaptive user<br />
modelling. Paper presented at the Proceedings of HCI 2004: Design for Life, Leeds Metropolitan University,<br />
Leeds, UK.<br />
Azevedo, R. (2005). Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of<br />
Self-Regulated Learning. Educational Psychologist, 40(4), 199-209.<br />
397
Stella Lee et al.<br />
Azevedo, R., Cromley, J. G., Seibert, D., & Tron, M. (2003). The role of co-regulated <strong>learning</strong> during students’<br />
understanding of complex systems with hypermedia. Paper presented at the Annual meeting of the<br />
American Educational Research Association.<br />
Barker, T., & Adisen, A. (2005). Experiments on visual and verbal skills: The development and testing of a skills<br />
profile. Paper presented at the Proceedings of the European Learning Styles Information Network<br />
Conference, Univerity of Surrey, Surrey, England.<br />
Barker, T., Jones, S., Britton, C., & Messer, D. (2002). The use of a co-operative student model of learner<br />
characteristics to configure a multimedia application. User Modelling and User Adapted Interaction, 12, 207-<br />
241.<br />
Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and<br />
intervention. Applied Psychology: An International Review, 54(2), 199-231.<br />
Brusilovsky, P. (1998). Adaptive educational systems on the World-Wide-Web. Paper presented at the Proc. of<br />
Workshop "Current Trends and Applications of Artificial Intelligence in Education" at 4th World Congress on<br />
Expert Systems.<br />
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated <strong>learning</strong>: A theoretical synthesis. Review of<br />
Educational Research, 65, 245-281.<br />
Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 <strong>learning</strong>: A<br />
systematic and critical review. London: Learning and Skills Research Centre.<br />
Fischer, G. (2001). User modeling in human-computer interaction. User modeling and user adapted interaction,<br />
11, 65-86.<br />
Hadwin, A. F., Wozney, L., & Pontin, O. (2005). Scaffolding the appropriation of self-regulatory activity: A sociocultural<br />
analysis of changes in teacher-student discourse about a graduate research portfolio. Instructional<br />
Science, 33, 413-450.<br />
Hadwin, F. H., & Winne, P. H. (2001). CoNoteS2:A Software Tool for Promoting Self-Regulation. Educational<br />
Research and Evaluation, 7(2-3), 313-334.<br />
Kolb, A., & Kolb, D. (2005). Learning styles and <strong>learning</strong> spaces: Enhancing experiential <strong>learning</strong> in higher<br />
education. Academy of Management Learning & Education, 4(2).<br />
Kolb, D. (1984). Experiential <strong>learning</strong>: Experience as the source of <strong>learning</strong> and development. Englewood Cliffs,<br />
NJ: Prentice-Hall.<br />
Koohang, A., & Du Plessis, J. (2004). Architecting usability properties in the eLearning instructional design<br />
process. International Journal on ELearning, 3(3).<br />
Liu, Y. & Matthews, K.S., (2005), Vygotsky’s philosophy: Constructivism and its criticisms examined, International<br />
Education Journal, 2005, 6(3), 386-399.<br />
Marzano, R. J. (1992). A different kind of classroom: Teaching with dimensions of <strong>learning</strong>. Alexandria, VA:<br />
Association for Supervision and Curriculum Development.<br />
Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning Styles: Concepts and Evidence.<br />
Psychological Science in the Public Interest, 9(3), 105-119.<br />
Perry, N. E., Phillips, L., & Hutchinson, L.R. . (2006). Preparing student teachers to support for self-regulated<br />
<strong>learning</strong>. . Elementary School Journal, 106, 237-254.<br />
Pintrich, P. R. (2000). The role of goal orientation in self-regulated <strong>learning</strong>. In P. P. M. Boekaerts, & M. Ziedner<br />
(Ed.), Handbook of self-regulation (pp. 451-502). San Diego, CA: <strong>Academic</strong> Press.<br />
Weber, G. (1999). Adaptive <strong>learning</strong> systems in the World Wide Web. Paper presented at the 7th International<br />
Conference on User Modeling, UM99, Banff, Canada.<br />
Winne, P. H. (1997). Experimenting to bootstrap self-regulated <strong>learning</strong>. Journal of Educational Psychology,<br />
89(3), 397-410.<br />
Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated <strong>learning</strong>. In M. B. P. Pintrich, & M. Seidner (Ed.),<br />
Handbook of self-regulation (pp. 531-566). Orlando, FL: <strong>Academic</strong> Press.<br />
Zimmerman, B., J,. (1990). Self-regulated <strong>learning</strong> and academic achievement: An overview. Educational<br />
Psychologist, 25(1), 3-17.<br />
398
Can eLearning Enhance Practice-Based Design Courses?<br />
Jake Leith, Joanna Zara and Malcolm McInnes<br />
University of Brighton, Brighton and Hove, UK<br />
J.Q.Leith@brighton.ac.uk<br />
J.M.Zara@brighton.ac.uk<br />
M.R.McInnes@brighton.ac.uk<br />
Abstract: In recent years the focus for UK Higher Education has been on improving student experiences;<br />
technology has played a crucial role in transforming the curriculum, evidenced in a series of initiatives funded by<br />
the Joint Information Systems Committee (JISC) (HEFCE, 2009). The adoption of online <strong>learning</strong> and teaching<br />
techniques on practice-based courses, within the field of art and design, have often been seen as controversial,<br />
due to the vocational nature of the discipline. This paper seeks to understand, through the application of action<br />
research, what affect the incorporation of blended <strong>learning</strong> has had on the professional practice element of a<br />
practice-based course and compare students’ responses to results published by one of JISC’s completed<br />
programmes, ‘Learner Experiences of eLearning.’ (JISC, 2009) Objectives To identify how a blended programme<br />
of eLearning and face-to-face teaching of business and professional practice can support and enhance <strong>learning</strong><br />
in design-led practice-based courses To understand how the eLearning method of teaching delivery can<br />
contribute to flexibility of <strong>learning</strong> in design-led practice based courses. To identify areas where such an<br />
approach contributes to the best use of resources in response to the impact of socio-economic conditions on the<br />
education sector. Work to be carried out To design and live test the delivery of a business and professional<br />
practice programme using a blend of eLearning and face-to-face teaching. To identify the perceived/real benefits<br />
and limitations of such an approach by the analysis of interviews/focus groups/questionnaires with students and<br />
staff from the School of Architecture and Design at the University of Brighton. To identify the elements of course<br />
delivery that best suit an online approach and those that are more suited to a face-to-face model. To identify the<br />
key resources required for this eLearning method of delivery and how they differ from those required for a more<br />
traditional face-to-face approach. Compare cohorts who have adapted to the blended <strong>learning</strong> model of teaching<br />
from the more traditional face-to-face <strong>learning</strong>, with those who have experienced blended <strong>learning</strong> from the<br />
beginning of their course.<br />
Keywords: online, blended, practice-based, eLearning, design<br />
1. Methodology<br />
Art and design education within the Faculty of Arts at the University of Brighton, follows a recognised<br />
constructivist tradition within the disciplines, encouraging a continuous process of inquiry,<br />
experimentation and innovation that developments a student’s working methods and identity. This<br />
personal construction of knowledge is augmented by a collective <strong>learning</strong> experience that enables<br />
students to place their work within the professional context of the creative industries.<br />
The professional practice components of the course at level 4 and 5, also adopt a constructivist<br />
approach, with the production of a student-centred group project that is facilitated by the tutor, which<br />
encourages students to work together and take responsibility for their <strong>learning</strong> against a set of defined<br />
criteria. The pedagogical design of the online materials and activities were underpinned by this<br />
existing Constructivist approach. The principles outlined by Meyes & De Freitas (2004) were<br />
employed to provide a constructivist <strong>learning</strong> environment that gave students ownership of the task,<br />
helped model thinking skills, provided a scaffold for the project, guided discovery and afforded an<br />
opportunity for reflection.<br />
Activities and <strong>learning</strong> materials re-purposed or produced for online delivery were aligned with the<br />
<strong>learning</strong> outcomes of the project and were specifically designed to help students complete<br />
components of the assignment alongside face-to-face delivery. (Biggs, 1999).<br />
An action research model outlined by the Open University (2005) was adopted as a method to assess<br />
the effectiveness of the changes implemented and recognized the commitment of the tutors to the<br />
incorporation of technology as a continuous cycle of development. It enabled them to be active<br />
participants in the process of exploring student perceptions of the <strong>learning</strong> experience, while gaining<br />
feedback and reflecting how to affirm good practice and make further adaptations to the blended<br />
<strong>learning</strong> mix. This paper represents the findings from the first cycle of this action research.<br />
Blended <strong>learning</strong> within the context of this paper is defined as the delivery of a combination of face-toface<br />
and computer-mediated <strong>learning</strong> (Graham, 2005). But also recognizes the complexities and<br />
399
Jake Leith et al.<br />
opportunities blended <strong>learning</strong> can provide through a variety of new media resources to enrich the<br />
student experience, described by Littlejohn and Pegler (2007).<br />
2. Introduction to the project<br />
The two practice-based design courses examined in this project are the Fashion & Textiles and 3D<br />
Design & Materials Practice undergraduate courses run within the School of Architecture and Design<br />
at the University of Brighton. Both courses have contained a “Business and Professional Practice” unit<br />
- in varying forms - for over fifteen years.<br />
Students studying on these 2 courses are predominantly taught in studios and workshops for the<br />
practical aspects of the course and these facilities do not have access to computers. Professional<br />
Practice is one of the academic components of the course that requires students to produce written<br />
work for assessment that requires IT.<br />
The make-up of the first and second year cohorts (who are the focus of this research to date) is<br />
similar to previous years, with a high percentage of females (86% across both year groups) and a<br />
high percentage of students with dyslexia (23.5% across both year groups, compared to what is<br />
already considered a high level of around 6.5% within the School of Architecture and Design as a<br />
whole).<br />
Delivery and assessment of the first year unit is based on a “Business Case Study”: students work in<br />
groups (of 6 or 7) developing a business case study by observing a local creative business.<br />
Delivery and assessment of the second year unit is based on a “Group Marketing and Finance<br />
Assignment”: students again work in groups to investigate either a high street retail brand or a design<br />
practitioner, and use the results of their investigations to propose a priced and costed range/body of<br />
work which could be produced and sold by the brand/practitioner they have examined.<br />
In the academic year 2010/11 new online content delivery was introduced into these Business and<br />
Professional Practice Units. The aim was to provide an online enriched environment for both students<br />
and staff that would also have the potential to significantly streamline the 2009/10 programme, in<br />
terms of delivery, co-ordination, the volume of assessment-related administration and in handling<br />
increasing sizes of cohorts.<br />
This paper is intended to illustrate the impact of such an approach in the context of a practice-based<br />
design course; clearly the methods and technologies used are not new, and have been used in other<br />
non practice-based design courses both within the University and further afield, for some considerable<br />
time now.<br />
Whilst there has been some individual engagement with online <strong>learning</strong> tools for some final year<br />
students on the courses referred to (e.g. e-portfolios and the online submission of assignments), the<br />
programmes studied here represent the first strategic shift towards a fully blended <strong>learning</strong><br />
programme within the School of Architecture and Design, with specific online <strong>learning</strong> activities for<br />
students and online support tools for teaching staff.<br />
3. Reasons for adopting a blended <strong>learning</strong> approach<br />
The change to a blended <strong>learning</strong> approach had three main drivers:<br />
A recognition that this was an opportunity to provide an enriched <strong>learning</strong> environment for both<br />
students and staff.<br />
A decision to break the assessment process down into more “manageable chunks” across the<br />
academic year, thus avoiding the problems associated with a heavy assessment burden on staff<br />
at one time of year, but without adding to the overall time spent by staff on assessment related<br />
administration.<br />
Socio-economic conditions: in particular the need to streamline resources and reduce HPL (hourly<br />
paid lecturer) staffing whilst handling increasing student numbers. We have yet to see the full<br />
impact of funding changes upon the University, but this change in approach should enable us to<br />
respond as flexibly as possible, whilst maintaining the quality of the courses we deliver. We need<br />
400
Jake Leith et al.<br />
to bear in mind students’ expectations – to graduate with a level of employability skills that reflect<br />
the investment they have made in the course.<br />
The sector has placed increasing emphasis on the importance of effective eLearning in higher<br />
education. The HEFCE report ‘Effective Practice in a Digital Age’ highlights the current position. “Yet<br />
technology-enhanced <strong>learning</strong> remains a source of concern for institutions. This finding may reflect<br />
the extent to which supporting such practice makes demands on institutional resources, but also<br />
suggests a need to understand better how to design and support <strong>learning</strong> involving technology.<br />
Access, especially to the internet and social software, may have increased, but this does not mean<br />
that technology is always used to its best advantage, either by teachers or learners.” (HEFCE, 2009)<br />
4. Course changes and development<br />
During this first year of implementation the focus has been upon the identification of and the<br />
appropriate adaptation of existing content as a practical approach to undertaking <strong>learning</strong> re-design,<br />
given the limitation of available resources. The project demonstrates how existing materials can be<br />
repurposed to take advantage of some of the recognised benefits of eLearning (e.g. flexibility,<br />
<strong>learning</strong> at one’s own pace - perhaps particularly important in the case of the cohorts here, given the<br />
high proportion of students with dyslexia). It also acknowledges the findings of the LXP2 final<br />
synthesis report that technology should and can discretely facilitate <strong>learning</strong> for those with disabilities,<br />
e.g. podcasts for dyslexic students. (Sharpe, 2009)<br />
There was also a deliberate decision taken on the types of technology chosen. For instance, in order<br />
to retain the “personalisation” that exists in face-to-face delivery, regular audio podcasts and screen<br />
cast presentations were developed. The use of Turnitin software for online submission also enabled<br />
the use of formative peer assessment.<br />
Course content was structured around a “week by week” approach. Students accessing the unit were<br />
able to click on a link which would take them to the specific online content for that week, together with<br />
a note of any “live” delivery scheduled. This was supplemented by a traditional full printable academic<br />
year timetable available for download from the VLE (Virtual Learning Environment).<br />
Online content and tools provided included:<br />
Weekly podcasts from the course leader<br />
Staff introducing themselves in short video clips<br />
Videos of previous years’ students talking about the group projects they had undertaken<br />
Online access to previous years’ work submitted<br />
Online lectures in various formats (PowerPoints captured with Camtasia Relay; PowerPoints with<br />
recorded audio, recordings of live lectures)<br />
Online exercises;<br />
Links to online resources (for instance, i-player, Box of Broadcasts, market intelligence<br />
databases, reading lists);<br />
Online group tools within the VLE (email, blog, file exchange);<br />
Discussion boards (for use by students when undertaking group projects);<br />
Online submission and peer assessment (Using Turnitin software).<br />
5. Research methods<br />
Following the delivery of the units, initial questionnaires were sent to each year group studied; non<br />
course-specific questions were based as closely as possible upon those used in the LEaD (Learners’<br />
Experiences Across the Disciplines) project, undertaken by the University of Edinburgh as part of<br />
phase 2 of the JISC Learner Experiences of eLearning Programme conducted between 2005 and<br />
2009. This project had a similar focus, looking at the experiences of students upon entering HE, but<br />
was across a range of disciplines. The intention was to make use wherever possible of baseline data<br />
in order to understand whether there were any identifiable differences that could be explained in the<br />
context of the particular practice-based course focused on in our own project.<br />
401
Jake Leith et al.<br />
Data was collected anonymously after the group projects had been handed in; access to the survey<br />
questions was kept open for both year groups for one month. Some students may have received<br />
marks for the units in question by this stage, but due to the nature of the questions asked this should<br />
not have had a significant impact upon the replies given.<br />
20 responses were received from the first year cohort of 113 students, and 13 responses from the<br />
second year cohort of 96 students. We recognise that this is a small sample, but the responses seem<br />
to be representative of the cohort as a whole taking into account the qualitative data in the form of<br />
observations captured by staff over the course of the unit.<br />
One feature of the data captured so far is that there appears to be little discernable difference<br />
between the responses of the different year groups surveyed – for the next stage of this research we<br />
will be conducting focus groups from within each year group and will be looking for further evidence<br />
on this point.<br />
We have also documented the experiences of the professional practice staff team involved in the<br />
implementation of the programme.<br />
6. Findings<br />
6.1 Ownership and use of technology<br />
Computer ownership is almost ubiquitous: only one respondent did not have a computer of his/her<br />
own for exclusive use for University studies. This is comparable to the data collected from the LEaD<br />
project (94% PC ownership). Ownership of other electronic devices was similar across both cohorts<br />
as shown in Figure 1:<br />
Figure 1: Ownership of electronic devices, with comparison between cohorts<br />
Over 80% of students from both years owned an MP3 player, digital camera and some form of<br />
external memory (e.g. usb sticks or external hard drive). This finding matches that of the LXP2 report<br />
which states "There is agreement that the majority of learners arrive at university confident, positive<br />
and enthusiastic about their use of technology (LeAD, 2009); (STROLL, 2009); (LexDis, 2009). The<br />
projects that conducted surveys show that most learners arrive with their own personal technology,<br />
notably laptops and mobile phones." (Sharpe, 2009)<br />
402
Jake Leith et al.<br />
Figure 2 below shows the interesting diversity in the popularity of internet-based services: for example<br />
social networking sites (e.g. Facebook) were used regularly by the vast majority of students, whilst<br />
Twitter was used regularly by only 3 respondents. This demonstrates that we should not take for<br />
granted students’ use of social media across the board.<br />
Figure 2: Levels of usage of internet based services across both cohorts (Level 4 + Level 5)<br />
The popularity of social networking sites might explain the responses to a later question about<br />
organising group assignment activities, where communication by Facebook was more popular than<br />
the group discussion areas set up in the VLE, particularly amongst the second year students where<br />
no respondents used these.<br />
6.2 Levels of confidence in the use of technology<br />
Students in the sample expressed varying levels of confidence in their use of technology, but fewer<br />
than 10% across both year groups considered themselves unconfident as shown in Figure 3:<br />
Figure 3: Levels of confidence in the use of technology<br />
Most students saw themselves as technologically literate. However, some criticism of the way the VLE<br />
worked (see comments below) is likely to come from their own experience with more “user friendly”<br />
social networking sites. We therefore need to recognise the need to educate students on the use of<br />
403
Jake Leith et al.<br />
their informal skills in the formal environment of a VLE, whilst trying, within the limitations of the<br />
<strong>learning</strong> platform provided, to emulate some of the experiences students prefer. Again these findings<br />
are backed up by the LXP2 Final Synthesis Report that found "most learners, despite access to<br />
technology and self-reported confidence, aren’t appropriating the technology they know into their<br />
study.” (Sharpe, 2009)<br />
6.3 Response to the blended <strong>learning</strong> approach<br />
Email and announcements via the VLE were considered the most helpful form of communication<br />
between staff and students (83% considered email useful or very useful, 75% considered<br />
announcements useful or very useful); towards the end of the year the department introduced a text<br />
messaging service which was not included in the questionnaire. Anecdotally this appears to be very<br />
popular and follow up interviews will be designed to cover this. Figure 4 shows the responses of both<br />
cohorts combined:<br />
Figure 4: Forms of online communication considered the most useful across both cohorts<br />
There was no conclusive evidence on the perceived benefits of online lectures, with both groups<br />
divided approximately equally between finding them useful/not useful. A significant number (28%) of<br />
respondents answered “don’t know/unsure” when asked if videos had been useful or helpful – in this<br />
regard a comment by one respondent may be significant here: “I think some people had trouble<br />
accessing podcasts and videos. My internet access at home sometimes cannot support these<br />
functions”.<br />
Comments on the online materials in general indicated that it was the accessibility of materials that<br />
students found most helpful: “they were always there and accessible”; “could access it at a computer<br />
at any time”; “could individually work at own pace/when convenient yet keep group updated without<br />
sending large files via email”.<br />
Comments on the content of materials (both online and face-to-face) indicate that, as might be<br />
expected, there is a wide range of abilities and previous experience in the cohorts examined with<br />
regards to business and professional practice, and consequently students’ expectations varied widely<br />
(“the topic was approached without first covering the basics so lots of people felt quite lost and<br />
bewildered”; “I felt some of the “introduction” lectures were incredibly basic..”). Online <strong>learning</strong><br />
activities will be particularly useful in providing more differentiation in future years.<br />
Difficulties highlighted included a lack of consistency in terms of material location within the VLE:<br />
“sometimes – and this is across all courses – where things are located can be confusing…”; student<br />
central [the VLE] is difficult and illogical”.<br />
404
Jake Leith et al.<br />
One comment in particular highlighted the significance of problems associated with lack of internet<br />
access at crucial points in the course: “for the entire first term we had problems installing the internet<br />
at home, this made connecting with information that was only available on the internet an obstacle in<br />
accessing information. It shouldn’t be assumed that everyone has full internet access”.<br />
Students still valued the face-to-face elements of the teaching – in particular talks from visiting<br />
lecturers (“I enjoyed the Tom Savigar lecture the most, it was very interesting”; “BJ Cunningham<br />
lecture was great EVERYONE was talking about it”), and face-to-face tutorials. We will be looking at<br />
the possibility next year of including live web conferencing that should simulate many of the benefits<br />
of face to face talks from external practitioners whilst reducing the number of hours required of their<br />
time.<br />
Opinion was divided on the usefulness of online peer assessment (40% found it useful or very useful).<br />
7. Staff team experiences on the impact of the design and implementation of<br />
blended <strong>learning</strong><br />
7.1 Designing the new course content<br />
The short timeframe in which the new course content had to be implemented meant that in many<br />
cases the material was being produced “as we went along”.<br />
The weekly audio podcasts provided by the Course Leader during this academic year were intended<br />
to aid students with general feedback in an informal, ‘human’ manner, more akin to the aural qualities<br />
associated with face to face delivery. These included ‘tip of the week’, which became a weekly feature<br />
of the podcast. The Course Leader, new to the production of podcasts, found it helpful to make bullet<br />
points and prepare written elements of text before making the audio recording. An added advantage<br />
on one or two occasions was the ability to record podcasts in advance and then use the VLE’s ‘timed<br />
release’ function to publish it for students at a predetermined future date.<br />
There was a steep <strong>learning</strong> curve for staff and HPL’s involved in the production of captured lectures.<br />
Again short notice and lack of time available resulted in perhaps not enough “redesign” of content. We<br />
also recognise the need to address intellectual property issues surrounding the capture of lectures by<br />
HPL’s.<br />
7.2 Delivery<br />
As noted above, email was a prime method of communication for the majority of students.<br />
Unfortunately due to circumstances beyond our control, some first year students were not able to<br />
access their university email for several weeks at the start of the first term. This caused considerable<br />
problems, but validated the need for a blended approach to <strong>learning</strong>, as the face-to-face introductory<br />
and project briefing sessions at the start of the term became essential in allowing us to recognise and<br />
deal with the problem.<br />
The online format of the “week by week” programme noted above was supplemented by a hard copy<br />
timetable and project briefing document. These were also stored on the VLE in downloadable form.<br />
We realised early on that the documents (in particular the timetable) needed to be more visually<br />
consistent with the online format, as this could have assisted the students in navigating the VLE.<br />
7.3 Assessment<br />
Online formative peer assessment (using Turnitin software) was conducted to help focus the students’<br />
thinking and provide an opportunity to critically reflect on each other’s projects. This aspect of the<br />
assessment process was complemented with a face to face tutorial session. Students appreciated<br />
being able to see what other groups were doing and assess their progress against this.<br />
Several additional points of assessment were introduced by breaking the projects down into “chunks”.<br />
This was helpful not only to give students structure to their project but also in providing staff with<br />
clearly defined stages to pick up on progress with students. It was also recognised during the delivery<br />
of the project that these stages needed to be more clearly integrated with the delivery of online<br />
materials to facilitate the students’ understanding of these.<br />
405
Jake Leith et al.<br />
For the summative assessment staff were obliged to use the School’s prescribed paper based<br />
summative assessment feedback forms, it was therefore not possible to provide feedback<br />
electronically, via the Turnitin software, and so staff did not benefit from the anticipated efficiencies in<br />
administering this process. This will be addressed in the next academic year so that online marking<br />
can be incorporated.<br />
7.4 Technical support<br />
Course design and implementation was conducted on an ongoing basis throughout the year of live<br />
testing. Neither of the main staff involved have a technical IT background, so it was vital to have a<br />
flexible and responsive point of contact for IT support within the University. One of the University’s<br />
Learning Technologies Advisors supported the unit’s development and was available throughout the<br />
course’s delivery. This proved invaluable in enabling staff to undertake the work involved.<br />
8. Review of work carried out<br />
The design and live testing carried out so far has highlighted a number of recommendations which are<br />
set out below, and will be supplemented with further information gathered from student focus groups<br />
in September 2011:<br />
Elements of the course already identified as best suiting an online approach are those lectures<br />
with more “technical” content – online delivery allows learners to follow these at their own pace,<br />
and also allows differentiation of delivery to take place via online exercises;<br />
Elements of the course that best suit a face-to-face approach fall in to two categories: the initial<br />
introductory and briefing sessions (these proved invaluable in identifying online problems early<br />
and also “humanise” the blended <strong>learning</strong> environment from the start) and secondly the lectures<br />
from visiting practitioners.<br />
Key resources: The infrastructure required for online delivery was already in place. Hardware<br />
requirements for staff were minimal (e.g. large flat screen monitors for online review and<br />
assessment, MP3 recorders for podcasts and P.C.’s with up-to-date web browsers).There has<br />
been a reduction in the need for physical space for lectures – this is significant as the cohort size<br />
requires the use of a large capacity lecture theatre. Staff have also been able to use their time<br />
more flexibly – for instance podcasts and lectures being recorded in advance, and assessment<br />
conducted without the need to access and share hard copy documents.<br />
Comparison of cohorts: to date as already noted there has been little discernable difference in the<br />
responses obtained from the two year groups. We will follow this up in our focus group work.<br />
9. Conclusions<br />
From a student perspective the online element has not been detrimental to the subject area.<br />
The issues and concerns raised by students, are that they feel that they are computer literate but, do<br />
not have the skills to translate this into formal <strong>learning</strong>.<br />
Whilst they feel confident with their usage of computers and internet services in specific areas<br />
(Facebook is used externally by many), students cannot translate the skills they have into the formal<br />
<strong>learning</strong> arena. However their experience of these services make them a critical audience when<br />
considering the design and quality of the online <strong>learning</strong> delivered through the University’s <strong>learning</strong><br />
environment.<br />
It is important that students are provided with clear technical standards and understand the<br />
technologies that will be used to deliver the online content and that time is built into the curriculum<br />
allow time to try these out, to provide them the opportunity to resolve any technical glitches.<br />
Whilst students’ response to the delivery online is broadly neutral, they are quick to recognise the<br />
benefits associated with this form of delivery and the flexibility that this affords them.<br />
Staff need to consider carefully the layout and design of the online space within the virtual <strong>learning</strong><br />
environment to ensure that this provides a logical and consistent experience to the students.<br />
406
Jake Leith et al.<br />
The online developments undertaken by Professional Practice has generated interest from the studio<br />
based provision who have now begun to think how aspects of online <strong>learning</strong> might be used to deliver<br />
instructional elements in the course, thereby freeing staff to devote greater time to working with<br />
students on the creative process.<br />
One example of this is in Fashion Design, through the delivery of digital pattern cutting. The traditional<br />
face-to-face teaching for this unit will be supported by the delivery of online instructional tutorials,<br />
demonstrating how to use the pattern cutting software. Another project recently launched within the<br />
Faculty of Arts allows students to build e-portfolios through their own usage of an e-portfolio and<br />
social networking web application (Mahara). One of the aims of this project is that the portfolios will<br />
also feed into a central database from which material can be collected for alumni profiling.<br />
10. Recommendations<br />
The layout of the online content within the VLE will be kept as streamlined and user friendly in<br />
order to emulate where possible the social networking spaces students are familiar with.<br />
Supporting hard copy documents will be designed to be visually consistent with the online<br />
presentation of the programme delivered.<br />
However, we also need to manage expectations from the start, and we are examining the<br />
possible adoption of the checklist for new students developed from the LEaD project.<br />
Face-to-face contact at the start of the academic year is considered vital in orientating new<br />
students and picking up problems early on.<br />
Contracted visiting lecturers for the next academic year should be asked to provide an<br />
introductory podcast to their lecture / workshop. This should aid students in their preparation for<br />
sessions and provide a good ‘taster’ to the forthcoming lecture, ‘flagging’ its relevance to the<br />
programme of study.<br />
Online exercises should be developed to improve differentiation of teaching in the more technical<br />
topics taught.<br />
Summative assessment should be carried out online within the Turnitin software in order to<br />
maximize the time saved by staff on assessment administration.<br />
The introduction of live webinars with HPL’s and other visiting speakers should be investigated.<br />
Acknowledgements<br />
We would like to thank Adam Bailey University of Brighton Learning Technologies Advisor for his<br />
support during the first year of this new course delivery and his contributions to this research paper.<br />
References<br />
Biggs, J. B. (1999) Teaching for Quality Learning in University, Society for Research in Higher Education and<br />
Open University Press, Buckingham.<br />
The Open University, (2005) Action Research, A Guide for Associate Lecturers. [online]<br />
http://www.open.ac.uk/cobe/docs/AR-Guide-final.pdf<br />
Mayes, T and De Freitas, S. (2004) JISC Design for <strong>learning</strong> key outcomes, Work package 1b, eLearning Models<br />
desk study, Review of eLearning theories, frameworks and models. [online]<br />
http://www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20(Version%201).pdf<br />
Littlejohn, A. and Pegler, C. (2007) Preparing for Blended eLearning, Routledge, London.<br />
Graham, C. (2005) The Handbook of Blended Learning. [online], Wiley<br />
http://media.wiley.com/product_data/excerpt/86/07879775/0787977586.pdf<br />
407
Sophisticated Usability Evaluation of Digital Libraries<br />
Stephanie Linek and Klaus Tochtermann<br />
ZBW – Leibniz Information Centre for Economics, Kiel, Germany<br />
s.linek@zbw.eu<br />
K.Tochtermann@zbw.eu<br />
Abstract: Digital libraries are an important headstone for distant education. Thereby, modern digital libraries<br />
encounter not only the classical provision of documents, but also other related services like publishing portals or<br />
lists of hot topics. Another upcoming challenge is the connection with Web 2.0. Thus, the functionalities of online<br />
libraries will change and becoming more and more complex. Accordingly, the usability evaluation of these<br />
complex functionalities has to be assured and adapted to the new challenges. The proposed methodology of<br />
sophisticated usability evaluation follows a spiral model for prospective recommendations. The usability<br />
evaluation is done by an iterative process. This is in line with the state of the art and most modern models in<br />
usability engineering. However, in contrast to the existing models, we propose an explicit combination between<br />
systematic quantitative investigations and focused usability studies. The proposed multi-method approach<br />
addresses different important elements of a sophisticated scientific usability evaluation. The core elements of the<br />
cyclic process are repeated usability benchmarking, focused usability studies, derivation of recommendations<br />
and decision on the planned improvements in the face of the overall strategy. The repeated benchmarking allows<br />
a quantitative measurement of advantages and drawbacks as well as improvements and impairments. The<br />
quantitative benchmarking data can also help assigning the gravidity of usability-problems or the importance of a<br />
specific innovation. Depending on the requirements of the users and the developments of the digital library, the<br />
specific usability-studies could be qualitative or quantitative or a mixture of both. The overall aim of these studies<br />
is the formulation of concrete practical recommendations. The recommendations have to be aligned with the<br />
technical possibilities and strategic decisions. After the improvements and innovations were implemented, a new<br />
benchmarking cycle can take place. The four elements can be flexible combined. This procedure ensures not<br />
only a holistic and sophisticated usability evaluation, but also the openness for new challenges and opportunities<br />
of the 21st century. The described approach will be exemplarily explained by the ZBW – Leibniz Information<br />
Centre for Economics.<br />
Keywords: iterative evaluation, usability, digital libraries, multi-method approach<br />
1. Introduction: Digital libraries and the importance of usability evaluation<br />
In modern times, it’s quite usual to regard to electronic literature instead of written documents.<br />
Information (including specialist literature) is more and more searched among the internet and social<br />
platforms are often used for knowledge sharing.<br />
Even though a lot of libraries are online, many of the digital libraries appear a little bit old fashioned. In<br />
most cases, there is only a lose connection with social platforms like Facebook or Xing. Also new<br />
forms of <strong>learning</strong> and knowledge sharing (e.g., by means of serious games) are not considered. Even<br />
worse, the usability of the web-based digital libraries and its services is often problematic and usability<br />
evaluation is still an exception.<br />
There are many slightly different definitions of usability. Besides accessibility, most of the definitions<br />
are based on four elements: Effectiveness, efficiency, usefulness (appropriateness for subjective aims<br />
of the user) and joy of use (see for example Rubin and Chisnell, 2008). The usability of Web sites in<br />
general and digital libraries in specific is of rising importance. Nowadays the end-users expect webservices,<br />
software and online platforms not only to be useful for the original purpose, but also to be<br />
self-explaining, and quick and easy to handle. In the best case, the use of software is not only helpful<br />
but also includes “joy of use”.<br />
Thus, usability evaluation is of rising importance in the modern media world. For digital libraries an<br />
adequate usability evaluation is the fundament for being capable of competing with simple search<br />
instruments (like Google) that deliver quick and easy results. While classical digital libraries offer a<br />
complete and high class literature list, many end-users prefer the “quick and dirty” search by Google,<br />
Alta Vista or similar services – just because of the better usability.<br />
If digital libraries take their role as educational service seriously, they have to assure a good (or at<br />
least sufficient) level of usability as prerequisite to reach their target end-users.<br />
408
Stephanie Linek and Klaus Tochtermann<br />
2. Aims of usability evaluation of digital libraries<br />
Usability evaluation should not only aim at the usability improvement of an existing online platform,<br />
but also has to account for upcoming innovations and improvements. Additionally, usability studies<br />
can also accompany the establishment of a completely new online platform or digital library (i.e., if a<br />
normal library wants to go online.)<br />
Overall, the usability evaluation of web-based digital libraries and their services can address multiple<br />
goals:<br />
Identification of specific and general usability problems of existing digital libraries<br />
Addressing specific and isolated usability questions<br />
Enabling a comparison with general usability standards and comparison with similar Web sites or<br />
online-services<br />
Quality check of innovations and improvements: Usability validation and measurement of<br />
improvements and enhancements<br />
Possibility to formulate concrete recommendations for usability improvements<br />
Enabling and fostering the development of an online-platform of a library, that accounts for the<br />
basic requirements of usability<br />
Enabling and fostering the development of new online services of an existing digital library in the<br />
face of basic requirements of good usability<br />
Flexible methodological procedures of usability evaluations that could be applied in the case of<br />
new forms of online services, e.g., Web 2.0 applications and game-based <strong>learning</strong><br />
The listed goals are not unique for digital libraries. Especially, the assurance of good usability and the<br />
openness for innovations are main aims of every modern Web site and online services that have to<br />
consider new technological possibilities and the growing requirements of end-users (Dumas and<br />
Redish 1999). In this sense, the proposed methodology could also by applied for other online-services<br />
and Web sites.<br />
It is important to note, that not all of the listed goals are of equal importance. Depending of the actual<br />
status of the Web site and online-services as well as the planned developments, different goals can<br />
be in the centre of usability evaluation. For existing Web sites, that want to incorporate innovative new<br />
applications the most important goal might be to have a flexible methodology that is apt for the<br />
usability evaluation of the new applications. In contrast, for Web sites that are still in their early<br />
development phases, a basic fundament of general usability standards could be the most important<br />
goal.<br />
Depending of the main goal of usability evaluation one has to choose the appropriate method. For a<br />
comparison with general usability standards a quantitative standardized measurement instrument is<br />
necessary. Contrariwise, for addressing prospective usability questions of planned innovations,<br />
qualitative interview data might be more fruitful.<br />
Nowadays there are many various methods available for usability evaluation, ranging from qualitative<br />
methods with single expert-ratings to quantitative methods with a large group of end-users. Each of<br />
these methods has its advantages and drawbacks. It has to be carefully selected which method is the<br />
best to address a specific usability problem in a specific context. Furthermore, from a practical point of<br />
view, it is also a question of resources. Even though, usability testing with end-users is often seen as<br />
the “silver bullet” for usability evaluation, sometimes it is too costly and time intensive. In literature,<br />
there is an ongoing debate which of the available methods is most efficient and effective for<br />
identifying usability problems (overview is given by Rubin and Chisnell 2008; Dumas and Reddish<br />
1999). However, this paper is not about the comparison of different methods of usability evaluation<br />
but rather propagates a multi-method approach that is oriented on valid identification and elimination<br />
of usability problems in the face of practical possibilities.<br />
In most cases, usability evaluation has to address more than one goal and thus, more than one single<br />
(assessment) method is necessary. Furthermore, the development of a Web site as well as the<br />
accompanying usability evaluation is a repetitive process. On the one hand, usability<br />
409
Stephanie Linek and Klaus Tochtermann<br />
recommendations are needed for further developments of the Web site. On the other hand, the<br />
ongoing development of a Web site results in new usability questions.<br />
3. Overview of the proposed multi-method approach<br />
The proposed methodological approach accounts for the interdependence between technical<br />
developments and usability evaluation. Thereby, the need for multiple methods is explicitly<br />
addressed. Overall, usability evaluation is composed of iterative cycles with the following elements:<br />
Repeated benchmarking of the overall usability by means of standardized questionnaires: These<br />
data deliver quantitative indicators of improvements based on a larger panel of users<br />
Specific usability-studies on selected concrete usability questions based on a small sample of<br />
users: These studies work with qualitative data, but in case might also require quantitative<br />
indicators<br />
Recommendations based on the results of usability-studies and the quantitative benchmarking<br />
Improvements in the face of the usability-recommendations and strategic decisions<br />
The iterative procedure is an important characteristic of the described approach. Usability evaluation<br />
is seen as an ongoing process that has to account for new technical possibilities as well as for<br />
changing requirements and expectations of the end users. This this is in line with the state of the art<br />
and most modern models in usability engineering (Rubin and Chisnell 2008; Dumas and Reddish<br />
1999). However, most existing models propagate iterative testing without a specification of the<br />
measurements or how different methods could be combined in the iterative cycles.<br />
Contrariwise, our proposed multi-method approach combines general benchmarking and specific<br />
usability studies in the face of the needed requirements and strategic decisions. In this context, it is<br />
important to note, that the whole process of usability engineering is conceptualized as an iterative<br />
cycle, including repeated benchmarking and specific usability test as well as the formulation of<br />
usability recommendations and strategic decisions.<br />
The listed elements above could be arranged and combined in a flexible way. Thereby, quantitative<br />
and qualitative methods will be combined to enable concrete usability recommendations that can be<br />
adapted to rapid changing technology and growing user requirements.<br />
4. Description of the single elements<br />
4.1 Quantitative benchmarking with standardized questionnaires<br />
Quantitative benchmarking with standardized questionnaires is necessary for a systematic ongoing<br />
usability evaluation and development of a digital library (as well as for any other long lasting<br />
professional Web service). Even though usability studies with pure qualitative data might be helpful to<br />
identify the most urgent usability problems, quantitative data with standardized measurement<br />
instruments are necessary to receive the required input for ongoing developments and a quality check<br />
of a digital library.<br />
Quantitative usability data have the following main advantages (see also Langdridge and Haggerjohnson<br />
2009):<br />
Possibility of a priority ranking of usability problems<br />
Quantification of improvements and enhancements (as quality check)<br />
Possibility of a comparison with general standards of usability<br />
Possibility of a comparison with competitors and alternative digital libraries (by the use of<br />
standardized existing questionnaires)<br />
Especially the priority ranking of usability problems is of high practical relevance, because normally<br />
there are not enough resources for addressing all usability issues from the developer’s side.<br />
Additionally, there might be a conflict between different aspects of usability. A classical example is the<br />
balance between simplicity of handling and the appealing design of a Web page. While qualitative<br />
data identify both problems, the quantitative measurement enables a decision for the developers: to<br />
make it either more simple (even though design will be boring) or to make the design more<br />
410
Stephanie Linek and Klaus Tochtermann<br />
sophisticated (even though it will be more complicated). This illustrates also the necessity to select<br />
quantitative instruments that are apt for the specific case.<br />
In principle, many different quantitative measurements are thinkable for usability benchmarking.<br />
Objective behavioral video data or logfile protocols as well as subjective interviews or questionnaire<br />
data can be analyzed in a quantitative way. However, the analysis of video data and logfile protocols<br />
is very resource-intensive and thus, often not feasible for a big sample or a broad variety of variables.<br />
Contrariwise, a questionnaire is timesaving and a less resource-intensive instrument. It can be easily<br />
distributed among a big sample of users; in case also online-surveys are possible.<br />
The use of a standardized existing questionnaire allows a valid and reliable comparison with other<br />
Web sites or services and thus, enables a comparison with competitors or alternative digital libraries<br />
as well as a comparison with general standards of usability. For the assessment of general usability<br />
several standardized questionnaires are available: short ones (e.g., System Usability Scale - SUS by<br />
Bangor, Kortum and Miller 2009) as well as long ones (e.g., ISONORM by Prümper 1999 or<br />
IsoMetrics by Gediga, Hamborg and Düntsch 1999).<br />
However, sometimes the use of standardized questionnaires might not be sufficient since they are<br />
very general and not apt for a benchmarking of special services, e.g., the quality of the specialized<br />
content. In such cases, an additional scale has to be developed for the evaluation of specific<br />
important features, for example for the content quality of the results of a literature search.<br />
4.2 Focused usability studies<br />
Focused usability studies target at specific usability questions. These questions could regard to the<br />
usability of new features, to the improvement of identified usability problems, to planned innovations,<br />
to the redesign of the Web site and to many more. In this sense usability studies are aligned with the<br />
requirements of end-users, the specific questions of developers and planned innovations of the<br />
management or the general policy of the library. Accordingly, the selection of the concrete method<br />
has to be oriented on the status quo, practicability and urgency of usability issues. Thereby, many<br />
different methods are possible including quantitative as well as qualitative data. For an existing page,<br />
usability test with end users can be seen as the “silver bullet”. Contrariwise, also heuristic evaluations<br />
can be an alternative. For the establishment of a new Web site, rapid prototyping with paper-based<br />
mock-ups could be a good way (Snyder 2003). With respect to planned innovations one might prefer<br />
a focus group or semi-structured interviews. (Like mentioned above, it would go beyond the scope of<br />
this article to explain and compare the different methods.)<br />
Each method has its advantages and drawbacks. The crucial thing is to select the most appropriate<br />
method with respect to the concrete situation, usability issue, and (last but not least) the available<br />
resources and strategic decisions. Thereby it is essential to conduct the usability study in a way that<br />
enables concrete recommendations. The “questions shapes the answers” (title of the review by<br />
Schwarz 1999). In the case of usability recommendations this implies to ask in a way that produces<br />
improvements and alternatives – and not only critique and the identification of problems.<br />
4.3 Derived recommendations based on usability studies<br />
Recommendations address not only the elimination of problems or the improvement of the usability in<br />
general, but also target at the accentuation of the favorite applications and the best aspects of the<br />
digital library.<br />
Recommendations for usability improvements have to be concrete and practicable. In principle, the<br />
recommendations have to be based on the results of the focused usability studies, but also the data<br />
of the benchmarking can deliver important input (depending on the chosen questionnaires and<br />
additional questions.) Benchmarking data can be used for screening, i.e. identification of overseen or<br />
hidden usability issues. Additionally, the comparison with competitors could be helpful for finding<br />
gaps, and obstacles as well as to identify the main incentive or appeal of the own Web site.<br />
Furthermore, also a literature research on existing usability guidelines could be a starting point,<br />
especially for the establishment of a complete new digital library or a redesign of a Web site. For more<br />
specific, concrete recommendations one has to carefully interpret the results of the own usability<br />
studies. The same is true for recommendations for planned innovations.<br />
411
Stephanie Linek and Klaus Tochtermann<br />
4.4 Making improvements: Balancing usability-recommendations and strategic<br />
decisions<br />
In a perfect world, usability recommendations could be fully implemented and we will have happy endusers<br />
and a proud development-team. However, in reality improvements can only be made in the face<br />
of available resources and strategic decisions regarding the overall policy of the (digital) library. If<br />
resources are spare, one has to address the most urgent usability issues first. However, if there is the<br />
strategic vision to become the most innovative digital library, this might lead to the decision to address<br />
first required innovations and second the elimination of usability problems. Additionally, researchbased<br />
recommendations could also contradict the policy of a digital library. For example, end-users<br />
might have the desire for a broader assortment of and easier access to the genre fantasy or horror<br />
while the management of the library wants to establish the image of an expert for sociocritical<br />
literature. It is a difficult decision, how to handle such contradictories and how to find a compromise.<br />
But think of Alice’s Adentures in Wonderland (Charles Dodgson alias Lewiss Caroll, 1865): It is<br />
sociocritical as well as fantasy literature.<br />
5. Cyclic interplay of the four elements: Adaptive iterative process<br />
The interplay of the four elements described above can be characterized as follows:<br />
Multi-method approach: Combination of quantitative and qualitative methods<br />
Iterative cycles of usability evaluation and improvements<br />
Flexibility: Flexible combination of the single elements and flexibility of the single elements<br />
High interdependency between the elements<br />
5.1 Multi-method approach<br />
The combination of quantitative and qualitative methods addresses the divers demands at different<br />
stages of usability evaluation (see also Langdridge and Hagger-johnson 2009). Qualitative methods<br />
are often a timesaving and cost-effective way to identify a broad range of the existing usability<br />
problems and user requirements. However, qualitative methods do not allow a comparison with other<br />
digital libraries or search instruments. Analogously, it is not possible to quantify the improvements<br />
made in comparison with prior version of the digital library and its services. Thereby, quantitative<br />
measurements provide the necessary information. Quantitative measurements allow prioritizing<br />
usability problems as well as user requirements. Additionally, the use of quantitative standardized<br />
questionnaires enables the comparison with other software and online platforms (including other<br />
digital libraries). An important advantage of the quantitative benchmarking is the possibility to make<br />
improvements and enhancements measureable. Just the fact that a problematic feature or application<br />
was modified doesn’t mean that this modification was actually an improvement. Indeed, end-users<br />
don’t like changes and don’t want to deviate from their ordinary handling of a Web site. Comments<br />
like “The old version was much easier” or “The old version was not perfect, but I really don’t like the<br />
new one” are not an exception. Thus, changes of an existing Web site and its services have not only<br />
to be communicated, but it has also to be evaluated if the changes are actual an improvement for the<br />
end-users.<br />
The combined interpretation of quantitative and qualitative data enables the provision of detailed<br />
concrete usability recommendations. Thereby, it is also possible to make a priority ranking between<br />
the recommendations. Based on the different methods it might be possible to make alternative<br />
suggestions how to handle a specific usability problem.<br />
5.2 Iterative cycles of usability evaluations and improvements<br />
The quantification of usability improvements is also a good example for the iterative character of<br />
usability evaluation and usability improvements. It is very hard to create good usability of a web-page<br />
or online service, especially for the more complex ones. Moreover, it is impossible to create an<br />
everlasting high usability. In line with the technological progress and the developments of the Web,<br />
e.g., the ongoing development of Web 2.0 technologies, also the requirements of the end-users will<br />
change. This in turn implies that existing Web sites and online-services have to change, too.<br />
Otherwise they will be old fashioned – or in other words they will lack usability since they are no<br />
longer efficient, effective, useful, and provide no joy of use. Thus, iterative usability measurements are<br />
necessary. On the one hand, specific usability tests have to be repeated in the face of innovations<br />
412
Stephanie Linek and Klaus Tochtermann<br />
and new technical possibilities. Thereby qualitative data can be used to identify new possibilities or to<br />
identify old-fashioned applications. On the other hand, quantitative measurements and repeated<br />
benchmarking can be used as quality check if the Web site keeps up to date.<br />
5.3 Flexibility<br />
The iterative process is a flexible combination and arrangement of single elements. The iterative cycle<br />
can start at different points and the arrangement of the elements has to be adapted to the concrete<br />
context.<br />
Also the single elements are flexible. Quantitative benchmarking can be made by different<br />
questionnaires and the qualitative studies can follow different approaches that should be selected with<br />
respect to the concrete practical aims and the available resources.<br />
For an existing Web site it makes sense to start with quantitative benchmarking, to assess the status<br />
quo as a baseline and thus enable a comparison of the different versions after substantial changes<br />
have been made. Such a comparison ensures that the changes are actually an improvement for the<br />
end-users and makes improvements quantifiable.<br />
However, for the construction of a completely new Web site one should start with small focused<br />
usability studies. Methods like rapid prototyping are advantageously to avoid wasting resources of the<br />
developer-team. As a prerequisite, the group of end-users has to be known.<br />
In some cases also usability-recommendations could be a starting point. Until now many usability<br />
guidelines are available and could deliver fruitful input for the construction of every Web site. Even<br />
though most of the guidelines are too general to allow specific design recommendations some of<br />
them could be very helpful when open design questions come upon, e.g. how to arrange the<br />
navigation menu. A very good and valuable source of such general guidelines (based on research<br />
findings) is available at http://www.usability.gov/guidelines/guidelines_book.pdf .<br />
Besides these rather practical, user-oriented starting points in some cases also a strategic decision<br />
could be the first step. For example, a public institution wants to invest in Web 2.0 tools or refuses the<br />
usual conventions of navigation menu to make their innovative and vanguard character visible. Such<br />
strategic decisions are a challenge for the usability researchers and require a very flexible use of<br />
usability methods.<br />
5.4 High interdependency between elements<br />
The elements of the approach are highly interconnected. Independent of the concrete starting point or<br />
the arrangement of elements, none of the elements can be managed isolated from the others.<br />
Quantitative benchmarking as well as focused usability studies enable concrete usability<br />
recommendations. Thereby, the measurement methods have to be carefully selected with respect to<br />
the concrete practical demands of end-users and the capacities of development. It doesn’t make<br />
sense to ask the end-users for innovations that could never be implemented. Analogously,<br />
recommendations have not only to be concrete but also practicable and in line with strategic<br />
decisions.<br />
6. Usability evaluation of the digital library ZBW – Leibniz Information Centre<br />
for Economics<br />
In this section the described methodological approach will be explained by the concrete example of a<br />
digital library, namely the ZBW – Leibniz Information Centre for Economics (http://www.zbw.eu/indexe.html).<br />
6.1 The digital library ZBW and its online services<br />
The ZBW is the world’s largest specialist library for economics, with locations in Kiel and Hamburg.<br />
The ZBW provides numerous services like EconBiz for literature search, EconStor as a publishing<br />
portal, EconDesk as a reference service and many more. The services are frequently used and very<br />
welcomed by the end-users. For example, for EconBiz there are about 15 000 queries per month with<br />
20 000 to 28 000 visitors per months. Also the GooglePageRank of 7 demonstrates the high visibility<br />
and degree of popularity of this service.<br />
413
Stephanie Linek and Klaus Tochtermann<br />
Until now, the usability of the online services and the Web site itself has not been evaluated in a<br />
systematic way. Furthermore, the services will be enhanced and improved in the future. Therefore, a<br />
new internal task force for usability evaluation was established at the ZBW. The aim of this task force<br />
was not only to enhance the usability of the existing Web site and its online services, but also to<br />
regard for the future development and the planned innovations with respect to Web 2.0.<br />
6.2 Strategic decision as starting point<br />
The starting point for usability evaluation was a strategic decision: To become a modern digital library,<br />
that is capable to face the challenges of Web 2.0. Thereby, it was recognized, that good usability is<br />
the key to attract end-users that would be otherwise only use Google-search or similar search tools.<br />
6.3 Benchmarking questionnaire as foundation of the usability evaluation<br />
In a first step a benchmarking questionnaire was created, to assess the status quo of the already<br />
existing Web site and the services of the ZBW.<br />
The benchmarking questionnaire comprises standardized existing scales, an additional scale for<br />
assessing the quality of literature search results and several prospective questions. These three main<br />
parts of the benchmarking questionnaire regard to different purposes:<br />
Standardized questionnaires (ISONORM & SUS) serve two main aims. First, the comparability with<br />
other software products and Web sites, and second, the establishment of an internal standard, that<br />
makes improvement measureable with respect to general standards of usability.<br />
Additional scale for assessing the quality of literature search results of the online service EconBiz tries<br />
to establish a measurement instrument for an internal quality standard of the very heart of a digital<br />
libraries online service: The literature search. The scale was constructed in close collaboration with<br />
the product manager of EconBiz and librarians. The wording and format of the scale was analogous<br />
to the SUS. Even though the validity and reliability was not proven, the scale assesses quality criteria<br />
that were judged as important cornerstones of the service EconBiz. Thereby, the scale included<br />
general criteria (e.g., completeness of the literature list) as well as specific aspects that regard to<br />
enhancements and improvements (e.g., sophisticated ranking and filter options).<br />
The prospective additional questions were included to assess end-users’ requirements and to receive<br />
a more holistic view. Additionally, the questions tackle also planned innovations. Besides, the endusers<br />
are asked about their typical work with the ZBW services, especially literature search with<br />
EconBiz, to receive the required information for the creation of test scenario for subsequent usability<br />
test scenarios. (More details on the benchmarking questionnaire can be found in Linek and<br />
Tochtermann 2011.)<br />
6.4 Focused usability studies for the future development, innovations and practical<br />
needs<br />
The usability studies of the ZBW will focus on two main areas: First, on the planned innovations based<br />
on the strategic decision (modern digital library with connection to Web 2.0 applications), and second,<br />
on the main usability issues revealed by the baseline data. These two main areas have several<br />
intersections, especially with respect to the core service of the ZBW, i.e., EconBiz, the online service<br />
for literature search. Additionally, it is planned to redesign the ZBW homepage.<br />
The usability studies at the ZBW will mainly be done by means of videotaped usability tests. The<br />
created test scenario will be based on the typical tasks of different end-user groups. Besides usability<br />
test, the focused usability studies are designed in the face of the concrete requirements, e.g. rapid<br />
prototyping with paper-based mock-ups as information source for the redesign of the homepage and<br />
interviews with end-users as method for requirement engineering for the planned innovations.<br />
6.5 Research-based recommendations in the light of the strategic decision<br />
The recommendations for the improvements and further developments/innovations will be based on<br />
the benchmarking-questionnaire as well as the focused usability studies. For example, rapid<br />
prototyping and interviews with end-users will enable concrete recommendations for the structure of<br />
414
Stephanie Linek and Klaus Tochtermann<br />
the sites and the labels for navigation. The quantitative benchmarking data deliver the necessary<br />
base for a priority ranking of required improvements. Additionally, the open questions of the<br />
benchmarking questionnaires provide additional information on user requirements.<br />
After the redesign of the Web site including essential changes and innovations, the benchmarking<br />
questionnaire will be presented again to a big sample of end-users. This procedure enables a quality<br />
check, if the changes made are in fact an improvement. Thereby, the standardized usability-scales<br />
(ISONORM & SUS) as well as the scale on content quality remain constant. However, the open<br />
questions will be modified in the light of the future development and planning of the ZBW.<br />
7. Resume and outlook<br />
The proposed methodological approach offers a flexible way for a sophisticated usability evaluation<br />
that is on the one hand research-based and on the other hand oriented on practical needs. The<br />
combination of quantitative and qualitative data provides a more holistic view of the situation and<br />
allows the derivation of concrete usability recommendations. Additionally, quantitative benchmarking<br />
enables the comparison with older versions of the Web site as well as the comparison with general<br />
standards of usability and competitors. The flexible arrangement of elements allows a wide spread<br />
application of the methodological approach. Thereby it is important to consider the high<br />
interdependency between the elements and to adapt on the concrete situation.<br />
The described example of the digital library ZBW demonstrates the different possibilities of the<br />
described approach. Even though the preliminary evidence (of pilot tests) is promising, it is far too<br />
early to draw conclusions yet. The described example of the ZBW is only one application case and<br />
thus, the methodology has to prove its value for other instances. Thereby, the proposed multi-method<br />
approach and the described benchmarking questionnaire are open for adaptive variations,<br />
modifications and enrichments.<br />
References<br />
Bangor, A., Kortum, P. and Miller, J. (2009) “Determining what individual SUS score mean: Adding an adjective<br />
rating scale“, Journal of Usability Studies, 4(3), 114-123.<br />
Dodgson, C. alias Lewis Caroll, (1865) Alice’s Adventures in Wonderland, Macmillan: United Kingdom. Full text<br />
online available at http://www.cs.cmu.edu/~rgs/alice-table.html or at:<br />
http://ebooks.adelaide.edu.au/c/carroll/lewis/alice/<br />
Dumas, J. S. and Redish, J. C. (1999) A practical guide to usability testing, Intellect Books, Oregon, USA.<br />
Gediga, G., Hamborg, K. C. and Düntsch, I (1999) “The IsoMetrics usability inventory. An operationalisation of<br />
ISO 9241-10 supporting summative and formative evaluation of software systems”, Behaviour and<br />
Information Technology, 18, 151-164.<br />
Langdridge, D. and Hagger-johnson, G. (2009). Introduction to research methods and data analysis in<br />
psychology (2 nd ed.), Prentice Hall.<br />
Linek, S. B. and Tochtermann, K. (2011) “Assessment of usability benchmarks: combining standardized scales<br />
with specific questions”, Proceedings of the 14 th International Conference of Interactive Collaborative<br />
Learning (ICL 2011), Piešt’any/Slovakia, September 2011.<br />
Prümper, J. (1999) “Test IT: The ISONORM 9241/10”, in H. J. Bullinger and J. Ziegler (Eds.), Human-Computer<br />
Interaction – Communication, Cooperation, and Application Design (1028-1032), Erlbaum: Mahwah, New<br />
Jersey.<br />
Rubin, J. and Chisnell, D. (2008) Handbook of usability testing, Wiley Publishing Inc., Indianapolis.<br />
Schwarz, N. (1999) “How the questions shape the answers”, American Psychologist, 54 (2), 93-105.<br />
Snyder, C. (2003) Paper prototyping: The fast and easy way to define and refine user Interfaces, Morgan<br />
Kaufmann, San Francisco.<br />
415
Social Networks, eLearning and Internet Safety: Analysing<br />
the Stories of Students<br />
Birgy Lorenz, Kaido Kikkas and Mart Laanpere<br />
Institute of Informatics, Tallinn University, Estonia<br />
birgy.lorenz@gmail.com<br />
kaido.kikkas@tlu.ee<br />
mart.laanpere@tlu.ee<br />
Abstract: eLearning has recently become more social and open, involving open educational resources, open<br />
courses, open <strong>learning</strong> using personal <strong>learning</strong> environments or social networks. We believe the schools are not<br />
ready for this yet, as strategies and regulations supporting open <strong>learning</strong> are not up to date. It may seem easier<br />
to restrict the use of Twitter, Facebook etc rather than integrate them into the <strong>learning</strong> process. Our study which<br />
was based on the qualitative analysis of 201 stories written by secondary school students shows that a typical<br />
safety incident is not solved adequately when existing regulations are used by the schools. We carried out the<br />
qualitative analysis of 201 e-safety related short stories presented by students (aged 12 to 16), parents, teachers,<br />
school IT managers and police, collected through the Safer Internet in Estonia EE SIC campaign. 2/3 of the<br />
stories are fictional – they may be based on urban legends which however appear to refer to real stories. 1/3 of<br />
the stories reflect real incidents. We mapped typical behaviour patterns and templates, beliefs regarding privacy<br />
as well as regulations and limitations concerning the use of social networks at schools. We found that most of the<br />
solutions used by schools to ensure e-safety are either technical or purely regulation-based, only some schools<br />
appeared to have studied or elaborated on pedagogical or behavioural aspects. Problems are defined by<br />
limitations and regulations while actual safety incidents (whether in- or outside school) remain largely unsolved<br />
(or even undetected). Schools tend to excuse their unawareness with „we'll react when it happens“ (secretly<br />
believing that it will not happen). Most schools also try to delegate such problems to parents - who in turn look<br />
upon schools for help, as their only reaction to safety incidents is often just applying time limits on Internet use.<br />
Thus there is an urgent need for information and working guidance mechanisms for managers, teachers, parents<br />
and students. These matters must be solved well before schools reach the critical mass in using eLearning,<br />
social networks and modern gadgetry as parts of curriculum. Schools are expected to apply new technologies in<br />
teaching and <strong>learning</strong>, but safety of student and teachers is paramount in this context.<br />
Keywords: online safety, schools, policy, new technologies, social media<br />
1. Introduction<br />
The internet safety issues in the context of eLearning are quite often addressed in a simplified, blackand-white<br />
manner, using rare cases of criminal privacy violation to scare teachers and parents. We<br />
believe that fear is not the best teacher in this domain.<br />
Our goal was to:<br />
Evaluate what kind of e-safety stories are students and experts (teachers, it-managers, and<br />
police) telling and how they are related;<br />
Find typical e-safety stories to promote development of regulations in these areas;<br />
Previous studies in this matter can be divided into 6 bigger groups: cyber-bullying (Berson 2002);<br />
moral issues – pedophilia, inappropriate content, behavioural errors (Akdeniz 1997; Carr 2004;<br />
Mitchell 2004; Peters 2009); programs and materials for schools (Wishart 2004); threat analysis for<br />
eLearning (Alwy 2010); normal teenage internet usage (Bullen 2000; Enochsson 2005; Dworschak<br />
2010); Internet usage analysis (Livigstone, 2011; The Safer Internet for Children Qualitative Study in<br />
29 European Countries 2007; Towards a Safer Use of the Internet for Children in the EU – a Parents’<br />
Perspective 2008). While according to the EUKids Online II survey (Livingstone 2010) Estonian<br />
students are among the top users of Internet and have had substantial exposure to online security<br />
risks, there has been next to no attempt of national-level regulation in this field.<br />
We analysed the typology and sources of safety incidents, the real solutions offered, the students'<br />
thoughts and feelings stemming from the situation, the solutions suggested by students and whether<br />
these typical solutions actually apply in real-life situations. The practical experience of students,<br />
teachers, IT managers and police indicates a gap between the measures used in education and real<br />
life.<br />
416
2. Background<br />
Birgy Lorenz et al.<br />
The new National Curriculum for Primary and Secondary Schools prescribe the use of multitude of<br />
digital tools in the <strong>learning</strong> process: virtual <strong>learning</strong> environments (VLE), personal <strong>learning</strong><br />
environments (PLE) and other web 2.0 resources like social networking to be used in a classroom<br />
(National Curriculum 2010). A recent study shows that even with the good existing training<br />
opportunities and guided by the requirements of the new curriculum, most of the Estonian teachers<br />
are not ready to facilitate students in the matters of e-safety (Maadvere 2010).<br />
In the teachers' communities there are some discussions about the differences between the real life<br />
needs of a 21st century learner and how the schools should meet these needs. There are rising new<br />
challenges like digital divide, globalization as being a part of global internet village, digital sociality,<br />
familiarity in communicating; the changes in the roles of student and teacher and in understanding the<br />
policies and responsibilities (Veldre 2010, Murumaa 2010).<br />
There are a lot of materials (usually in English) that urge teachers to discuss these matters with<br />
students. But they are usually used as a reaction rather than a preventative act. At first nobody<br />
believes it could happen to them; yet if something happens, it's not announced and discussed as<br />
people feel ashamed (Hoiser 2009).<br />
Teachers are usually reluctant about e-safety - likely most schools have some video posted in the<br />
internet featuring a teacher who is unaware that he/she is a 'movie star' or some students secretly film<br />
other students (Vasli 2011). The discussions of incidents on e-safety tend to lead to internet usage<br />
restrictions at school. Schools and parents are interested in monitoring children’s behaviour in the<br />
name of preventing cyber-bullying or meeting with strangers (Hunter 2005).<br />
Studying school policies and practices, we could only find technical rules regarding computer<br />
wellbeing, time or operational policies (TDL arvutiklasside kasutamise reeglid 2004). Most schools<br />
regulate nothing at all, being afraid to create new precedents, or rely on the 'ostrich effect' claiming<br />
that having no incidents yet gives them no reason to act. In comparison, British schools opt for much<br />
more regulation (Children & Young People's Services 2011). In the United States, similar documents<br />
regulating teacher-student relations are given to sign by both parties (Ohio Policy reference manual<br />
2011).<br />
3. Methods<br />
We carried out two studies of e-safety stories and bottom-up policy development.<br />
The first study (Stage I) used the Grounded Theory (Chamaz 2006).<br />
The aim of Stage I was to decide who will be the focus group and what the main topics in the stories<br />
are. We used the 'go-around' exercise (Fundamental Team and Meeting Skills 2003) with empty cards<br />
and clustered the results with 50 participants (10 teachers, 6 experts, others were students aged 12-<br />
16). People were divided into 8 groups (6-7 people in the group); 7 groups with students and 1<br />
teacher, 1 group containing teachers and experts) and the goal was to:<br />
Decide who is involved in e-safety incidents and whom children look upon to get the information<br />
or help from;<br />
Gather and cluster data on what kinds of incidents can usually happen (the overall topics) and<br />
what would be important to the community.<br />
The results of Stage I were 13 different categories to code the e-safety stories into.<br />
Stage II used storytelling as a method. The story is a useful tool for <strong>learning</strong> - when we analyse<br />
stories, we can understand better how the world works for children (Vilke 2000).<br />
The stories were gathered from the people selected at Stage I: experts (police, teachers,<br />
psychologists, ICT experts) and students. The experts were selected from the Estonia EE SIC project<br />
using directed focusing (Teddlie 2009). Later we also used the snowball method (Gray 2007) as esafety<br />
educational experts are rare in Estonia. The stories from students were collected using a<br />
storytelling competition at InSafe project workshops in Estonia. We analysed the collected stories in<br />
line with qualitative case study method (Gagon 2010).<br />
417
Birgy Lorenz et al.<br />
135 students participated in the workshop titled “tell me your e-safety story”. We cannot prove<br />
them as authentic e-safety stories but we assume that they are not all myths. These are the<br />
stories that parents tell their children and children tell each other.<br />
19 stories that children claimed to be real.<br />
20 experts' stories were gathered from the interviews or surveys.<br />
27 stories provided by the police are a representative collection of what they think mostly happens<br />
to children that end up reported to the police.<br />
The experts presented their stories as situations with solutions.<br />
To code stories we used 13 themes (found in the results). They are more human behaviour related<br />
than typically found in the studies about technical e-safety risks in eLearning (Alwy 2010) or e-safety<br />
studies like EUKids Online II survey (Livingstone 2010), where the main goal is to address sexual<br />
themes. We were more interested in average students' typical internet/computer interaction<br />
experience. For that we used 180 stories out of 201, some stories were excluded due to being too<br />
short.<br />
4. Results<br />
We used the open coding approach to categorise the stories into 13 topics: spam, gaming, computer<br />
overuse, virus, fraud, passwords, fake accounts, cyber-bullying, harassment, slandering, privacy,<br />
pornography and meeting strangers. These were the topics or labels that emerged after reading the<br />
stories several times. Every story was then labelled with one topic (primary code), which appeared to<br />
be the main focus of this story. The Figure 1 illustrates the differences of stories collected from police,<br />
experts and children.<br />
Figure 1: The distribution of primary codes after the first round of coding<br />
As most of the stories actually contained references to multiple aspects and not just to the primary<br />
topic, we decided to do the second round of coding so that each story could be assigned multiple<br />
secondary codes. The Figure 2 illustrates the differences of occurrences of secondary codes in<br />
stories collected from police, experts and children.<br />
The analysis of the interviews suggests that neither teachers nor students do not really understand<br />
what e-safety is about (or there are too many different understandings what is or what is not e-safety)<br />
and who is responsible in this matter to help the child.<br />
Claims of the teachers:<br />
We have not faced any real e-safety incidents in past 3 years;<br />
Our school is e-safe because our students are not allowed to use WiFi or phones during the<br />
school time or they use Web filtering;<br />
418
The e-safety of children is a parents' concern;<br />
Birgy Lorenz et al.<br />
When something happens we will create new procedures.<br />
Figure 2: Frequency of secondary codes<br />
Claims of the students:<br />
We can access the Internet whenever we want, the filtering of the Web does not work;<br />
Cyber-bullying is an everyday act, it can start anywhere and it will not stop when teachers try to<br />
stop real-life bullying, sometimes they also punish wrong people.<br />
We also found evidence that the story usually starts with connection to one environment and later<br />
changes to another; for example, the first connection is established via web or e-mail, but afterwards<br />
it moves to a social network or direct messaging system. Most of the connections are made in social<br />
networks (Facebook, Rate, Orkut, gaming sites); 90% of the stories took place at home, but<br />
schoolmates are usually involved in some extent as well. So we can conclude that the stories are also<br />
discussed at school. Compared to the police stories, the children's stories do not involve mobile<br />
phones yet.<br />
The children's stories are mostly about gaming, fraud and passwords. There is a relationship between<br />
gaming and computer overuse and viruses, some stories can state relationships between gaming and<br />
harassment and pornography. Usually the games are single-player or standalone role-playing games<br />
but there was growing evidence about online gaming as well. The fraud category has direct<br />
relationships to passwords, spam, harassment, slandering and privacy. Fraud is usually seen in web<br />
or social networks, less in direct messaging. Passwords (scamming, phishing or hacking) are related<br />
to the web, less so to direct messaging or social networking. When we clustered harassment and<br />
cyber-bullying then it can relate to spam, fake accounts and slandering, while harassment is related to<br />
gaming, fraud, fake accounts and pornography.<br />
The real cases from children tell us typical stories about fraud, stolen passwords, harassment and<br />
slandering. Fraud was mentioned in relation to privacy and pornography. The stolen passwords were<br />
mostly related to cyber-bullying and slandering. Slandering and harassment were related to privacy<br />
infringement, and mentioning of pornography co-occurred with ones of gaming. There were no stories<br />
about spam, computer addiction and meeting strangers. There were lots of indications that direct<br />
messaging is taking place in social networks or chat rooms and strangers do not get added to one's<br />
MSN or Skype account – but this is what parents and teachers typically address in e-safety trainings.<br />
419
Birgy Lorenz et al.<br />
The experts' stories address mainly cyber-bullying, slandering, privacy infringement, somewhat less<br />
fraud and stolen passwords. There are direct relationships between privacy and slandering, bullying<br />
and passwords. Most stories are related to networking or using the web.<br />
The police stories show us what kind of help parents need from the legal system. They are usually<br />
related to fraud and pornography, less related to passwords, harassment and meeting strangers.<br />
There are some relationships between gaming and passwords; harassment and pornography,<br />
meeting strangers and pornography. Fraud is usually related to mobile phone or is carried out via<br />
mobile phones. As a rule, parents turn to police when they have lost money. The police cannot<br />
usually interfere when there are only moral problems. There are some relationships between direct<br />
messaging, pornography and meeting strangers because in direct messaging one can use video<br />
transfer.<br />
The stories from police indicated which paragraphs in the Estonian Penal Code are applicable for<br />
Internet crimes: PenC § 179 for showing pornography to minors, PenC § 217 for password hacking,<br />
PenC § 157-2 for identity theft, PenC § 213 for other computer frauds (Penal Code 2002).<br />
Our analysis of e-safety policies found on the official Web sites of ten Estonian secondary schools<br />
revealed that many schools address e-safety issues not with policy measures, but with technical<br />
access restrictions, e.g. filtering some web sites or blocking some services. Only a small number of<br />
schools had e-safety policies, which addressed topics like cyber-bullying or sexting (sending sexuallythemed<br />
texts or images via mobile phones). Some schools had regulations on taking pictures in<br />
school premises or sharing the visual material originating from the school (see Table 1). The<br />
government does not impose any relevant regulations on schools, besides generic legislation (e.g.<br />
penal code). While there are regulations used to develop schools' internal rules, they contain no<br />
mention of e-safety. Moreover, while schools are subjected to Lesser ISKE (Estonian three-level IT<br />
baseline protection system) framework, no practical support is provided.<br />
Table 1: Typical school rules regarding to use of ICT (based on policies of 10 schools)<br />
Type Rules<br />
Technical Every class or user has a different account<br />
The right to install or run software or print files is restricted or limited<br />
Technical filtering of the web<br />
People There are goals and priorities of the tasks what you can do with the school computers<br />
There are rules regarding the use of social network sites and direct messaging<br />
Computer lab working hours, health related rules (how long can you use a computer)<br />
Physical well-being of the computer workstation<br />
Monitoring the students<br />
Gaming is usually not allowed<br />
Other Penalties for breaking the rules<br />
Information where to turn for help<br />
One can make suggestions<br />
From the stories we also found the indications about what kind of regulations are needed. The typical<br />
recommendation was that the school must be present in the network because it influences students to<br />
behave better; the school must develop rules regarding to mobile and other devices usage in the<br />
schoolhouse; the school must filter programs and the web in the computer class; an "x" school<br />
student is not to make a fake account or post under someone else’s name; an "x" school student<br />
should not share his/her passwords with others; an "x" school student cannot slander others in the<br />
web; an "x" school student is to ask permission from other people before taking pictures in the school<br />
premises.<br />
5. Discussion<br />
The analysis of the students’ e-safety stories revealed several issues. It is clear that many students<br />
don’t apparently understand what e-safety means. Students do not usually think that they are in any<br />
way involved in an e-safety incident, even if they have been harassed or bullied on the Internet e.g. in<br />
a YouTube video about the teacher. The collection of stories from the “storytelling’” exercise and from<br />
the web-based competition provided similar results. Also, in a test with a control group giving students<br />
an e-safety topic (like privacy or virus) to write about, they first wrote something 'to please us' and<br />
then changed the story to something that they wanted actually to write about. It was quite interesting<br />
that they were not thinking about the given topic but rather writing about their real problem. It is<br />
something we should study further on in a deeper level.<br />
420
Birgy Lorenz et al.<br />
The students’ stories are highlighting the privacy issues in social networks, for it is easy to make fake<br />
accounts, gather personal information from search engines or even take pictures with mobile phones<br />
and post them online. Instead, people tend to believe that privacy is someone else's problem. They<br />
often also state that it is difficult or they do not have time to change privacy settings - but after an<br />
incident, they suddenly come to believe that they could learn to do that.<br />
The children's stories do not include stories about pedophilia and meeting strangers what is usually<br />
considered the biggest threat regarding e-safety. Children do not understand the differences between<br />
harassment and cyber-bullying – these terms are foggy to them. It can point to weak sexual education<br />
in Estonian schools because sexual topics are identified only 4 times out of 17 cases about<br />
harassment. But it can also be a topic that is not openly discussed among students. Also, the line<br />
between privacy and personal data protection is not really understandable to students. They tend to<br />
prefer black-and-white solutions: it's all private or not at all.<br />
The stories collected from police and IT experts differ because police deals with stories mostly only<br />
when there is direct monetary loss. On the other hand, schools often prefer to deal with incidents<br />
secretly, striving to find a solution between the parties without releasing the case to public.<br />
Looking at the results from the e-safety related policies of schools, we will see an interesting<br />
phenomenon - even when we did present typical sex-related stories to be discussed they were not<br />
considered a priority issue for schools. As seen by principals, harsh E-safety regulations at schools<br />
could be one of the future reasons for teachers not to use VLE, PLE and m-<strong>learning</strong> with students and<br />
only stay using the teacher's computer and projector for presenting their own materials, which is<br />
considered safe.<br />
Although our study focused on analysing e-safety incidents, it also informs the eLearning community<br />
about the need to raise awareness of teachers and students about potential threats to their privacy.<br />
This need becomes even more evident in the light of new trends related to the use of social media<br />
(blogs, wikis and social networking sites) as a new type of online <strong>learning</strong> environment (Becta 2008).<br />
And the most interesting is that in e-safety trainings usually are addressing MSN conversations and<br />
strangers issue where the prescribed solution is to block unpleasant persons, but today's interaction is<br />
moved to social networks where there is also opportunity to chat. When a child feels pressure to have<br />
more friends than other people then blocking an unpleasant person is not really an option. This also<br />
creates a privacy problem – the child opens his/her life to strangers even without direct<br />
communication.<br />
6. Conclusions<br />
Our overall conclusion is that typical e-safety policies must stress topics that all stakeholder groups<br />
agree being important: gaming, fraud, password, harassment, pornography and meeting strangers.<br />
There are direct relations between gaming and passwords; fraud and privacy in social networks;<br />
passwords and slandering in social networks; harassment and pornography. Students also point out<br />
problems regarding viruses, fake accounts, cyber-bullying, slandering – these are topics at which<br />
police is usually powerless to help. Our analysis shows that only a few schools have explicit policies<br />
which target e-safety issues. Yet, even these few existing school-level policy documents fail to<br />
address the topics which were most frequently mentioned in the stories written by students.<br />
Next, we should turn our attention towards evaluating the e-safety risks by themselves and influence<br />
of dangerous behaviour patterns to the online <strong>learning</strong> activities. As Internet usage has changed<br />
children’s values and patterns of online behaviour, the question is how we (as parents and educators)<br />
can adjust children's behaviour on the net if we still live in a different e-world. We should acknowledge<br />
that although we mostly use the same digital tools with the new generation, we still identify and<br />
handle the threats related with our online activities differently.<br />
References<br />
Becta (2008) Analysis of emerging trends affecting the use of technology in education, Research to support the<br />
delivery and development of Harnessing Technology: Next Generation Learning 2008–14<br />
Berson, I., Berson, M. and Ferron, J. (2002) Emerging risks of violence in the digital age: lessons for educators<br />
from an online study of adolescent girls in the United States, Meridian: A Middle School Computer<br />
Technologies Journal Vol. 5 No. 2, NC State University, Raleigh, NC<br />
421
Birgy Lorenz et al.<br />
Akdeniz, Y. (1997) The Regulation of Pornography and Child Pornography on the Internet, The Journal of<br />
Information, Law and Technology (JILT), UK<br />
Alwy, N. and Fan, I. (2010) Threat analysis for eLearning, Int. J. Technology Enhanced Learning Vol. 2 No 4, UK<br />
Bullen, P. (2000) The Internet: its effects on safety and behaviour implications for adolescents, Department of<br />
Psychology University of Auckland, Netsafe, New Zealand<br />
Carr, J. (2004) Child abuse, child pornography and the internet, ISBN 0900984805, NCH, London<br />
Chamaz, K. (2006) Constructing grounded theory: a practical guide through qualitative analysis, SAGE Publ.<br />
London, GB<br />
Children & Young People's Services (2011) Internet safety and model policies on the acceptable use of the<br />
internet for schools [online] http://www.bristol-cyps.org.uk/services/ict/acceptable.html<br />
Dworschak, M. (2010) The Internet Generation Prefers the Real World, Spiegel Online Internetional<br />
Enochsson, A. (2005) A gender perspective on Internet use – Consequences for information seeking on the net,<br />
Information Research, 10(4) p. 237<br />
Fundamental Team and Meeting Skills: Tools and Techniques (2003) New York State Governor's Office of<br />
Employee Relations [online]<br />
http://www.goer.state.ny.us/Training_Development/Online_Learning/FTMS/300s4.html<br />
Gagon, Y.(2010) The Case Study Research Method: practical Handbook, Presses de Universitete du Quebee,<br />
Canada<br />
Gray, P., Williamson , J., Karp, D. and Dalphin, J. (2007) The Research Imagination: an introduction to<br />
quantitative and qualitative methods, Cambridge University Press, GB<br />
Hoiser, F. (2009) Countering “it won’t happen to me”, Safety News Alert [online]<br />
http://www.safetynewsalert.com/countering-it-wont-happen-to-me/<br />
Hunter, N. (2005) “Jenny´s Story” – An internet safety resource developed to combat child abuse on the internet,<br />
Lancashire Constabulary, Pretton, UK [online] http://www.popcenter.org/library/awards/goldstein/2005/05-<br />
06.pdf<br />
Linstone, H. A., Turoff, M. and Helmer, O. (2002) The Delphi Method - Techniques and Applications, Murray<br />
Turoff and Harold A. Linstone.<br />
Livingstone, S., Haddon, L., Görzing, A., & Olaffson, K. (2010) Risks ans Safety on the Internet: The perspective<br />
of European children. London: LSE: EU Kids Online.<br />
Maadvere, I. (2010) IKT uues põhikooli riiklikus õppekavas [online]<br />
http://tiigrihypeharidustehnoloog.blogspot.com/2010/06/uusoppekava2.html<br />
Mitchell, K., Finkelhor, D. and Wolak, J. (2004) Victimization of Youths on the Internet, Victimization of Children:<br />
Emerging Issues (pp 1-39) New York: The Haworth Maltreatment and Trauma Press, NY<br />
Murumaa, M. (2011) Digisotsiaalsus õpetaja ja õpilase suhetes, Õpetajate leht Vol. 16 [online]<br />
http://www.opleht.ee/?archive_mode=article&articleid=5367<br />
National Curriculum - Põhikooli Riiklik õppekava (2010), Riigi Teataja [online]<br />
https://www.riigiteataja.ee/akt/13273133<br />
Ohio policy reference manual (2011) Sample Regulation computer/Online services [online]<br />
http://www.ohioattorneygeneral.gov/getattachment/48c7a1bc-6ccc-4486-8c79-9190f9a3f97a/Teacherstudent-relations-sample-policy-for-accept.aspx<br />
Okoli, C. and Pawlowski, S. D. (2004) The Delphi method as a research tool: an example, design considerations<br />
and applications, Information & Management Vol. 42 pp.15–29<br />
Penal Code - Karistusseadustik (2002) Riigi Teataja [online] https://www.riigiteataja.ee/akt/184411<br />
Peters, R. (2009) How Adult Pornography Contributes To Sexual Exploitation of Children, [online]<br />
http://www.obscenitycrimes.org/news/HowAdultPornographyHarmsChildren.pdf<br />
Rubtsova, P. (2011) Privacy in Social Media, Tallinn University.<br />
Safer internet for children qualitative study in 29 european countries, summary report (2007) Eurobarometer,<br />
[online]<br />
http://www.internetsafety.ie/website/ois/oisweb.nsf/0/F7650C73182B83B6802574D5004A932B/$File/Safer<br />
%20Internet%20for%20Children-%20Summary%20Report-march-may07.pdf<br />
TDL arvutiklasside kasutamise reeglid (2004) Tartu Descartes'i Lütseum [online]<br />
http://portal.tdl.ee/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=45&MMN_positi<br />
on=65:59<br />
Teddlie, C. and Tshakkori, A. (2009) Foundations of Mixed Methods Research: integrating Quatitative and<br />
Qualitative approaches in the social behavioural sciences, SAGE Publ. London, GB<br />
Towards a safer use of the Internet for children in the EU – a parents’ perspective Analytical report (2008)<br />
Eurobarometer, [online] http://www.internetsafety.ie/Website/OIS/OISWeb.nsf/page/DPCY-<br />
7MELY61694917-en/$File/Eurobarometer%20Survey%202008.pdf<br />
Vasli, K. (2011) Sotsiaalmeedia imed: 11 aastat vana raadionalja "Jah narkootikumidele" taassünd, Õhtuleht<br />
Online [online] http://www.ohtuleht.ee/427422<br />
Veldre, A. (2011) Kohutavate muutuste anatoomia, Õpetajate leht Vol. 16 [online]<br />
http://www.opleht.ee/?archive_mode=article&articleid=5368<br />
Vilke, M. (2000) Research study on children’s story-telling, Psycholinguistics on the Threshold of the Year 2000,<br />
pp. 453-461. Porto: Faculdade de Letras da Universidade do Porto.<br />
Wishart, J. (2004) Internet safety in emerging educational contexts, Computers & Education archive Vol. 43 No.<br />
1-2, August 2004 Elsevier Science Ltd. Oxford, UK, UK<br />
422
Learning Management Versus Classroom Management in<br />
Technology-Supported Blended Learning<br />
Arno Louw<br />
Centre for Technology Assisted Learning (CenTAL), University of<br />
Johannesburg, South Africa<br />
alouw@uj.ac.za<br />
Abstract: The comprehensive University of Johannesburg (UJ), South Africa, had to adapt to a fast growing<br />
student and academic community. Since 1998 a <strong>learning</strong> management system (LMS) has been introduced as to<br />
broaden the physical boundaries of classrooms over four campuses, student residences and residential areas<br />
(worldwide). The Centre for Technology Assisted Learning (CenTAL) was established to accommodate students,<br />
teaching and research staff and empower them to keep up with current and future trends in technology-assisted<br />
<strong>learning</strong>. Currently a blended <strong>learning</strong> approach, with direct face-to-face (F2F) contact with students, is followed<br />
and driven by the “<strong>learning</strong> to be” philosophy of teaching and <strong>learning</strong>. F2F contact is supplemented with a<br />
remote e-<strong>learning</strong> environment hosted in an LMS. Consequently, the implementation of technology in teaching<br />
and <strong>learning</strong> became an adopted phenomenon and promised increased success in higher education. However,<br />
changing curricula, staff turnover and increasing student numbers resulted in the efficacy of the LMS used, to be<br />
questioned. Moreover, the actual <strong>learning</strong> experiences of students are often dismissed as a new paradigm and<br />
statistical evidence leads to a contradiction about blended <strong>learning</strong> efficacy during annual course evaluations.<br />
This occurrence often causes confusion when effective use of an LMS is discussed. Subsequently, this inception<br />
forces the perception of key role players in two directions: managing virtual <strong>learning</strong> environments and managing<br />
<strong>learning</strong> within an electronic <strong>learning</strong> environment. Ultimately, <strong>learning</strong> management as viewed by the<br />
constructivist, and classroom management as viewed by the behaviourist, are aligned by presenting various<br />
examples from an LMS for operational e-<strong>learning</strong>. Therefore, this paper points out the different perceptions of<br />
lecturers towards the effective use of an LMS as remote component for blended <strong>learning</strong> at UJ. Emerging from<br />
the above rationale, this paper points out traditional perceptions of lecturers towards e-<strong>learning</strong> and further poses<br />
the question: Is <strong>learning</strong> management or classroom management the platform for effective blended <strong>learning</strong><br />
design? Findings and conclusions further addresses the consolidated data substantiated over a period of three<br />
years in nine faculties during research in professional development of faculty members related to effective,<br />
blended <strong>learning</strong>. The data and examples are presented as qualitative and, where more clarity is required,<br />
substantiated by quantitative data. The actual examples presented, point out teaching practices where classroom<br />
management was used as point of departure and then contrasted with examples where <strong>learning</strong> management<br />
was used as a starting point. This paper further reveals e-<strong>learning</strong> environments where this dichotomy becomes<br />
distinguishable but inseparable. An in-depth discussion extrapolates instructivist and constructivist approaches to<br />
be extremes of a continuum whereby ideal balance becomes a superlative teaching application for both F2F and<br />
online <strong>learning</strong> environments. In this case study this pedagogical paradigm escalates to an imperative for<br />
implementing effective technology-supported blended <strong>learning</strong> at the University of Johannesburg.<br />
Keywords: technology-assisted <strong>learning</strong>; e-<strong>learning</strong> environment; <strong>learning</strong> management system (LMS); <strong>learning</strong><br />
management; classroom management; technology-supported blended <strong>learning</strong><br />
1. Introduction<br />
Blended <strong>learning</strong> at UJ mainly uses ICTs and specifically an LMS whereby on and off line activities<br />
and participation encompasses the style of teaching. This approach is conducted from a policy on<br />
teaching and <strong>learning</strong> strategies. The background to this is presented followed by a brief description<br />
on the methodology followed. Thereafter, the perception of students, as the Y generation is described.<br />
The discussion about the impact and considerations escalates into the perceptions of students and<br />
lecturers. After the perceptions have been clarified, the context of classroom management and<br />
<strong>learning</strong> management within blended <strong>learning</strong> is discussed. This is then followed by examples as part<br />
of the findings and conclusions followed by a summary.<br />
2. Background and the adopted philosophy<br />
Recognising the change which enhances teaching and <strong>learning</strong> strategy, the University of<br />
Johannesburg (UJ) has adapted a philosophy for teaching and <strong>learning</strong>. This philosophy has become<br />
university policy and is circumscribed under Section 1 of the Principles of a Teaching and Learning<br />
Strategy as follows: UJ recognizes the complexity and rapidly changing nature of the social, economic<br />
and intellectual environment for which its students are being prepared. It is imperative, therefore, that<br />
teaching and <strong>learning</strong> at UJ should transform from a primary concern with the transmission of<br />
knowledge (<strong>learning</strong> about) to a primary concern with the practices of a knowledge domain (<strong>learning</strong><br />
to be). Therefore, in its teaching and <strong>learning</strong> activities and in the design of its modules and<br />
423
Arno Louw<br />
programmes, and within the parameters of reasonably practicable implementation. (UJ Senate<br />
Teaching and Learning Committee 2009).<br />
A proof to the growth in student numbers and the accompanying lecturer growth is shown in reports<br />
from CenTAL for 2008 to 2010. The reports reveal an increase in students and lecturers using the<br />
LMS. One can arguably agree that these numbers will inevitably become unlimited as usage of the<br />
Internet, ICTs, Web 2.0 <strong>learning</strong> tools, and an LMS as <strong>learning</strong> tool increases over the next few years.<br />
Consequently, the same approach for using these numbers was change when the 2010 report was<br />
published. Therefore, the approach to what is seen as effective use of the Internet and World Wide<br />
Web (WWW) as <strong>learning</strong> tools, has become biased toward the actual teaching needs from lecturers<br />
and the <strong>learning</strong> needs of students. This discrepancy highlights an immediate need for lecturers to be<br />
related to the rate of adoption to incorporating the Internet and the WWW into blended <strong>learning</strong>.<br />
3. Methodology and data gathering<br />
Over this decade, ICTs, Web 2.0 facilities and a new generation of students have also become part of<br />
daily teaching practices and many faculties had to adapt to new and innovative ways of teaching and<br />
<strong>learning</strong>. Since the University of Johannesburg has adopted a blended <strong>learning</strong> approach, the LMS<br />
has been upgraded in 2002, 2006 and in 2010 the pilot launch of the new Blackboard system had<br />
been launched in 2011. I have been involved in these processes and implementations and have<br />
worked closely with over one hundred lecturers in nine faculties over four campuses. The process<br />
followed for presenting the research in this paper, comes from notes, conversations, being involved in<br />
implementation procedures and training many lecturers and students since 1998. The subsequent<br />
involvement in many research project in this time, has delivered trends which I share within the scope<br />
and limitation of this paper. It is intriguing to observe how lecturers adapt and come up with new<br />
initiatives in spite of growing student numbers and limited physical classroom space on campuses.<br />
Therefore, the discussions, conclusions, and given facts in this report can be substantiated with<br />
literature, data from and data generated by the LMS currently in use. Apart from using data from<br />
many cases, I will mostly focus on projects involving large classes in modules for Commercial Law, B<br />
Tech bridging courses, Language Skills for Science, and Business Management. The findings,<br />
conclusions and arguments are presented together with the discussions as to promote the integrated<br />
nature of this research. However, a separate section for findings and conclusions will also be given as<br />
an accumulation of what is assumed to be effective blended <strong>learning</strong>.<br />
4. Impact and considerations<br />
The diffusion theory has its origins in work done in 1943 in rural sociology. According to Jones,<br />
Jamieson and Clark (2005), the Diffusion, per se, is defined as “a process that occurs over time and<br />
can be seen as having five distinct stages – knowledge, persuasion, decision, implementation, and<br />
conformation.” The diffusion theory framework involves four main elements: innovation,<br />
communication channels, time and the social system. Furthermore, these elements “influence the<br />
adoption or rejection of an innovation in a complicated, interdependent way” (op. cit.). The same<br />
authors refer to Jacobsen (2000) who used the five stages in adopting technology for teaching and<br />
<strong>learning</strong> and various other authors (Bronack & Riedl 1998; Schefstrom et al. 1998; Surry & Farquahar<br />
1997) who used several diffusion theories used and relates specifically to an instructional technology<br />
model. The common elements of most diffusion theories are quoted in table 1 as proffered by Jones,<br />
Jamieson and Clark (2005) and given in relation to the UJ context.<br />
Table 1: Common elements of diffusion theory<br />
Influential Elements Description<br />
Diffusion is a process that occurs over time and can be<br />
Diffusion process<br />
seen as having five distinct stages:<br />
Adopter categories<br />
Perceived attributes<br />
Rate of adoption<br />
Lecturers vary greatly in their willingness to adopt a<br />
particular innovation. Individual characteristics can be<br />
used to divide the population into categories:<br />
How innovation is perceived, influences the adoption<br />
decision. Five perceived attributes of an innovation<br />
have been shown to have strong influence:<br />
The relative speed with which lecturers in a social<br />
system adopt innovation. Innovations are diffused over<br />
time in a pattern that resembles an s-shaped curve.<br />
424
Arno Louw<br />
Devedšić poses (op. cit.) that organising and linking materials on the Web, as well as extracting and<br />
interpreting the content for the sake of effective <strong>learning</strong>, places the burden on the human user. I<br />
argue that the perceptions of lecturers and students have an influence on the rate of adopting blended<br />
<strong>learning</strong> because hard and soft skills, accompanying values, and expected attitude(s) must be part of<br />
a curriculum. I maintain further that a power play for the ideal <strong>learning</strong> process in blended <strong>learning</strong><br />
causes misperceptions. Therefore, it is not only stakeholders of the <strong>learning</strong> process who should<br />
adopt, but also the content which needs to be aligned with demands in industry upon graduating a<br />
student (suggested by the teaching and <strong>learning</strong> philosophy of UJ).<br />
4.1 Students perception and expectations from blended <strong>learning</strong><br />
To know the background of students might be considered cliché. However, within the South African<br />
context, various background and cultures, 11 official languages, accommodating people with<br />
disabilities, growing student numbers, and comprehensive 4 campuses will always influence<br />
perception. However, common trends of the current student generation contribute to enlighten our<br />
understanding of the student of today.<br />
According to Smith and Grant-Marshall (2004), the South African generational divide differs from the<br />
international division. They define the generations in South Africa as follows: Baby Boomers were<br />
born between 1950 – 1969; Generation X people were born between 1970 – 1989 and Generation Y<br />
were born between 1990 – 2005. This division, as with all divisions, is however not clear cut. People<br />
who were born, either early or late in a generation, is known as cuspers. Cuspers show traits of both<br />
generations. To understand today’s students, it is necessary to briefly look at some of the traits of<br />
each of the generations, with particular emphasis on technology as there is definitely a digital divide<br />
between the various generations. Baby Boomers grew up following World War II. They grew up with<br />
one-way technology, being the radio. Their introduction to technology was in the 80s with the advent<br />
of the personal computer (PC).<br />
Generation X (also known as the lost generation) loves to communicate by e-mail and want instant<br />
answers to questions and know how to get speedy access to information. They are far more adept<br />
with technology than their parents (Baby Boomers). However, they often lack the technology savvy of<br />
Generation Y (or the Millenials). Generation Y grew up in the full swing of technology – 24/7<br />
television, the Internet, social networking including Facebook, Blackberry applications, iPhone<br />
facilities etc. Generation Y wants lots of knowledge and they want it instantly. They thus rely on<br />
technology to assist them in obtaining the knowledge and to share their knowledge with others. In<br />
addition, expensive cell phones, bigger head phones, and iPods are displayed. A recent report from<br />
UJ (Bester, 2011) conveys that computer orientation for first year students delivered results shown in<br />
Figure 1 and Figure 2.<br />
Figure 1: First year access to different technologies<br />
425
Arno Louw<br />
It should be clearly stated that this is personal access to these technologies and the interpretation to<br />
the question is often confused with “owning” the technology rather than have actual access. In<br />
addition, many students have access to computers and the Internet via hotspots in residences, per<br />
immediate family households or communes, or at home where a computer is shared by many people.<br />
Informal discussions with students brought forth that the computer is used for social networking (e.g.<br />
Facebook) and gaming, yet applications such as the word processor, and spread sheet is never used<br />
for formal writing although presentation software is not all that unfamiliar to them. Hence figure 4<br />
reveals the computer proficiency of current second and first year students.<br />
Bearing all said in mind, Baby Boomers were taught by the Traditionalists (born during 1930 – 1949),<br />
in accordance with the behaviourist theory, i.e. rote memorisation. Baby Boomers rebelled against<br />
this format and taught Generation X in line with the cognitivist approach to teaching and <strong>learning</strong>.<br />
Rote memorisation still forms the cornerstone of the cognitivist approach, but it goes one step further<br />
in that the information that was memorised then is used by applying it to a set of facts as one would<br />
experience it in real life. Generation Y’s education follows the social constructivist approach, which<br />
strongly relies on collaborative <strong>learning</strong>, which taps neatly into Generation Y’s preference for working<br />
in groups. Even more so, the <strong>learning</strong> theory of Connectivism emerges from the <strong>learning</strong> styles of<br />
these students (Siemens 2005). This trend, in teaching and <strong>learning</strong> in society, enlightens our<br />
paradigm for understanding the <strong>learning</strong> needs of our students.<br />
Figure 2: Computer proficiency of first years 2010 – 2011 (comparison)<br />
These students adapt easy to e-<strong>learning</strong> when analysed against the diffusion theory and are classified<br />
in five categories, namely: innovators, early adopters, early majority, late majority, and laggards<br />
(Keller & Cernerud 2002). In addition, familiarity with e-<strong>learning</strong>, an LMS, and ICTs as components of<br />
e-<strong>learning</strong>, are largely accepted by modern students. In a later study, Buzzetto-More (2008) finds that:<br />
“When prior educational exposure [to e-<strong>learning</strong> components] was examined, the majority of students<br />
indicated that they had used a computer to solve a problem as part of a class assignment,<br />
participated in group work that involved using computer software, and delivered a presentation using<br />
computer software.” She also found that “students’ perceptions and experiences with online <strong>learning</strong><br />
were similar to findings reported from studies conducted at majority institutions where students have<br />
reported that they want to see traditional <strong>learning</strong> supported by e-<strong>learning</strong> strategies; however, faceto-face<br />
instruction is preferred over fully online <strong>learning</strong>.” (op. cit.).<br />
4.2 Pedagogical design for blended <strong>learning</strong> interventions<br />
The affordance of the Web bears the following constraints that must be addressed as it influences the<br />
theory behind applying <strong>learning</strong> in the online environment. Anderson (2004) describes the following<br />
426
Arno Louw<br />
limitations that I will list without an in depth discussion as it is recognisable and contentious within the<br />
South African educational realm.<br />
Anderson (2004) proffers the following model that shows the types of online interaction that occurs in<br />
a virtual educational environment.<br />
By using the model in Figure 3, online instructors and designers are faced with crucial decisions<br />
based on different <strong>learning</strong> outcomes that will be best learned through specific types of <strong>learning</strong><br />
activities in an online environment. The decision on the design of the different <strong>learning</strong> activities, as<br />
part of the instructional design process, arguably is not based on how learners learn but rather on<br />
How do they learn what?<br />
Figure 3: Online <strong>learning</strong> and types of interaction (Anderson, 2004:49)<br />
4.2.1 The dimension of user activity<br />
Two stereotypes of <strong>learning</strong> environments are distinguished, namely, mathemagenic and generative.<br />
These types of <strong>learning</strong> environments also become the distinguishable ranges on the continuum in<br />
the pedagogical dimension of user activity. Reeves (1997) describes a mathemagenic environment,<br />
according to Hannifin (1992), to “access various representations of content.” Contrarily, other<br />
generative <strong>learning</strong> environments are described to “engage learners in the process of creating,<br />
elaborating or representing knowledge.” (op. cit). I draw on a conclusion that the online <strong>learning</strong><br />
environment should attempt to become more generative in its nature. The environment should by<br />
nature attempts to engage both learners and facilitators to interactively mentor, facilitate and promote<br />
peer discussions and participation, and is socially constructivistic. Furthermore, the activities should<br />
be designed to introduce virtual and mental collaboration tools to the participants. Reeves (1997)<br />
substantiates by stating that “Generative <strong>learning</strong> environments are aligned more closely with<br />
contructivist pedagogy whereas mathemagenic environments are often based on instructivist<br />
pedagogy, but is not necessarily always obvious.” Thus, the activity design becomes cardinal in the<br />
promotion of constructivist pedagogy in blended <strong>learning</strong>.<br />
4.2.2 The dimension of learner control<br />
This pedagogical dimension has been the most researched as a topic. This dimension ranges from<br />
allowing learners to have control by allowing them to “make decisions about what sections to study<br />
and/or what paths to follow through interactive material.” (Reeves 1997). The range in this<br />
pedagogical dimension varies the continuum from unrestricted to non-existent. Furthermore, an<br />
accurate explanation of this dimension is given by Powell (2000) stating that the issue of control “is<br />
427
Arno Louw<br />
somewhat an unwritten contract between the site user and the developer. There is a give and take in<br />
the relationship.” The author adds that “... users are imperfect like everyone else; if we give them<br />
complete control, they make serious errors. If you control things too much and the user notices that<br />
they can’t resize their window or press certain buttons, they may become angry or frustrated.”<br />
Moreover, the same author argues that “Users can do everything they need to do and nothing more.<br />
People need to feel like they are in control but the control should have limits. Good interfaces exhibit<br />
this control.”<br />
4.2.3 The dimension of flexibility<br />
The continuum of a <strong>learning</strong> program can be unchangeable or more learner accommodative.<br />
Therefore, this dimension ranges between a teacher-proof programme to an easily modifiable<br />
programme. A programme must make provision for changing content, adaptation to the immediate<br />
need(s) of the learners as well as the design of activities that can be adapted according to changes in<br />
the dynamic curriculum that might be influential to good classroom practice. Earle (2002) explains that<br />
flexibility within an online environment refers to the customisability. She refers to Schatz who believes<br />
that “we are on the verge of being able to provide <strong>learning</strong> customized for each specific learner at a<br />
specific time, taking into account their <strong>learning</strong> style, experience, knowledge and <strong>learning</strong> goals.”<br />
Moreover, the author focuses the discussion on flexible <strong>learning</strong> as to be the successor of<br />
programmes designed to be flexible. She poses the following ten stipulations as to describe flexible<br />
<strong>learning</strong>. In this regard, Earle (2002) further draws on Reigeluth (1999) and lists the inclusion<br />
embedded in these values, as: authenticity, ownership, and relevance of the <strong>learning</strong> experience for<br />
students; rich social contexts and multiple perspectives for <strong>learning</strong>... the critically of “what to teach”<br />
and the considerable variability of “how to teach it”; <strong>learning</strong> that is driven by an ill-structured problem<br />
(or question, case, project). The overseer of a workable pedagogy is the lecturer and therefore the<br />
perception of the lecturer during the diffusion process is important.<br />
4.3 Lecturers perception and expectations from blended <strong>learning</strong><br />
Bombarding lecturers with all that is said in this section is certainly overwhelming. Even more so,<br />
considering the different generations and the rate of adoption, poses a challenge toward the process<br />
of professional development. Systematically introducing lecturers to the blended <strong>learning</strong> approach,<br />
integrated between F2F and online teaching in an LMS, caused just as much acceptance, resilience<br />
and resistance. The acceptance of blended <strong>learning</strong> usually calls for immediate implementation. After<br />
considering whether students should really be encouraged to bring cell phones to class, Orlando<br />
(2011) says: “For one, students will eventually be entering the working world where these devices are<br />
ubiquitous. Look at any business meeting and you will find everyone with smart phones or laptops.<br />
The restriction on their use amounts to telling students that they need to go back to using pen and<br />
paper for notetaking, like the 60’s. We are preparing students for a world that no longer exists.” He<br />
pleads further that instead of “fighting the digital movement, try treating it as a collaborator in the<br />
<strong>learning</strong> process and a way to get all of your students involved in class.” This positive perception<br />
motivates older lecturers until the realisation, with the amount of planning for successful blended<br />
<strong>learning</strong>, becomes certainty.<br />
The LMS becomes more of an organisational tool. Lecturers all start to use the LMS for managing<br />
groups of students. The following uses often occur when lecturers adopt using an LMS to extent their<br />
F2F meetings:<br />
Large classes receive class notes (mainly Power Point slides) after the lecture via the LMS to<br />
ensure that students pay attention in class.<br />
Announcements for venue changes, due dates, and quick arrangements are ensured to reach all<br />
students.<br />
The burden of checking for plagiarism in paper submissions becomes the responsibility of the<br />
student.<br />
Assignments can be submitted electronically via the LMS and hard copies do not get lost. A<br />
proper submission record is also viable.<br />
Assessments (usually tests) can be done online and record keeping is assured.<br />
Statistical analysis of “activities” can deliver data for quick reports.<br />
There is better control over class lists and group submissions.<br />
428
Arno Louw<br />
Occasionally other resourceful web sites can be visited as to allow for access to more resources.<br />
Consultation times are setup so that all students are aware.<br />
Teaching assistants and tutors become maintainers of the online component of certain courses.<br />
(The young ones know how the system works…).<br />
The LMS can be remotely accessed when necessary.<br />
The evaluation of the online component of courses, proofs delivers ‘dump’ sites and the<br />
functionality is largely underutilised.<br />
This occurrence is observable with many lecturers although there are the few champion users who<br />
use the integrated approach differently. However, I question the students’ involvement in the <strong>learning</strong><br />
process as commended by Orlando (2011). Thus far, I conclude that the stipulated occurrences tend<br />
more towards classical classroom management and not the actual <strong>learning</strong> management expected<br />
from an LMS in use. Herein lies the topic of this paper and I shall now refer to these concepts.<br />
5. Classroom management versus <strong>learning</strong> management<br />
Many authors describe the term Classroom management. Objective to being partial, classroom<br />
management emphasises a manager in a classroom (in this paper referred to as the ‘lecturer’), who<br />
mediates the process of implementing a curriculum as efficiently as possible. (Allyn & Bacon, 1986;<br />
Moskowitz & Hayman, 1976; Berliner, 1988; Brophy & Good, 1986; Allen,1986; Gootman, 2008;<br />
Pintrich & De Groot,1990).<br />
On the other hand, it becomes a challenge to describe Learning management. However, bearing this<br />
caveat in mind, the research followed a direction that greatly resembles the discussions of many<br />
authors. Bart (2011) denotes Jacoby who recommends four steps towards designing and planning<br />
critical reflection which adds to breadth and death to student <strong>learning</strong>. Furthermore, also bearing in<br />
mind that <strong>learning</strong> should be designed as a blended approach interrelated between a F2F and online<br />
ecology, I refer to chain the work of Bart (2011) and Shibley (2011). In this instance, Shibley (2011)<br />
accurately explains that: “Blended <strong>learning</strong> combines the best elements of face-to-face classes and<br />
online classes. Get proven strategies on how to create a blended course that improves the teaching<br />
and <strong>learning</strong> experience.”<br />
5.1 The confusion between classroom management and <strong>learning</strong> management<br />
Lecturers and students are both exposed to the opportunities available for implementing successful<br />
blended <strong>learning</strong>. To make a decision on which aspects are classroom management related and<br />
which are <strong>learning</strong> management related is difficult at this stage although examples can be given as to<br />
which is the most appropriate for each aspect. At UJ, the following list of contributing factors causes<br />
confusion for distinguishing between classroom management and <strong>learning</strong> management when using<br />
the LMS:<br />
There is a general confusion about administration and housekeeping of certain courses;<br />
Unexpected increases in student numbers;<br />
Changing curricula;<br />
Late student registration;<br />
First generation students;<br />
Freedom of choice from students;<br />
Late organisational adjustments;<br />
Large classes (600+ students);<br />
Increased responsibility for both lecturer and students;<br />
Getting class lists in time after registration;<br />
More frequent assessments which results in shorter tests;<br />
Student tracking capabilities;<br />
Saving paper, and<br />
Extended degree courses to give all students an equal opportunity.<br />
429
Arno Louw<br />
Lecturers assume that <strong>learning</strong> only occurs in the LMS environment and use the classroom to<br />
present content only.<br />
Consequently, these factors also become reasons for lecturers finding an online component (LMS)<br />
more lucrative to use. This is as far as the LMS is used and cause true blended <strong>learning</strong> never to<br />
escalate to its full capacity in certain courses.<br />
5.2 Considerations for proper classroom and <strong>learning</strong> management<br />
Within an online environment, the following should always be considered when assigning students to<br />
work independently from classroom facilitation:<br />
What is there is no teacher present?<br />
The use of netiquette.<br />
Is navigation an obstacle for <strong>learning</strong>? (Number of clicks.)<br />
How is <strong>learning</strong> managed in a classroom as opposed to how <strong>learning</strong> is managed in a classroom<br />
with shifting borders?<br />
I argue and maintain that an LMS is a tool for teaching and <strong>learning</strong> and not only for teaching or<br />
<strong>learning</strong>. Perceptions of the lecturers over the past few years have been that the LMS becomes more<br />
an organisational tool to consolidate large classes and ensure the integrity of disseminating<br />
information. However, information is often distinguished from actual subject content and thus does not<br />
form part of actual blended <strong>learning</strong> pedagogy. This often becomes the assumption upon which the<br />
use of the LMS is based. It should also be mentioned that, in a few exemplary cases, the exact<br />
opposite is also true which presumes that a course could have been presented online in its totality.<br />
Does this really equip students with hard and soft skills upon graduation? Moreover, does this fulfil the<br />
goal of the university’s <strong>learning</strong> to be philosophy of teaching and <strong>learning</strong> as opposed to <strong>learning</strong> to<br />
do?<br />
6. Findings and conclusions<br />
The contextual use of F2F and online component of blended <strong>learning</strong> gives of view of classroom<br />
management and <strong>learning</strong> management. The context will be given, briefly described from the<br />
literature followed by an example. The contexts can be: learner-centred, assessment-centred,<br />
collaborative, or community-centred.<br />
A lecture is presented by DVD in many<br />
venues or by streaming from a remote<br />
classroom. The lecturer is shown on a big<br />
screen in the laboratory.<br />
Figure 4: Example 1: Language and skills for science (DVD)<br />
Students are watching the lecture from a computer<br />
laboratory on campus and are interactively involved<br />
with activities on the computer. An assignment has<br />
to be completed by the end of the lecture. Tutors<br />
assist in the classroom management.<br />
430
Lecturers and tutors are online after<br />
the F2F meeting.<br />
Figure 5: Example 2: Law discussions<br />
Students download a video clip<br />
with new vocabulary verbally<br />
given to them. This is done via<br />
the LMS.<br />
Figure 6: Example 3: Cell phone vocabulary<br />
A community is available in the<br />
LMS.<br />
Arno Louw<br />
Students participate in a formal discussion group under<br />
the headings:<br />
“What you want us to know” and<br />
“What we want you to know”<br />
LIBRARY<br />
Thereafter, a<br />
dictionary should be<br />
consulted in the<br />
library to find the<br />
correct spelling and<br />
definition of the given<br />
vocabulary.<br />
Figure 7: Example 4: Writing Centre Community<br />
A paragraph must be submitted online<br />
using the vocabulary.<br />
Participation begins online with individual students and<br />
grows to workshops with groups in a F2F classroom.<br />
431
A treasure hunt has to be done to find<br />
locations for pedagogical design. Students<br />
have to describe the route in order to reach<br />
their outcome. This is done independent of the<br />
LMS.<br />
Figure 8: Example 5: Google maps example<br />
Arno Louw<br />
This activity culminates in a workshop session<br />
that happens F2F.<br />
Students have to make a journal entry after every lecture. They are rewarded marks if they can<br />
explain what knowledge and skills they have gained by making examples of real life situations.<br />
Figure 9: Example 6: Reflection journal (ONB)<br />
7. Summary<br />
This paper presented an argument that there is a meaningful difference between classroom<br />
management and <strong>learning</strong> management. The perception causing this difference is situated in the<br />
adoption of lecturers toward using blended <strong>learning</strong>. Blended <strong>learning</strong> at UJ mainly uses ICTs and<br />
specifically an LMS whereby on and off line activities and participation encompasses the style of<br />
teaching. This approach is conducted from a policy on teaching and <strong>learning</strong> strategies. The<br />
background to this was presented followed by a brief description on the methodology used.<br />
Thereafter, the perception of students, as the Y generation was described. The discussion about the<br />
impact and considerations escalated into the perceptions of students and lecturers was presented.<br />
After the perceptions had been clarified, the context of classroom management and <strong>learning</strong><br />
management within blended <strong>learning</strong> was discussed. This is then was followed by examples as part<br />
of the findings and conclusions. Thereafter, a summary of the paper was given.<br />
432
References<br />
Arno Louw<br />
Allen, J.D. 1986: Classroom management: students' perspectives, goals, and strategies. American Educational<br />
Research Journal, 23, 437-459.<br />
Anderson, T 2004: Online Learning Theory. Towards a theory of online <strong>learning</strong>. Chapter 2. Anthabasca<br />
University. Online book. Available:http://www.cde.athabascau.ca/online_book/ch2.html Accessed: 22<br />
November 2006.<br />
Bart, M. 2011 Critical Reflection Adds Depth and Breadth to Student Learning. FACULTY FOCUS, May 11 2011.<br />
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com<br />
Berliner, D. C. 1988: Effective classroom management and instruction: A knowledge base for consultation. In J.<br />
L. Graden, J. E. Zins, & M. J. Curtis (Eds.), Alternative educational delivery systems: Enhancing<br />
instructional options for all students (pp. 309-325). Washington, DC: National Association of School<br />
Psychologists.<br />
Brophy, J. E., & Good, T. L. 1986:. Teacher behavior and student achievement. In M. C. Wittrock (Ed.),<br />
Handbook of research on teaching (3rd ed., pp. 328-375). New York: Macmillan.<br />
Buzzetto-More, N.A. 2008: Student perceptions of various e-<strong>learning</strong> components. Interdiciplinary Journal of E-<br />
Learning and Learning Objects. 4(2008).<br />
Cambridge, D.; Kaplan, S. & Suiter, V. 2005 Communities of Practice Design Guide.<br />
http://www.educause.edu/ir/library/pdf/NLIO531.pdf.<br />
Clark, W. 2009 Beyond Web 2.0: mapping the technology landscapes of young learners Journal of Computer<br />
Assisted Learning. Article first published online: 16 JAN 2009 DOI: 10.1111/j.1365-2729.2008.00305.x<br />
Devedzic, V.B. 2003: Key issues in next-generation web-based education Available:<br />
http://scholar.google.com/url?sa=U&q=http://ieeexplore.ieee.org/iel5/5326/27780/01238675.pdf%3Farnumb<br />
er%3D1238675 Accessed: 20 January 2006.<br />
Drucker, P. 1999 Management Challenges for the 21st Century ,Harper Business<br />
Garrison, D.R. & Akyol, Z. 2009 Role of instructional technology in the transformation of higher education.<br />
Journal of Computing in Higher Education. Volume 21, Number 1, 19-30, DOI: 10.1007/s12528-009-9014-7<br />
Gootman, M.E. 2008: The caring teacher's guide to discipline : helping students learn self-control, responsibility,<br />
and respect, K-6. 2008, p.36<br />
Keller, K. & Cernerud, L 2002: Students’ perceptions of e-<strong>learning</strong> in university education. Journal of Educational<br />
Media. 27(2002) Nos. 1-2.<br />
Kim, B. 2001 Social constructivism. In M. Orey (Ed.), Emerging perspectives on <strong>learning</strong>, teaching, and<br />
technology. http://www.coe.uga.edu/epltt/SocialConstructivism.htm<br />
Kim, B. 2010 Web 2.0 and Emerging Learning Technologies.<br />
http://en.wikibooks.org/wiki/Web_2.0_and_Emerging_Learning_Technologies /Innovative_Pedagogies<br />
Kohonen,V. 2009 Learning to learn through reflection – an experiential <strong>learning</strong> perspective.<br />
archive.ecml.at/mtp2/Elp_tt/.../Supplementary%20text%20E.pdf<br />
Moskowitz, G., & Hayman, J. L., Jr. 1976: Success strategies of inner-city teachers: A year-long study. Journal of<br />
Educational Research, 69, 283-289.<br />
Orlando, J. 2011: Bring your computer to class day. FACULTY FOCUS, June 6 2011.<br />
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com<br />
Pintrich, P.R., & De Groot E. V. 1990:. Motivational and self-regulated <strong>learning</strong> components of classroom<br />
academic performance. Journal of Educational Psychology, 82, 33-40.<br />
Reeves, T.C. & Hedberg, J.G. 2003: Evaluating interactive <strong>learning</strong> systems. Athens, GA: University of Georgia,<br />
College of Education.<br />
Siemens, G. (2005). Connectivism: A Learning Theory for the Digital Age. Internationa Journal of Instructional<br />
Technology & Distance Learning , 2 (1).<br />
Shibley, I 2011 Putting the Learning in Blended Learning. FACULTY FOCUS, February 21 2011.<br />
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com<br />
Smith, G. & Grant-Marshall, S. 2004: Mind the Gap. Johannesburg: Penguin Books.<br />
University of Johannesburg Senate Teaching and Learning Committee: Teaching and Learning Strategy,<br />
3/2009(1))<br />
Wenger, E 2011: Communities of Practice (on line: http://www.ewenger.com/theory/.accessed 16 May 2011.<br />
Wolfgang, C.H. & Glickman, C.D. 1986: Solving Discipline Problems. Allyn and Bacon.<br />
433
How to Represent a Frog That can be Dissected in a Virtual<br />
World<br />
Robert Lucas<br />
Department of Physical Sciences, The Open University, Milton Keynes UK<br />
r.j.lucas@open.ac.uk<br />
Abstract: One of the problems in creating a realistic virtual dissection is how to represent the solid nature of a<br />
substance and any subcomponents it may have using surface modellers such as SketchUp (Sketchup 2011) and<br />
Maya (Maya 2011). Although solid modellers are available, models produced by them are not generally usable by<br />
modern game engines. This paper describes a practical method of representing and manipulating objects so that<br />
they can be cut at arbitrary points by a planar knife and behave as though they were solid. Although this paper is<br />
mostly concerned with cutting through 3D models in order to give the illusion of them being made of a<br />
homogenous material, some discussion of how other aspects of the dissection process is also included.<br />
Background on how models are represented by meshes made from lists of triangles is given. The representation<br />
and the algorithms to support the cutting operations are given at a fairly high level. Complete implementation<br />
details and discussion of all possible pathological cases are not included. Some examples of the cutting process<br />
being used on a simple and more complex object are given. In particular we show how a model of a Malteaser<br />
can be cut in half where its internal texture is a photograph of the actual insides of a real Malteaser. This gives a<br />
very realistic cut that entirely creates the illusion of the 3D model being solid. We also show how the skin of a frog<br />
can be represented so that it appears to be solid when cut. This paper will be of interest to implementers of any<br />
3D virtual application requiring the realistic representation of arbitrary planar object cutting.<br />
Keywords 3D, virtual, dissection, cutting, surface modelling<br />
1. Introduction<br />
For many years 3D graphics has been used to provide games on PCs and consoles and the huge<br />
profits obtainable from successful games have been a huge driving force in the parallel development<br />
of hardware such as graphics cards and consoles, and software that implements 3D modelling and<br />
animation environments. Recently there has been an explosion of 3D simulation in science including<br />
many examples by the author, which include a 3D Celestial Sphere (Lucas 2010), A Telescope<br />
Simulator used to train students embarking on a course in observational astronomy (Lucas, Kolb<br />
2009), some science experiments (Lucas 2009) and an archery simulation for teaching the motion of<br />
projectiles (Lucas 2010).<br />
Many simulations of scientific processes can be achieved fairly straightforwardly as the above<br />
applications demonstrate. However, in order to create a realistic simulation of a dissection of an<br />
animal such as a frog many components need to be in place. Organic materials range from being soft<br />
to hard. They often contain liquids and gases. They will sometimes be elastic and sometimes plastic<br />
in how they deform under pressure. There will often be some surface tension that will pull the sides of<br />
an incision apart for example. A common operation in dissection is pulling a surface tissue back and<br />
perhaps pinning it. These are all issues that need to be addressed when constructing a 3D simulation.<br />
There are many issues surrounding dissection and whether we should be doing it with actual animals<br />
and in particular with frogs (Save the frogs 2011). These issues, although interesting are not<br />
discussed here. However, if we are to replace real dissection with something virtual then we want this<br />
something to be as realistic as we can possibly make it. Existing dissection applications consist of<br />
models with discrete components that simply allow the user to take them apart, such as in the case of<br />
a virtual frog, where it is possible to remove its organs as entire entities. More realistic applications<br />
(Digital Frog 2011) allow cuts to be followed with a mouse and then play a movie of an actual<br />
dissection of that part. This is a far cry from an actual dissection where a scalpel is used to cut<br />
through tissue, organs and whatever the blade comes up against. In all 3D graphic systems that are<br />
used in games and virtual worlds such as second life; objects are created using a mesh to represent<br />
the geometry of the surface and a texture used alongside a shader (i.e. a material) to render<br />
something that looks solid. Cutting into such a model would instantly reveal it to be hollow (or not<br />
visible at all). Although it is possible to put one model inside another, and then another model inside<br />
this one and so on, this is a poor approach as this method is incapable of representing the<br />
homogeneity of a substance such an amorphous tissue which might fill the space between the<br />
surface of one object and the surface of another. What is required is a method of cutting the meshes<br />
that constitute the geometry of an organ, and as the mesh is cut (or as soon as the cut is completed),<br />
434
Robert Lucas<br />
rebuilding the mesh on-the-fly with the appropriate texture. Cutting is a common technique in 2D<br />
graphics, sometimes used in mapping applications to enable arbitrary regions to be cut from a larger<br />
map. Often this will be done so that where a public utility gets cut, for example, the properties that<br />
belonged to the original are re-attached to the cut parts. This is the approach taken here and is<br />
extended to 3D. We use a set of materials consisting of textures and shaders that are associated with<br />
the various components that construct the object to enable us to recreate the cut objects and its<br />
components. Components are recursively built from subcomponents to the level of detail required,<br />
everything else is assumed to be homogeneous and hence can be represented by its associated<br />
texture when cut. The Unity 3D development environment, which uses the PhysX physics engine from<br />
NVIDIA, is used (Unity3D 2011). This gives complete programming control over mesh construction<br />
and texturing.<br />
2. Some 3D graphics basics<br />
Polygonal 3D models are generally built out of meshes that are a collection of triangles. Triangles<br />
consist of three vertices, three edges and a single surface called a face. The single surface means<br />
that a triangle can only be seen from one side. In general it is standard that if a triangle has had its<br />
vertices defined clockwise from the viewing angle, then it is visible, and from the opposite side it is<br />
invisible. Thus if you cut a sphere (built from triangles) in half and remove the half nearest you, you<br />
won’t see anything at all of the remaining hemisphere as all the faces will be orientated away from<br />
you and can only be seen from the far side. Triangles are the chosen primitive for 3D polygonal<br />
models because they have the property that all three vertices always lie in a single plane and<br />
graphics cards can be optimised to take advantage of this. Quadrilaterals or quads do not share this<br />
property. However, a quad can always be split into two triangles by adding a diagonal. Although it is<br />
the case that some modelling packages will let you develop in quads, there will always be a<br />
‘triangulation’ facility. This paper is only concerned with polygonal models built from triangles. Figure 1<br />
shows a polygonal model of a frog’s poison gland as shown within a perspective view in the Maya<br />
surface modelling application (Maya 2011).<br />
Figure 1: A frog’s poison gland as a smooth shaded polygonal model<br />
To understand the descriptions and algorithm a reasonable familiarity with vectors is needed. A good<br />
reference is [Spiegal 1974]. The first chapter introduces most if not all of what is needed. Note that in<br />
this paper vectors will nearly always be 3 vectors, that is 3D vectors with x, y and z components.<br />
435
Robert Lucas<br />
Occasionally we will also refer to 2 vectors that are used to map textures onto surfaces. In both cases<br />
vectors are represented by a bold letter, such as V1 in any formula or sentence.<br />
3. Basic data structures<br />
In the Unity (Unity 2011) world a mesh consists of an array of triangles. This is a list of indices into<br />
another array of vertices where each vertex is a 3-vector. This defines the position of the vertex in 3D<br />
space. Each consecutive three entries of the triangles array define a triangle. To make this tangible<br />
figure 2 shows how a cube is represented by twelve triangles where each triangle consists of three<br />
vertices. The ‘triangles’ array that corresponds to this contains: 3, 1, 2, 2, 1, 0, 5, 3, 4, 4, 3, 2, 7, 5, 6,<br />
6, 5, 4, 1, 7, 0, 0, 7, 6, 5, 7, 3, 3, 7, 1, 2, 0, 4, 4, 0, 6. In fact, it is not quite as simple as this as there is<br />
a good deal of redundancy in the vertices array with the same point being represented as many times<br />
as it appears in the definition of a triangle. The very first operation applied in our cutting process is to<br />
remove this redundancy to make the triangles array as defined above. This makes it very much easier<br />
to compare vertices as the vertex number in the triangles array uniquely defines a point in space.<br />
Thus if two entries in the triangles array are the same, then they must refer to the same point in 3D<br />
space.<br />
Figure 2: The vertices of a cube drawn as a mesh of triangles<br />
4. Developing a cutting algorithm<br />
Developing software for complex operations can sometimes be rather akin to the old joke: when you<br />
ask an Irishman the way, he replies ‘Well I wouldn’t want to start from here’. This is because the<br />
space the problem spans can just be too large to get to grips with. However much we like to pretend<br />
in the software world that all things can be achieved by a top-down design approach (Lucas 2000), in<br />
practice we commonly use a combination of top-down and bottom up design approaches, if for no<br />
other reason than it can prevent us going insane from the complexity of what we are trying to deal<br />
with. The bottom-up approach does allow us to start from ‘somewhere else’ which will indeed make it<br />
easier to get to our destination.<br />
4.1 Representing a planar cut as a data structure<br />
In this case it seems sensible to start with a definition of a cut as a data structure that will give us a<br />
handle on actually performing the cut. Then we can address the issue of how we might create such a<br />
cut data structure at a later time when we are entirely happy that our representation has made the<br />
problem tractable. This concentrates the mind wonderfully, what do we mean by a cut? What do we<br />
want to be the result of the cut? These questions come to mind as soon as we address the problem of<br />
representing the cut. For simplicity we will consider planar cuts. Elementary geometry tells us that a<br />
plane can be represented by a normal to the plane and a single point on the plane, so there is<br />
certainly no problem in representing that; two 3-vectors will suffice. If we look at Figure 3 and imagine<br />
a planar cut caused by a blade occupying a plane containing the x and y axes passing through the<br />
436
Robert Lucas<br />
origin at the centre of the cube, we can imagine the points of intersection of this plane with each<br />
affected triangle.<br />
Figure 3: Visualizing the cut points of a plane with a cube<br />
This gives us the necessary insight to define a cut-point as consisting of two vertices and a number<br />
representing how far from the first vertex toward the second is this cut-point. Thus a single cut point is<br />
represented by the triple:<br />
(VertexA, VertexB, lambda)<br />
Where the cut occurs on this edge between VertexA and VertexB at the point<br />
(VectorB-VectorA)* lambda<br />
Where VectorA is the vector from the origin to VertexA and VectorB is the vector from the origin to<br />
VertexB.<br />
Thus the entire planar cut can then be represented by the ordered list:<br />
{CutA, CutB,…,CutH}<br />
It is clearly the case that this representation can be used on any arbitrary complicated mesh, just as<br />
long as the mesh is composed of triangles. More importantly, we can now perceive how a cutting<br />
algorithm might proceed by taking each cut point and creating a vertex at the point of the cut and<br />
rebuilding each triangle that has a vertex on the edge containing the cut point. Finally we will need to<br />
fill in the missing faces that the cut has created. In pseudo code (Lucas 2000) the algorithm is:<br />
Begin<br />
CutPoint1 = Cut[0]<br />
VertexA = CreateVertex(CutPoint1)<br />
For each remaining cut point do<br />
CutPoint2 = Cuts[next];<br />
VertexB = CreateVertex(CutPoint[next])<br />
If both cut points share same first vertex then<br />
CreateTriangle(VertexA, VertexB, SharedVertex)<br />
Else<br />
CreateTriangle(VertexA, VertexB, CutPoint1.StartVertex)<br />
CreateTriangle(VertexA, CutPoint2.StartVertex, CutPoint1.StartVertex)<br />
Endif<br />
VertexA = VertexB<br />
Repeat<br />
End<br />
437
Robert Lucas<br />
Note the need to create two triangles where there isn’t a shared vertex between the pair of cuts. The<br />
algorithm needs to be extended to remember all the created vertices as these will be needed to fill the<br />
hole created by the cut.<br />
4.2 Filling the hole created by the cut<br />
This is a well-established problem with several solutions. It is commonly referred to as triangulating a<br />
polygon. The added vertices form a 2-D polygon. In the example of the cube, this polygon is simply<br />
defined by the set of 3-vectors given by the points A through H. For a solution see (Chazelle 1991) or<br />
(de Berg 2000) or perform a Google search on ‘triangulating a polygon’ for algorithms, comments and<br />
related problems and diversions, such as the Art Gallery Problem proposed by Victor Klee in 1973<br />
when he was a young mathematician at Montreal University. One thing to note is that, unless we want<br />
to create a lot of unwanted and useless triangles (i.e. they are not really triangles at all) we need to be<br />
careful about vertices being collinear, that is we don’t want to create a triangle from three vertices that<br />
are in a straight line. We can easily test for two vectors being collinear using the vector dot product,<br />
but just how we implement this in terms of how accurate we want this test to be can be a source of<br />
problems. Where the test is too stringent we won’t catch all the collinear vertices, where it is too lax,<br />
we will be leaving visible gaps in our filled face.<br />
4.3 Creating the cut data structure<br />
In order to create all the cut points we need to take every edge of the mesh and calculate the<br />
intersection of this edge with the plane that represents the blade performing the cut. Given two points<br />
represented by the vectors V1 and V2 and a plane represented by the normal N and a point on the<br />
plane given by the vector P. The point of the intersection is given by the vector (Bourke 1991):<br />
I = V1 + lambda (V2-V1)<br />
Where lambda is the scalar given by:<br />
lambda = N dot (P – V1) / N dot (V2 – V1)<br />
where dot is the vector dot product.<br />
Where lambda is between zero and one, then the intersection is actually on the line between the two<br />
points. This is exactly what we want to test for. So the algorithm for determining the cut points needs<br />
to take each edge of the original mesh and calculate the value of lambda and where this is between<br />
zero and one we have a cut point formed from the two vertices and the value for lambda. The cut<br />
points then need to be topologically sorted so that we deal with each adjacent edge containing a cut<br />
in succession to ensure that the triangles are created correctly.<br />
4.4 Texture mapping<br />
Textures are applied to a surface using what are commonly known as uv coordinates. These are<br />
defined for every vertex of every triangle. The uv coordinates are two dimensional and in Unity are<br />
represented by a 2-vector. These uv vectors dictate how the texture is to be mapped onto the triangle.<br />
They range from zero to one, where one would dictate the rightmost position on the texture (for u) or<br />
highest position on the texture (for v). Thus a particular patch of a texture can be mapped onto each<br />
triangle by manipulating the uv coordinates for each vertex.<br />
Being able to map a texture onto a surface is very important. This is what gives our model realism.<br />
We can also use photographs for our textures and as shown in [Lucas 2010] this can be of huge<br />
advantage in scientific simulations by giving extra authenticity to the model. With a cut surface the<br />
situation is complicated by the fact that this surface has been programmatically created to fill the hole<br />
left by the cut using some number of different shaped triangles. The texture must be mapped such<br />
that there is continuity of the texture over the edges of one triangle to the next. This is not as<br />
complicated as it may first appear. Consider the vertex marked A Figure 4 that is common to many<br />
triangles. We want there to be continuity in the textures appearance as it is mapped over all the<br />
triangles meeting at Vertex A.<br />
438
Robert Lucas<br />
Figure 4: A set of triangles filling a planar hole<br />
If we calculate the uv values for A based on the coordinates in space of A then clearly this will map<br />
identically for all the triangles that contain A (A has the same coordinates regardless of which triangle<br />
you pick!). This is true of any other vertex and as the mapping from one vertex to another is linear, will<br />
quite automatically get continuity at the edges as long as we calculate the uvs from the coordinates.<br />
To map the texture uniformly onto the surface we need to find the horizontal and vertical coordinate<br />
axes on the cut face. You can consider this to be like placing a sheet of graph paper onto the face.<br />
We already have the normal to the cut face as this is the same as the normal to the cutting plane. If<br />
we take the cross product of the normal and the ‘up’ vector we get the horizontal coordinate, and the<br />
cross product of the normal and the ‘right’ vector we get the vertical coordinate. The up and right<br />
vectors are defined for you in Unity. The algorithm is then (in pseudo code):<br />
Begin<br />
N = Cut plane normal<br />
H = N cross Right<br />
V = N cross Up<br />
For every triangle T do<br />
uv[1st Vertex] = Vector2(Vertex1 dot H, Vertex1 dot V)<br />
uv[2nd Vertex] = Vector2(Vertex2 dot H, Vetex2 dot V)<br />
uv[3 rd Vertex] = Vector2(Vertex3 dot H, Vertex3 dot V)<br />
Next<br />
End<br />
We can scale the texture in both directions by putting a multiplying factor in front of the Vector2<br />
constructor.<br />
We can choose to treat textures differently for different objects by separating the texture for the<br />
original surface from that intended for the cut surface. For example we can have one texture for the<br />
surface of an egg, the shell, and another for the interior white. This effectively makes the shell<br />
infinitely thin, but saves on the number of meshes and triangles needed and can be a reasonable<br />
compromise where memory is tight. The materials associated with the textures are being stored within<br />
the code as this makes it unnecessary for external references to be made at run-time. Figure 5 shows<br />
a cross section of a Malteaser. This is constructed from two polygon models of a sphere with twelve<br />
vertical and horizontal divisions. The outer sphere has a brown chocolate texture and the middle<br />
sphere is given a cut-face texture made from a photograph of the centre of an actual Malteaser.<br />
439
Robert Lucas<br />
Figure 5: A model of a Malteaser which has been cut to make it appear solid with a generated face<br />
and a cut face texture made from a photograph of the real thing<br />
This was good enough to fool my four and six year old into thinking it was a photograph of a real<br />
Malteaser and there is no doubting their expertise in this area! When it was explained that it was a 3D<br />
model and were shown it being rotated, one of them asked if it was possible to print it in such a way<br />
as they could eat it!<br />
Figure 6 shows the cross section of some frog skin which has been created as a model. This has<br />
been constructed from polygons representing the main skin volume, the poisonous glands (bulbous<br />
bottle shaped as given in figure 1) and the mucous glands (white).<br />
Figure 6: Cross section through a frog’s skin<br />
In this case all the textures were drawn from a diagram obtained from the internet (R Hamley 2011),<br />
but it should be clear that the textures could have been obtained from photographs. Clearly this view<br />
is at some higher level of magnification than normal. However, there is no reason why we cannot<br />
include magnified views in our 3D world. The virtual spectroscope described in (Lucas 2010) includes<br />
a telescope that is implemented using a software camera capable of yielding whatever magnification<br />
is required of whatever it is pointed at in the virtual world.<br />
5. Discussion and future development<br />
There is much that can be done with what has been implemented. It certainly felt at first quite strange<br />
to be able to cut through models that we know to be hollow and have them display internal structure<br />
as though they are solid. Effectively it is an illusion very much like a magic trick. However, any 3D<br />
graphics application is actually an illusion in any case! There is still some exploration to be done with<br />
this tool as it stands without further development as the results are quite dramatic, being both visually<br />
and intuitively appealing. We can consider dissecting objects that might be too difficult for standard<br />
tools, such as small insects and microscopic organisms. Considering performance, the array<br />
operations that are required to manipulate a mesh in real time have adequate performance for these<br />
type of cutting operations on complex meshes. The Unity Scripting Reference (Unity 2011) states that<br />
two million vertices can easily be handled within a second. As even complex models do not usually<br />
have more than a few hundred thousand vertices, we can see that we are well within the performance<br />
needed for real-time operation.<br />
440
Robert Lucas<br />
However, in a real dissection, cuts are not usually made in a single action but are the result of a<br />
number of incisions. Having successfully implemented the planar cutting algorithm with the associated<br />
texture mapping to the cut surface, it is now possible to see how incisions can be implemented. A<br />
knife blade can be represented as a 3D model and the collision between it and the object being cut as<br />
well as the geometry of the blade can be used to calculate the appearance of the incision. The<br />
incision can be implemented in much the same way as the cuts in the approach described above, but<br />
now we will want to add in some behaviour of the cut material, such as how much it pulls back from<br />
the blade to create a groove or valley in the cut surface. The sides of this groove can be constructed<br />
as per our cut surface above. Some experimentation with how a ‘peeling back’ type operation can be<br />
performed. We can identify a contact point and force from a dissection tool indicated by its movement.<br />
We can warp a mesh differentially by moving its vertex positions along the direction of the force from<br />
the tool whilst maintaining distances between adjacent vertices to maintain the surface integrity of a<br />
substance like skin. We can even introduce some stretching if we need to. We can propagate the<br />
force from the tool through the tissue being ‘peeled back’ so that points nearer the tool move the most<br />
whilst those distant from the tool behave as though anchored. With a suitable controlling device, there<br />
is no reason why we couldn’t offer some feedback to the user on the force required to perform an<br />
incision or operation to move or peel-back an object. At this point we will be well on the way to<br />
providing a realistic, if not total, dissection experience. This will be a very different experience from<br />
that offered by current dissection packages as there will be no constraints on where you can put the<br />
knife in, nor what you can put it into.<br />
6. Conclusions<br />
We cannot realistically expect to perform authentic dissections using 3D models in a games<br />
environment without putting in place some algorithmic and representational solutions to some of the<br />
many facets of the various processes that make up a dissection activity. This paper has shown how<br />
we can achieve planar cuts on surface models that make them appear to be solid by cutting the faces<br />
of the original object and replacing them with faces along the cut and filling the hole created by the<br />
cut. It has also been shown that textures can be added to the cut face that can span many triangular<br />
faces and present a coherent continuous texture that can be derived from a photograph to add extra<br />
realism. It is proposed that these techniques can be extended to provide partial cuts or incisions. It is<br />
asserted that these techniques take us a significant step towards performing virtual dissections using<br />
standard tools available to games programmers.<br />
References<br />
Bourke P (1991) Intersection of a point and a plane http://paulbourke.net/geometry/planeline/ (accessed on<br />
07/06/11).<br />
Chazelle, B (1991). "Triangulating a Simple Polygon in Linear Time." Disc. Comput. Geom. 6, 485-524, 1991.<br />
de Berg, M.; van Kreveld, M.; Overmans, M.; and Schwarzkopf, O. "Polygon Triangulation: Guarding an Art<br />
Gallery." Ch.3 in Computation Geometry: Algorithms and Applications, 2 nd rev. ed Berlin: Springer-Verlag,<br />
pp.45-61, 2000.<br />
Digital frog 2011 http://www.digitalfrog.com/ (accessed on 07/06/11).<br />
R Hamley 2011 “Comparing Frog and Human Skin”.<br />
http://slohs.slcusd.org/pages/teachers/rhamley/Biology/Frog%20Dissection/integumentary%20system.html<br />
(accessed on 07/06/11).<br />
Lucas R. (2000) “Programming Methodology”<br />
http://www.keylinkcomputers.co.uk/Tom/Programming/programming%20methodology.pdf (accessed on<br />
07/06/11).<br />
Lucas R J. (2009) “3-D Immersive Screen Experiments”. Proceedings GIREP 2009. , (Groupe International de<br />
Recherche sur l'Enseignement de la Physique) 17-21 August 2009 Leicester University. (ISBN 978-1-4461-<br />
6219-4).<br />
Lucas R J, Kolb U. (2009) “Use of 3-D Virtual Environments in Teaching Physics and Astronomy”. Proceedings of<br />
Frontiers In Science Education Research. Cyprus March 2009 Eastern Mediterranean University press<br />
2009.<br />
Lucas, R J. (2010). “Using a virtual world for teaching the motion of projectiles”. In Norton, Andrew ed. Electronic<br />
Resources for Teaching and Learning (In press). The Open University, pp. 58–64.<br />
Lucas, R J (2010). . “The Celestial E-Sphere”. In Norton, Andrew ed. Electronic Resources for Teaching and<br />
Learning (In press). The Open University, pp. 58–64.<br />
Maya (2011) http://usa.autodesk.com/maya/ (accessed on 07/06/11).<br />
Save the frogs (2011) http://www.savethefrogs.com/gifts/digital-frog.html (accessed on 07/06/11).<br />
Sketchup (2011) http://sketchup.google.com/ (accessed on 07/06/11).<br />
Spiegel M. R. 1974 “Theory and Problems of Vector Analysis” McGraw-Hill Book Company, New York.<br />
Unity 3D (2011) http://unity3d.com/ (accessed on 07/06/11).<br />
441
Learning by Wandering: Towards a Framework for<br />
Transformative eLearning<br />
Marie Martin and Michaela Noakes<br />
Duquesne University, Pittsburgh USA<br />
Martin684@duq.edu<br />
Noakes495@duq.edu<br />
Abstract: As technology continues to flatten the world and as Web 2.0 changes the way knowledge is created<br />
and shared, tertiary education institutions are turning increasingly to eLearning to extend access to students<br />
globally as well as to improve the quality of their <strong>learning</strong> experience. Learning Management Systems (LMS)<br />
currently dominate the delivery of eLearning at this level. Though these systems have extended functionality by<br />
including some Web 2.0 tools, they are generally perceived as a ‘walled garden’, essentially embodying the<br />
traditional transmission paradigm of teaching and <strong>learning</strong> rather than the philosophy of Web 2.0. This is leading,<br />
particularly in the blogosphere, to calls to break down the walls of the LMS and to explore more open online<br />
courses. There is, however, an emerging view that Web 2.0 ideals can be realised within an LMS environment,<br />
provided the environment is aligned with these ideals. This paper supports that view. It presents a case study of<br />
an eight-week eLearning course based on this premise, offered in spring 2011 as part of a doctoral programme in<br />
Instructional Technology by Duquesne University, Pittsburgh, USA, and designed and delivered within an LMS by<br />
an instructor living in Northern Ireland. The course is underpinned by the concept of <strong>learning</strong> by wandering. The<br />
pedagogy is aligned with the fundamental Web 2.0 philosophy. Within broad parameters, it is flexible, studentcentred<br />
and, from an early stage, student-led. Students are encouraged to use a variety of Web 2.0 tools,<br />
according to their preferences, to collaborate in preparation for their leadership role and as a language to express<br />
their ideas and to share their <strong>learning</strong>. The teacher’s role is identified as sage at the side. This case study is<br />
intended to contribute to the provision of a framework for transformative eLearning through fostering a Web 2.0<br />
ethos within a traditional <strong>learning</strong> environment.<br />
Keywords: <strong>learning</strong> management systems; Web 2.0 ethos; case study; <strong>learning</strong> by wandering; sage at the side;<br />
transformative eLearning<br />
1. Introduction<br />
As technology continues to flatten the world (Friedman, 2007), and as Web 2.0 changes the way<br />
knowledge is created and shared (Guth and Helm, 2010), tertiary education institutions are turning<br />
increasingly to eLearning to extend access to students globally as well as to improve the quality of the<br />
<strong>learning</strong> experience (Beetham and Sharpe, 2007). Learning Management Systems (LMS) currently<br />
dominate the delivery of eLearning at this level (Kyong-Lee and Bonk, 2006). Though these systems<br />
have extended functionality by including some Web 2.0 tools, they are still generally perceived as a<br />
“walled garden”, essentially embodying the traditional transmission paradigm of teaching and <strong>learning</strong><br />
rather than the philosophy of Web 2.0 (Katsifli, 2010; Lee and McLoughlin, 2011). This is leading,<br />
particularly in the blogosphere, to calls to explore ‘massive open online courses’ (Stein, 2008;<br />
Siemens, 2011). Huijser and Sanker (in Lee and McLoughlin, 2011: 267-283) argue, however, that<br />
Web 2.0 ideals can be achieved within an LMS, provided the environment is aligned with these ideals.<br />
This paper offers a descriptive case study of an eight-week eLearning course that endeavoured to<br />
create this environment and to bring the new Web 2.0 mindset to bear on the delivery of eLearning<br />
within a traditional LMS. The study is based on qualitative data derived from the following sources:<br />
notes made by the lead author as participant observer (Quinn Patton, 2002), archives of the<br />
synchronous weekly class, discussion board activity, student assignments, and written feedback from<br />
students. It aims to highlight the human dimension and provide an inside view of the eLearning<br />
experience. The focus was on two questions: How can this course, delivered within a traditional LMS,<br />
embody the Web 2.0 ethos? Does this process lead to a transformative eLearning experience for the<br />
students?<br />
The eight-week online course on Education in a Global Society was offered as part of a doctoral<br />
programme in Instructional Technology by Duquesne University, Pittsburgh, USA and designed and<br />
delivered by an instructor living in Northern Ireland (the lead author). The paper will outline the<br />
rationale, aims and objectives of the course, the demography of the doctoral cohort, the course<br />
design in the context of high expectations of delivery by eLearning, the pedagogy, the technologies<br />
used, and the lessons learned. It is hoped that this case study can contribute to a framework for<br />
transformative eLearning for both students and teachers and support the argument that the Web 2.0<br />
ethos can be realised within a traditional LMS environment.<br />
442
2. Rationale, aims and objectives<br />
Marie Martin and Michaela Noakes<br />
The rationale for the course was to address the need to raise the level of global awareness of<br />
students with regard to education in order to prepare them to function effectively as educators in a<br />
multicultural society and in a world without borders. The aim was to enable students to understand the<br />
socio-cultural context of education globally, beginning with their own as a basis for the comparative<br />
study of other systems. The objectives were to understand the history of educational ideas in the<br />
western world and evaluate these in the context of a global society; to compare and contrast<br />
American education with education systems in other selected countries; to assess the students’<br />
personal educational experience and philosophies, and to critique how their philosophies affect the<br />
roles of educators, students, and organizations.<br />
3. The doctoral cohort<br />
The doctoral cohort comprised eight graduate students, all with responsible positions in education or<br />
training, all with limited exposure to other cultures, and all with previous experience of eLearning. The<br />
level of digital literacy ranged from good to very high. By contrast, the level of digital literacy of the<br />
teacher was quite basic, a situation that led to unexpected and invaluable outcomes which will be<br />
discussed later in the paper.<br />
4. The course design<br />
The design was grounded in the firmly held view of the teacher/designer that eLearning could actually<br />
break down the barriers to <strong>learning</strong> encountered in the ‘limited and closed world of the traditional<br />
classroom’ (Martin, 2010: 75) and, with appropriate e-pedagogy, could ‘sustain a form of <strong>learning</strong> that<br />
is equivalent, if not superior, to that provided by traditional classroom settings’ (Kuriloff, 2005). It was<br />
also grounded in the belief that this form of <strong>learning</strong> could provide students with a challenging,<br />
enjoyable and transformative eLearning experience. The design was underpinned by the concept of<br />
<strong>learning</strong> by wandering – using technology, in accordance with one’s own way of <strong>learning</strong>, to embark in<br />
a spirit of ‘serious playfulness’ and with an ever-open mind on a largely uncharted voyage over the<br />
seas of cyberspace in an endless quest of other ways of knowing, thinking and being in the world<br />
(Martin, 2010: 85). This way of <strong>learning</strong> necessarily involves being willing to ‘travel tangentially … and<br />
to share with and learn from others’ (Martin, 2010: 24), and the course design sought to facilitate that<br />
approach. The sharing with and <strong>learning</strong> from others was to take place asynchronously in Blackboard<br />
and in real time in Wimba Classroom where weekly class sessions were to be held. Virtual visits were<br />
to be undertaken first as a class group to a small number of identified educationally high performing<br />
countries. Additionally, each student was given complete freedom to select a country - other than one<br />
already visited - for individual wandering and exploration. All such <strong>learning</strong> and reflections were to be<br />
shared. For this purpose, students were given the option of using Web 2.0 technologies.<br />
5. An appropriate e-pedagogy<br />
Mindful that many educators tend to regard ‘on ground’ classroom-based teaching as the optimal<br />
<strong>learning</strong> experience and therefore tend to see online <strong>learning</strong> simply as an ‘alternative delivery system<br />
for traditional pedagogy’ (Kuriloff, 2005), the teacher sought to create an appropriate e-pedagogy that<br />
would help release the transformative potential of eLearning. Foremost in the pedagogy was the<br />
establishment of a high quality relationship with the students. Hargreaves (2003) stresses that good<br />
teachers understand the importance of caring relationships and emotional engagement with <strong>learning</strong>.<br />
This applies particularly in eLearning because of the potential in this environment for personal and<br />
social isolation and disaffection (Croft, Allison and Duff, 2010). Additionally, within broad parameters,<br />
in accordance with the philosophy of <strong>learning</strong> by wandering, the e-pedagogy was flexible – to allow for<br />
some productive off-course wandering and reversal of roles. It was also aligned with the open,<br />
collaborative and relational mindset of Web 2.0 (Guth and Helm, 2010: 22). Blackboard and Wimba,<br />
and - in accordance with students’ preferences – Web 2.0 technologies were used as shared spaces<br />
for ‘collective intelligence’, and there was a strong focus on ‘participative and collaborative user<br />
experiences’ and on ‘dialogical conversations’ (Guth and Helm, 2010: 41). Specifically, the epedagogy<br />
was student-centred and, from an early stage, student-led, as students worked in pairs to<br />
lead a part of the group <strong>learning</strong> journey, as well as undertaking independent individual explorations<br />
to a country of their choice. A shared <strong>learning</strong> approach within a community of learners was fostered<br />
throughout the course to allow them to experience <strong>learning</strong> as a collaborative, social and enjoyable<br />
activity, inclusive of both students and teacher. The underpinning metaphor for this approach is<br />
Thornburg’s (2004) Campfires in Cyberspace, with the campfire as the more formal <strong>learning</strong> place<br />
where the elders (the teacher or the student leaders) tell the story and initiate discussion, and the<br />
443
Marie Martin and Michaela Noakes<br />
watering hole as the informal space where wanderers take turns to be storytellers and listeners and<br />
where peer <strong>learning</strong> takes place.<br />
6. Implementation<br />
In the context of maintaining a caring relationship in which each student felt that his/her progress<br />
mattered to the teacher, the latter made two major commitments. One was to offer students who were<br />
unsure about the direction of their assignments the opportunity to submit them as work-in-progress for<br />
monitoring by the teacher without prejudice to the final grade. This option was welcomed by the<br />
students and availed of responsibly - usually in the form of Google docs to facilitate pre-submission<br />
sharing and editing as the student deemed appropriate. The second commitment was to email<br />
individual formative feedback after each assignment. This was enormously appreciated by the<br />
students and increased their motivation to give of their best. One student’s reaction was typical: I<br />
appreciate that you personally email each of us after a project or assessment. ... I find it validates the<br />
hard work we put into our assignments. ... I understand it takes time, but it does mean so much to me<br />
that you send a personal email with strengths and weaknesses.<br />
The first form of <strong>learning</strong> by wandering to which the students were introduced was ‘time travel’ to<br />
enable them to follow in outline the story of educational ideas in the western world from ancient<br />
Greece to the Digital Age. This served as context for the ongoing exploration into the direction<br />
education should be taking in our global society. Another form of tangential wandering introduced at<br />
an early stage was ‘The Journey into Self’. This encouraged students to keep a reflective journal to<br />
monitor their own inner wandering and to consider whether this journey was transformative of them as<br />
learners.<br />
Travelling tangentially was encouraged throughout the course. Initially, this took the form of looking<br />
briefly at the theme of <strong>learning</strong> by wandering in myth and legend (Martin, 2010) as well as in ancient<br />
and more modern history and in literature. The students saw tangential travelling as a rich<br />
contributory source to their <strong>learning</strong> and felt comfortable with going off course to share readings and<br />
<strong>learning</strong> experiences from their ‘real life’. An example was the sharing at one point by a student<br />
discussion leader of an issue, which, though introduced under the subject title of ‘entirely unrelated’,<br />
evolved into a fecund sharing of experiences on the importance of the teacher in the <strong>learning</strong> process.<br />
7. Scaffolding students as leaders of <strong>learning</strong><br />
The strategy of having students lead the <strong>learning</strong> by putting two in charge of a particular section of the<br />
group wandering proved to be highly effective in terms of <strong>learning</strong> as well as being an enjoyable and<br />
potentially transformative experience for the leaders. Over the period of eight weeks, the class as a<br />
group virtually visited four countries selected from the course textbook with emphasis being placed on<br />
additional shared research. This meant that all students could take turns at being both teacher and<br />
learner – or storyteller and listener according to Thornburg’s campfire and watering hole metaphor.<br />
The teacher provided scaffolding for the leaders in a number of ways prior to their undertaking their<br />
roles. Simple guidelines for leading asynchronous discussion were made available. The teacher also<br />
drew up a content analysis model for online discussion, based on the model devised by<br />
Gunawardena, Lowe and Anderson (1997). The model illustrates how knowledge construction and<br />
negotiation of meaning online proceed through five phases, identifying at each stage the Process<br />
Variables (how learners negotiate and interact) and the Product Variables (content and outcomes of<br />
learner interactions). For convenience, the teacher’s version of this model is reproduced in table form<br />
in Table 1.<br />
This provided all students with a tool to enable them to analyse the content of the particular group<br />
discussion they were leading. It also helped them to self-evaluate their own contributions to<br />
discussions.<br />
Finally, the teacher held a short tutorial in the Wimba Classroom with each pair of leaders just prior to<br />
their undertaking the role. This was totally non-directive. Its purpose was simply to give the leaders a<br />
voice, to ensure they understood their mission and assure them of teacher intervention only when or if<br />
requested. They appreciated this freedom and empowerment, and without exception, rose to the<br />
challenge, using mainly wikis and Google docs to collaborate in preparation for their task. The weekly<br />
synchronous class was the forum where each pair of leaders prepared their peers for the next stage<br />
of the <strong>learning</strong> journey. They also took great care with the identification and formulation of the main<br />
discussion question to be followed up asynchronously in Blackboard, where they succeeded in<br />
444
Marie Martin and Michaela Noakes<br />
initiating and monitoring well-reasoned and stimulating posts, often enhanced by reference to<br />
websites researched by students and illustrated by videos from YouTube and other video-sharing<br />
websites.<br />
Table 1: Model for online discussion content analysis (based on Gunawardena, Lowe and Anderson,<br />
1997)<br />
Phase Process variables Product variables<br />
1<br />
Sharing and comparing<br />
information<br />
Statements, observations<br />
2 Discovery and exploration Questions, clarifications, elaborations<br />
3<br />
4<br />
Negotiation of meaning<br />
Co-construction of knowledge<br />
Testing ideas<br />
Revising ideas<br />
5 Awareness of new knowledge<br />
Joint meaning making<br />
Shared understanding<br />
Testing & revising against personal knowledge<br />
Metacognitive statements<br />
Reflections<br />
Summarising (to reflect consensus or diversity of perspectives)<br />
While the class travelled as a group to four selected countries, the teacher was engaged in individual<br />
exploration of other parts of the world. The weekly synchronous class was the ‘campfire’ around<br />
which the students gathered with the teacher as listeners to the tales of their peer leaders, before<br />
turning their attention to the teacher in the role of storyteller. In practical terms, the teacher’s input<br />
enriched the <strong>learning</strong> journey by increasing the number of countries the class could ‘visit’ in the short<br />
period of eight weeks. In terms of class dynamics, it subtly blurred the distinction between teacher<br />
and learner. This was to become a positive feature of the course and to lead to ‘a model in which the<br />
old teacher/student relationship [was] replaced by <strong>learning</strong> together’ (Papert, 1999).<br />
8. Teacher-as-learner<br />
The teacher-as-learner role became more pronounced when the class returned from their individual<br />
wandering. As indicated earlier, the teacher’s digital literacy was quite basic, but, aware of the<br />
students’ superior prowess in this domain, she encouraged them to work in whatever medium they felt<br />
they could best articulate their stories. She asked only that those who chose the digital option would<br />
provide her with a ‘dummy’s guide’ to their selected medium to help her create an assessment rubric.<br />
Half the class – four students – were in this category. The <strong>learning</strong> curve for the teacher was steep<br />
and exhilarating, as her mind and senses were drawn into stories told in the form of movies, a Glog,<br />
and a Prezi. In different ways, these digital formats vividly captured and imparted to teacher and<br />
students alike not just the factual, but the emotional reality of the social fabric and the education<br />
systems of the countries visited, illustrating that ‘technology is a language in which powerful ideas can<br />
be expressed’ (Papert, 1999) and multiple perspectives dramatically conveyed. Explaining his choice<br />
of medium for sharing his story of the ‘level of chaos for students and educators’ in a particularly<br />
troubled part of the world, one student wrote: No amount of data on politics, economics, or<br />
governmental oversight can ever adequately paint a picture of turmoil in areas currently afflicted by<br />
poverty, injustice, and violence, [so] I assembled a Glog which can be found below. The videos and<br />
songs paint a picture that, at least to my eyes, [is truer] of the daily world experienced by those under<br />
immense political and social pressures. From the teacher’s point of view, the discovery of the Glog<br />
with its apparently chaotic collage of videos was enlightening and invaluable. It seemed a near-perfect<br />
match of content and expression in that it captured visually the chaos which the student wanted to<br />
convey and which was authenticated by the scenes in each video and by the ‘insider view’ of the<br />
people caught up in the chaos. The fact that the videos could be viewed in any order without<br />
diminishing the impact also made the Glog a most appropriate form of expression for <strong>learning</strong> by<br />
wandering.<br />
445
Marie Martin and Michaela Noakes<br />
The student who used the Prezi explained her choice during a synchronous Wimba class by saying<br />
that, having bought into the freedom and the curvilinear approach of <strong>learning</strong> by wandering, she<br />
wanted some way of expressing herself in a non-linear, more impactful way than by the linear format<br />
of text or PowerPoint. A classmate with whom she discussed this introduced her - as part of our<br />
shared <strong>learning</strong> ethos - to the Prezi which is curvilinear in design, allowing the user to zoom as<br />
appropriate either on the big central picture in view on the screen or on the smaller pictures or texts<br />
surrounding it. This layout proved in effect to be a metaphor for her theme of the ‘mosaic’ of the<br />
country she was describing. The ‘dummy’s guide’ provided by one of the ‘movie makers’ gives an idea<br />
of the complex and time-consuming process required by her choice of medium: I used Google docs to<br />
create slides of relevant … information that I researched and read about both in books and online. I<br />
took screenshots as jpgs and inserted them into iMovie. Also looked for images that would enrich the<br />
movie. Then I worked on editing, cutting, cropping and animating everything. The last step was to<br />
research free Creative Commons podcast music and create background sound while editing for<br />
volume and ducking in and out. The same student remarked ruefully that it would have been so much<br />
simpler to write a paper, but she felt the movie was a more powerful form of expression. It should be<br />
stressed at this point that those students who did choose to write a paper also exploited their medium<br />
and enriched the <strong>learning</strong> of peers and of the teacher – albeit more within the latter’s comfort zone!<br />
Peer response to the digital stories was uniformly enthusiastic. The following is a typical example: I<br />
am so honoured to have such a talented cohort! You guys have given me so many great ideas for<br />
future projects. What a great way to teach kids about other cultures - instead of PowerPoints or<br />
lecture! Wow! These and other similar responses to the experience of sharing and <strong>learning</strong> with and<br />
from one another might perhaps give some indication of the strong element of what Guth and Helm<br />
(2010: 16) call the Web 2.0 philosophy, a ‘relationship revolution’ driven by ‘ideals such as sharing,<br />
openness, collective intelligence, flexibility and collaboration’.<br />
9. Teacher 2.0<br />
Dooley (in Guth and Helm, 2010: 277-303) speaks of ‘Teacher 2.0’ whose teaching approach is<br />
learner-centred, not technology-centred, who focuses on being able to use available technology as a<br />
means of collaboration and development of shared knowledge and of equipping students with the<br />
skills needed for professional life in today’s globalised world. She also depicts some of the traits of<br />
Teacher 2.0. Among these are a willingness to experiment and take risks, to integrate technologies<br />
that are more familiar to the learners than to the teacher, and effect a ‘symbiosis of truly collaborative<br />
<strong>learning</strong> ... by actively involving teachers in the <strong>learning</strong> process and students in the teaching<br />
process’. Teacher 2.0 also uses technology to provide opportunities for students to take ownership of<br />
the <strong>learning</strong> process. Above all, Teacher 2.0 makes the required shift from the transmission education<br />
paradigm to the ‘mutual sharing of knowledge-building between teacher-student, student-student and<br />
student-teacher’. Dooley asks bluntly: ‘Does Teacher 2.0 exist?’ The experiences of the lead author<br />
as teacher in the eLearning course which is the subject of this paper would suggest that it is only if<br />
this type of teacher does exist that eLearning can begin to reach its transformative potential.<br />
10. Teacher as sage at the side<br />
In moving away from transmission education in eLearning, the teacher has also to move away from<br />
the traditional role of being sole repository of <strong>learning</strong> and to seek a role that best empowers the elearner.<br />
This is frequently seen as moving from teacher to facilitator, from sage on the stage to guide<br />
at the side. The role adopted by the teacher in the case of the eLearning course described in these<br />
pages was rather that of sage at the side, setting the students free to learn by wandering and take<br />
ownership of their <strong>learning</strong>, scaffolding them when necessary, but always endeavouring to ‘bring<br />
wisdom, perspective and [experience] to the <strong>learning</strong>’ (Papert and Caperton, 1999). In this way, it<br />
becomes possible to move toward the goal that Hargreaves (2003) suggests is the ultimate<br />
achievement of the teacher - not to deliver <strong>learning</strong>, but to develop learners. This is exactly how the<br />
authors of this paper understand transformative <strong>learning</strong> – developing learners who love <strong>learning</strong> and<br />
for whom it becomes a lifelong commitment.<br />
11. Students as critical friends<br />
Students also responded as critical friends to the teacher’s request for suggestions for improving the<br />
course. In general, these took two forms. One, surprisingly, was a suggestion to do more <strong>learning</strong> by<br />
wandering along the ‘road less travelled by’– from Robert Frost’s poem which we had also made our<br />
own. One student felt that all would have benefitted from having ‘more room to wander’ as this might<br />
446
Marie Martin and Michaela Noakes<br />
have led to some interesting tangential discussion. The second suggestion, unsurprisingly, was to<br />
introduce other technologies, including Skype or videoconferencing. Ironically, the teacher of this<br />
course is a champion of the use of videoconferencing in education, has used this medium in previous<br />
eLearning courses, and had hoped to try Skype group conferencing with this cohort, but was<br />
frustrated by time and other logistics. She occasionally used the video feature in Wimba, but it<br />
functioned only on a one-to-one basis and tended to slow down the pace of the class. This will be<br />
more thoroughly explored for the next iteration of the course. The ideal would be, as the same student<br />
suggested, to seek videoconferencing opportunities with people from the target countries. Another<br />
very useful suggestion that will hopefully be acted upon was to ‘encourage more PBL (project-based<br />
<strong>learning</strong>) during the Wimba sessions, such as break-out activities, online whiteboard activities, etc’.<br />
As the course continues to evolve, it will obviously be important to incorporate the use of desktop<br />
sharing and to place more emphasis on integrating Web 2.0 technologies into the Blackboard<br />
management <strong>learning</strong> system. Ideas suggested by the co-author include screen capture video<br />
software for digital storytelling, desktop application sharing tools to enable future students to begin to<br />
build a timeless and virtual ‘Wandering Library’ to share their experiences as a teaching tool for those<br />
that follow them, and possibly Twitter to support community building among students and, if<br />
appropriate, to create unique user names to allow them reach out to the web in search of ‘virtual tour<br />
guides’ for them as they wander.<br />
12. Conclusion<br />
How did this course, delivered within a traditional LMS, endeavour to embody the Web 2.0 ethos? It<br />
sought to do so, firstly, by highlighting the importance of the teacher’s attitude and the impact of this<br />
on students (Katsifli, 2010). In particular, emphasis was placed on the course teacher’s high<br />
expectations of quality eLearning and awareness of importance of the human dimension. Secondly,<br />
the focus was on the need for appropriate pedagogy, leading to a paradigm shift from transmission<br />
teaching to student-centred and student-led <strong>learning</strong>. Additionally, the open mind aspect of the<br />
underpinning concept of <strong>learning</strong> by wandering was stressed in order to facilitate the necessary<br />
change of mindset. Finally, Blackboard and Wimba were used as shared <strong>learning</strong> spaces for<br />
collective intelligence, and creating space for students to use Web 2.0 tools for creative expression<br />
and sharing of <strong>learning</strong>.<br />
Did this process lead to a transformative eLearning experience for the students? Written feedback<br />
from them would suggest that something of this nature was experienced by at least a few. In an email<br />
to the teacher one student wrote: I thank you for being the kind of teacher that allows me to grow [not<br />
just] as a student but as an educator! As part of a Blackboard discussion about ‘teaching to the test’<br />
and the consequent lack of ownership of the <strong>learning</strong> process, a student’s post read: I think that is<br />
why [the] idea of wandering ... and student generated <strong>learning</strong> is so much more meaningful. And from<br />
the student who chose to present her final assignment as a “Jog”, a tool that enables multiple<br />
websites to be presented as a package: For the final reflective project I would like to put together a<br />
brief synopsis of <strong>learning</strong> from this class: A Jog on the Web (wandering). I would like to use this<br />
technology because it will help me in the future [to] refer back to some of the <strong>learning</strong> and contain the<br />
pertinent links to information we learned about. In the introduction to the ‘Jog’, she states: Like<br />
Ulysses, we as educators must not remain stationary, but wander and learn from the world around us.<br />
She also includes (with attribution) in her ‘Jog’ the wandering tales of some of her peers and she<br />
posted a note to them on the discussion board: Thanks to the class for sharing their <strong>learning</strong> this<br />
semester! ... It has been a blast! I am still adding to the <strong>learning</strong> from this course but here is the start<br />
of my Jog. Students also welcomed being given the ‘leeway to learn [and to share their <strong>learning</strong>] in<br />
the manner that [suits them]’ (personal email from a student). Another student wrote to express<br />
appreciation of the teacher’s ‘willingness to push the envelope with Web 2.0 technologies’. Above all,<br />
students valued the teacher as sage at the side. One student expressed this in her final email: I<br />
appreciate all of the wisdom and insight you were able to offer as you guided us through our journey.<br />
… It truly was a spectacular <strong>learning</strong> experience!<br />
The literature reflects the tension between two opposing views of the LMS. The first is the widely held<br />
view that, even with the addition of Web 2.0 tools, an LMS remains essentially an electronic<br />
replication of existing practice. The more recent view stresses the transformative potential of LMS<br />
mediated <strong>learning</strong> when it is informed by the Web 2.0 ethos. The eLearning course described in this<br />
paper supports the latter view. While very much a work-in-progress, it still serves to illustrate that the<br />
quality of eLearning is determined not by simply adding Web 2.0 tools to the traditional environment,<br />
but rather by embodying the Web 2.0 mindset within that environment. In this context, the case study<br />
447
Marie Martin and Michaela Noakes<br />
is offered tentatively as a contribution to the development of a framework for transformative<br />
eLearning.<br />
References<br />
Beetham, H. and Sharpe, R. (2007) Rethinking pedagogy for a digital age: designing and delivering eLearning,<br />
London and New York: Routledge.<br />
Croft, N., Allison, D. and Duff, M. (2010) Overcoming isolation in distance <strong>learning</strong>: Building a <strong>learning</strong> community<br />
through time and space, [Online], Available: http://ctiweb.cf.ac.uk/jebe/pdf/NicholasCroft5(1).pdf<br />
Friedman, T. L. (2007) The world is flat, New York: Picador/Farrar, Straus and Giroux.<br />
Gunawardena, C., Lowe, C. and Anderson, T. (1997) ‘Analysis of a global online debate and the development of<br />
an interaction analysis model for examining social construction of knowledge in computer conferencing’,<br />
Journal of Educational Computing Research, vol. 17, no. 4, pp. 395 – 429.<br />
Guth, S.and Helm, F. (eds.) (2010) Telecollaboration 2.0: Languages, literacies and intercultural <strong>learning</strong> in the<br />
21 st century, Bern: Peter Lang.<br />
Hargreaves, A. (2003) Teaching in the knowledge society: Education in the age of insecurity, New York:<br />
Teachers College Press.<br />
Katsifli, D. (2010). The impact of Blackboard software on education globally over the past 10 years. [Online],<br />
Available:<br />
http://www.lms.unimelb.edu.au/elo/resources/The_impact_of_Blackboard_software_on_education_globally_<br />
(20100204W).pdf<br />
Kuriloff, P. (2005) ‘ Breaking the barriers of time and space: More effective teaching using e-pedagogy’, Innovate<br />
[Electronic], vol.2, no. 1, Available: http://www.innovateonline.info/<br />
Kyong-Jee, K. and Bonk, C. J. ( 2006) [Online]. ‘The future of online teaching and <strong>learning</strong> in higher education:<br />
The survey says ...’ Education Quarterly [Electronic], vol. 29, no. 4, Available:<br />
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/TheFutureofOnline<br />
TeachingandLe/157426<br />
Lee, J. W. and McLouglin, C. (eds.) (2011) Web 2.0-based eLearning: applying social informatics for tertiary<br />
teaching, Hershey, New York: Information Science Reference.<br />
Martin, M. (2010) Learning by wandering: An ancient Irish perspective for a digital world, Bern: Peter Lang.<br />
Papert, S. and Caperton, G. (1999) Vision for education: The Caperton-Papert platform [Online], Available:<br />
http://www.papert.org/articles/Vision_for_education.html<br />
Papert, S. (1999) Technology in schools: To support the system or render it obsolete, Milken Family Foundation,<br />
[Online], Available: http://www.mff.org/edtech/article.taf?_function=detail&Content_uid1.<br />
Quinn Patton, M. (2002.) Qualitative research and evaluation methods, 3 rd edition, Thousand Oaks, London, New<br />
Delhi: Sage Publications.<br />
Siemens, G. ( 2011) Re: Massive open online courses, blog ( http://www.elearnspace.org/blog/), 4 Aug.<br />
Stein, J. (2008) Re: Massive open online courses, blog ( http://jaredstein.org/2008/02/29/lmss-ples-walledgardens-and-yearnings-for-debate/),<br />
29 Feb.<br />
Thornburg, D. D. (2004) ‘Campfires in cyberspace: Primordial metaphors for <strong>learning</strong> in the 21 st century’,<br />
International Journal of Instructional Technology and Distance Learning, vol. 1, no. 10, October, [Electronic],<br />
Available: http://www.itdl.org/Journal/Oct_04/invited01.htm.<br />
448
Online Student Engagement: Unfulfilled Promises or<br />
Promises Unfulfilled?<br />
Linda Martin, Gary Spolander, Imran Ali and Beulah Maas<br />
Coventry University, UK<br />
hsx185@coventry.ac.uk<br />
aa2912@coventry.ac.uk<br />
aa2446@coventry.ac.uk<br />
aa4895@coventry.ac.uk<br />
Abstract: Engagement is often seen as a reliable proxy for <strong>learning</strong> (Coates, 2005) but also in general abilities<br />
and critical thinking (Gellin, 2003), student satisfaction (Kuh et al, 2007), cognitive development (Pascarella &<br />
Terenzini, 2005) and improved grades (Tross et al, 2000). Whilst Coventry University have made significant<br />
efforts to support early engagement and promote good technology support, experience indicates that some<br />
students of on line programmes continue to experience "engagement challenges". These include choosing to<br />
study in isolation; students maximizing offered flexibility of eLearning to the detriment of synchronous<br />
collaboration, accessing <strong>learning</strong> materials only shortly before the submission of assessed work is due and as a<br />
result failing to move from surface level to in-depth approaches of cognitive processing (Henri, 1992). As a result<br />
what was intended as a dynamic, interactive medium for shared <strong>learning</strong> can become a pragmatic approach to<br />
individualistic achievement. In response to these concerns on a distance based online Health and Social Care<br />
degree, a project was launched to explore the impact of collaborative peer assessment and whether this would<br />
result in greater motivation to develop and share approaches to <strong>learning</strong>. Would developing this increased<br />
"critical space" (Jankowska & Atlay, 2008) result in an improved level of academic work being presented for<br />
assessment?<br />
Keywords: on line; peer assessment; collaborative; student engagement; shared <strong>learning</strong><br />
1. Introduction<br />
ELearning presents a dynamic approach to study which has opened a multitude of flexible<br />
opportunities for students not previously experienced. Innovative, creative resources can contribute to<br />
the development of the knowledge base within the student group through more responsive<br />
participative methods, especially when compared to the large and small group discussions so<br />
frequently used in traditional 'classroom' teaching in higher education. ELearning presents a panacea<br />
to the challenges of flexible <strong>learning</strong>, and increased opportunity for student engagement experienced<br />
through offering both synchronous and asynchronous opportunities to communicate verbally, visually<br />
and in writing within <strong>learning</strong> communities. The flexibility of eLearning enables lecturers to respond to<br />
the individual <strong>learning</strong> needs of students.<br />
2. Developing communities of <strong>learning</strong><br />
At Coventry University, the Foundation, BA and Master’s degrees in Social and Health Care<br />
Management were launched three years ago as wholly online programmes to offer increased<br />
flexibility and innovative course delivery. In this time, significant differences and concerns emerged in<br />
the responses from students on these three programmes. Notably the Foundation degree students<br />
demonstrated lower levels of engagement and academic <strong>learning</strong>. Further analysis by the course<br />
team suggested that this was influenced by a complex array of factors linked to developments in the<br />
social care and health sector, HEI sector responses to government initiatives and challenges and the<br />
individual characteristics of students. In response the course team developed a pilot peer assessment<br />
process as a means of promoting engagement and with a view to developing a better understanding<br />
of academic expectations for the Foundation degree students. This paper outlines this process, by<br />
exploring the rationale, approach taken and the initial results of the pilot. This is a work in progress<br />
and further development is ongoing.<br />
There have been considerable changes in recent years in the social care sector, with the increased<br />
use of independent sector organizations to deliver care services. Although the need for leadership<br />
and management training is undisputed in the sector, it pays low wages (Low Pay Commission,<br />
2011), staffing levels are often stretched (Mansell, 2011) and there is little incentive for senior<br />
managers to support staff in acquiring professional training. In the health sector, the picture is slightly<br />
different with many staff in the position of needing to achieve degree status in order to be eligible for<br />
promotion in the National Health Service (NHS). The level of self motivation is therefore variable<br />
449
Linda Martin et al.<br />
across the sector. Higher education has responded to this situation and government incentives by<br />
promoting Foundation degrees which enable workers with few prior qualifications to access a degree<br />
programme and complete it within two years of full time study. Sector requirements called for flexible<br />
education that took into consideration location, speed of study and mode of delivery (Skills for Care,<br />
2008).<br />
eLearning was seen as providing a range of benefits to learners including; contributing to self<br />
regulation (Salovaara cited in Kelly et al 2010), a key management skill and 'collaborative interaction<br />
between learners' (Jonassen et al cited in Kelly et al, 2010), while Laird and Kuh (2005) found a<br />
positive link between online provision, active and collaborative <strong>learning</strong> and deep <strong>learning</strong>. Indeed,<br />
Harper and Quaye (2009) highlight that educational engagement requires students to both participate<br />
and be involved which includes being active, feeling and making sense of materials. This behavioral,<br />
emotional and cognitive engagement (Fredericks et al, 2004) also requires supportive <strong>learning</strong><br />
communities to develop active and collaborative <strong>learning</strong>, academic activity collaboration, formative<br />
<strong>learning</strong> and a feeling of being supported by those <strong>learning</strong> communities (Coates, 2007). The<br />
opportunities afforded by the Foundation degree appeared to offer students a range of benefits as<br />
highlighted by sector workforce plans.<br />
3. The <strong>learning</strong> challenge<br />
The three years of online course delivery and evaluation support the benefits of eLearning, but a<br />
chasm developed between the approach to <strong>learning</strong> of the undergraduate and post graduate<br />
students. The teams experience was that once post graduate students had overcome initial anxieties<br />
regarding accessing the virtual <strong>learning</strong> environment, their confidence increased quickly to enable<br />
them to develop communication and support mechanisms within their <strong>learning</strong> group, but this was not<br />
the case at undergraduate level. . Rather students reported and appeared to be working individually,<br />
at very different paces, engaging only with the lecturer in spite of encouragement to participate in<br />
<strong>learning</strong> community discussion forums and exercises in which individual responses were shared with<br />
other group members. This appeared to result from differences within the two student groups with the<br />
undergraduates being less experienced managers, having fewer previous educational achievements<br />
and receiving less support from their employers to study.<br />
This pattern of study was not envisaged by the course team during development and whilst it may<br />
have been accepted as an inevitability of the students' work commitments, had they been achieving<br />
educational success in their studies, the academic attainment was a concern, which unveiled the<br />
possibility of students compounding previous negative <strong>learning</strong> experiences through non<br />
achievement. The course had been designed to offer academic support through the provision of a<br />
study skills module, digital provision of all essential reading materials and individual tutorial support.<br />
Despite this a significant numbers of assignments were written anecdotally, showed little evidence of<br />
academic reading and failed to comply with academic requirements. Students were working to<br />
deadlines, accessing course materials only shortly before the submission of assignments was due, so<br />
failing to move from surface to deep <strong>learning</strong> (Henri,1992) and conforming to Salmon's (2000) 5 stage<br />
model of <strong>learning</strong> whereby student failure to access the technology results in failure to learn.<br />
4. Encouraging engagement through peer assessment<br />
The importance of engagement in student <strong>learning</strong> has gained momentum as universities have found<br />
themselves in an increasingly marketised and competitive environment. The increased use of quality<br />
assurance by universities and the emphasis on demonstrating added value have been instrumental in<br />
supporting efforts to encourage student engagement and academic success. The literature frequently<br />
highlights that engagement is considered a proxy for <strong>learning</strong> (Coates, 2005) but also in general<br />
abilities and critical thinking (Gellin, 2003), student satisfaction (Kuh et al, 2007), cognitive<br />
development (Pascarella and Terenzini, 2005) and improved grades (Tross et al, 2000, Kuh &<br />
Vesper, 1997). This highlighted the importance of needing to address the lack of student engagement<br />
and poor academic achievement consistently.<br />
Whilst the Foundation degree could theoretically be completed in two years full-time, all students are<br />
part-time due to the requirement that they are also in appropriate employment. The course supports<br />
the development of knowledge essential to professional practice and its application within the<br />
workplace and as a result a key underpinning principle is the commitment to <strong>learning</strong> collaboration,<br />
the sharing of <strong>learning</strong> from diverse work settings and reflection on practice. Both synchronous and<br />
asynchronous <strong>learning</strong> objects were developed to provide opportunities for the group to develop<br />
450
Linda Martin et al.<br />
“critical space” (Jankowska and Atlay, 2008), to work collaboratively with less tutor input and<br />
ultimately to develop independent <strong>learning</strong> skills, which would enable reflection, critical thinking, and<br />
experimentation as students embraced deep <strong>learning</strong> (Gellin, 2003; Henri, 1992, Shulman, 2002). All<br />
of this can improve grades (Tross et al, 2000), but only if students engage (Fredericks et al, 2004).<br />
Numerous models of engagement were reviewed including the assessment to online contributions,<br />
but peer assessment was identified as offering potential for students to engage more critically and in<br />
more detail with the process of <strong>learning</strong>. Defined by Lindblom-Ylanne (2006:52), as learners<br />
assessing 'the achievement, <strong>learning</strong> outcomes or performance of their fellow students' it offered a<br />
range of potential benefits for the learner. Participation encourages the development of skills in giving<br />
and receiving feedback which are an essential aspect of professional responsibility, judgment and<br />
autonomy (ibid). Additionally the process of formative feedback supports disciplinary understanding,<br />
encourages increased ownership of academic work, encourages understanding of <strong>learning</strong> outcomes<br />
and facilitates student autonomy (Sadler, 2010). Brew (1999) outlines that these opportunities<br />
therefore encourage deep <strong>learning</strong>. Falchikov (2005) (cited in Nicol, 2010: 509) argues that peer<br />
feedback was often considered more helpful than that of a tutor alone, whilst others (Race, 2001,<br />
Coleman, 2006) suggest that peer assessment enhances assessment outcomes. In particular, the<br />
team felt it would help the students engage more actively with the process of writing so that students<br />
are required to engage and share their understanding of concepts, scholarly identity and the ways<br />
that they express their ideas (Bass, 2010, Meyer et al, 2009). Peer assessment, it was felt, offered<br />
fresh perspectives to <strong>learning</strong> communities through sharing how others approach the same topic,<br />
promoting reflexivity by broadening and deepening students’ understanding of the subject and<br />
academic rigour (Yang, 2010) as well as engaging learners in an activity which resembles<br />
professional practice (van der Pol, 2008). From their research, Ballantyne, Hughes and Mylones (in<br />
Vickerman, 2009) report that students found the experience of assessing other students' work<br />
motivating as well as aiding knowledge development of subject content and increasing their<br />
understanding of the assessment process.<br />
Peer assessment is not without critics, Par and Joordens (2008: 528) raise the issue of the quality of<br />
peer reviewers comments and the related justification for 'peer-derived' grades while Brown, Rust and<br />
Gibbs (1994) and Hartley (1998, in Vickerman, 2009) highlight that judging boundaries can be<br />
challenging for staff, let alone learners who are not usually focused on assessing each others' work.<br />
Further Par and Joordens (2008) argue that the choice of assessment tool is important, a point<br />
supported by Fowler and Mayer (in Kelly et al, 2010), who recognise that while a range of<br />
pedagogical approaches can be adopted to support on line <strong>learning</strong>, appropriate selection of 'internet<br />
tools' based on contextual factors is important. Factors such as the anticipated <strong>learning</strong> outcomes and<br />
the role of the teacher and learner, i.e. locus of control, all contribute to student <strong>learning</strong>, performance<br />
and satisfaction. There is considerable concern about the use of peer assessment in summative<br />
assessment; however, research has demonstrated its reliability and validity (DeNisi & Stevens, 1981;<br />
Reilly & Chao, 1982; Bamberger et al, 2005) although this debate continues (Cho and Schunn, 2003).<br />
The decision to undertake summative peer assessment was based on concerns that students may<br />
not engage with formative peer feedback and the importance of the process encouraging improved<br />
course engagement. Two assessments were developed for each module, the first one being peer<br />
assessed and representing 40% of the module mark and the second one being assessed by the<br />
module tutor, representing 60% of the module mark. Students needed to provide feedback on the<br />
assessment and discuss this within a small group before allocating a mark. The mark was then<br />
moderated by the module leader. Due to small cohort numbers, it is too early to tell whether the<br />
change has impacted on student achievement. The tool selected for the assessment process is<br />
PeerMark, a facility of Turnitin, a plagiarism detection programme through which students regularly<br />
submit their work. Familiarity was the key determining factor. To date the process has been used in<br />
three modules involving students at different stages of their degree.<br />
5. Student perceptions<br />
Qualitative data was collected by questionnaire from the students who have been involved in the pilot.<br />
Six out of a possible thirteen students responded to the opportunity to give feedback. They were<br />
asked to comment on their memories of peer assessment, the challenges, the benefits and any<br />
contribution the exercise has made to their <strong>learning</strong>. It is interesting that although a range of positive<br />
comments were made about the project; the responses to the first question, about their memory of it<br />
were negative. The overriding feelings that have stayed with them are unease, discomfort and lack of<br />
451
Linda Martin et al.<br />
confidence. Although these feelings became more positive as the pilot progressed it is their anxieties<br />
which they recall immediately. Key issues identified by the students that responded can be<br />
summarized as follows:<br />
The challenges for them indicate that the pilot has encouraged them to reflect more on what<br />
contributes to a good piece of work. It was noted that it was easier to be critical of others than it<br />
was to identify the good points which they had to consciously think about.<br />
A significant point was also made regarding their own experience at the time of their involvement<br />
in the pilot. For some this was the first piece of work they had written. They had no indication of<br />
whether their own work was of a satisfactory standard or whether they had understood the<br />
question. This added to the feelings of anxiety.<br />
The benefits noted related to increased engagement. One student commented that prior to the<br />
pilot comments put on the discussion forums had received no response from others, so it<br />
provided an opportunity to communicate. The pilot also provided the opportunity for some group<br />
<strong>learning</strong> as the final mark had to be agreed by several students working together.<br />
Additionally, the exercise contributed to the students' understanding of the difference between a<br />
good and bad essay by being able to read the work of others. However, they did not feel that the<br />
feedback received from other students about their own work helped their <strong>learning</strong> as the<br />
comments tended to be short.<br />
For some, it has helped them to be more critical of their own work and has increased their<br />
confidence. However, part of this increase is due to feeling that their own work was better than<br />
the assignments they marked.<br />
6. Staff perceptions<br />
The three staff involved in the pilot were asked to comment on the challenges, the benefits and the<br />
lessons learned. The feedback can be summarized as follows:<br />
For staff implementing peer assessment there have been a series of process issues which have<br />
required an immediate response before being able to reflect on whether the exercise was<br />
achieving its aim. The technology was challenging. Although all the team were familiar with<br />
Turnitin and can use this with ease, the Peer Mark option proved to be much more challenging. In<br />
addition to the added time spent there were ongoing concerns about whether it would work as<br />
intended, providing a smooth efficient submission system which protected the identity of students.<br />
The student response during the pilot was disappointing. Some students had to be constantly<br />
reminded and guided through the process whilst others, who were waiting for them to complete<br />
their marking, had to be appeased. One refused to take part and another went on holiday without<br />
completing the process. This led to an unequal distribution of work and adjustments were made to<br />
the marking criteria on the next occasion to reflect this and encourage participation.<br />
Attempts were made to involve students at a very early stage of the process, determining the<br />
marking criteria, but only one student contributed to this while the others waited to be told what to<br />
do.<br />
Although the process issues created some negative responses, the results of the exercise were more<br />
encouraging. The marks suggested by students were usually closely related to those the lecturer<br />
would have given, although there was greater coherence in relation to the final agreed group mark<br />
rather than the individually suggested student marks. The agreed process was that the lecturer would<br />
only change marks given by the students where there was significant concern about the mark given,<br />
but this was not defined with exactitude. For level 1 student’s precise marks are of less significance<br />
as they do not contribute to the degree classification but for levels 2 and 3 the significance increases<br />
and students question their given mark more and expect the mark to be clearly justified.<br />
7. Discussion<br />
The pilot is currently being reviewed to determine how it will proceed into its second year but a<br />
number of issues have been raised by these preliminary results. The initial purpose of the pilot was to<br />
increase student engagement and as a result increase academic achievement. To date the<br />
experience has called to question whether engagement and achievement are linked, whether there<br />
are other more effective ways of improving student achievement, and whether engagement is a<br />
shared utopia between students and lecturers or only a vision of lecturers.<br />
452
Linda Martin et al.<br />
While it is commonly argued that negative experiences have a greater impact than positive ones<br />
(Normann,2000) it is important to address the overarching anxiety, unease, discomfort and lack of<br />
confidence remembered by the students. These feelings threaten to outweigh and override the more<br />
positive experiences stated by students as the pilot progressed. New challenges by their nature<br />
threaten existing comfort zones but confidence in one's ability to meet the challenge is vital to<br />
success. Failure will only compound one's lack of confidence and feelings of inadequacy. A pilot of<br />
this nature has the propensity to discourage students from further participation on the programme and<br />
a as result limit the potential to deepen their <strong>learning</strong>. For inexperienced students and for those<br />
already failing, further consideration needs to be given as to whether the benefits promoted from<br />
previous research of increased self confidence and motivation (Race, 1998) and enhanced<br />
assessment outcomes (Race 2001, Coleman 2006) are likely to be realised through this process.<br />
Critically, there are or can be significant differences between contrasting types of course presentation<br />
i.e. on line, distance <strong>learning</strong> or on campus programmes. This programme seeks to market itself on<br />
the strength of opportunities for interaction and shared <strong>learning</strong> that derive from online course<br />
delivery. Students initially value this but a common reason given by students who withdraw from the<br />
programme is isolation. As a result interaction and engagement may be processes they expect other<br />
students to provide, but of which they are passive recipients. Peer assessment is underpinned by the<br />
principle that you have to give on order to receive. However, this may be a principle that today's<br />
students are struggling to give priority to. Krause (2005) found that part-time students, who are also in<br />
paid employment, were less engaged than full-time students with more than half of part-time students<br />
stating that their paid work interfered with their studies. Rather than seeing a need to negotiate with<br />
their employers or make adjustments to their commitments, they expected study to fit around their<br />
lives. McInnes (2003) suggests that engagement can no longer be taken for granted as a shared goal,<br />
it needs to be negotiated with students and Yorke (2006) highlights the need to differentiate between<br />
performance and <strong>learning</strong> goals. For some students this has been a confidence-boosting exercise,<br />
but this seems to have resulted from a sense of their own work being of a better quality than that<br />
which they have marked, rather than from <strong>learning</strong> from each others' work and feeling able to write a<br />
better assignment as a result. It appears that whilst tutors continue to be dedicated to students'<br />
<strong>learning</strong>, students may be more focused on their individual achievement. While research suggests a<br />
close link between engagement and achievement, this is based on an understanding of achievement<br />
resulting from a process of <strong>learning</strong> rather than completion of a task. Without a commitment to<br />
<strong>learning</strong>, students are unlikely to move from surface to deep <strong>learning</strong>, focusing narrowly on successful<br />
task completion, and may therefore only gain more surface level understanding of the subject. They<br />
may be extremely committed to complete each task but, reflecting the world in which they work, are<br />
outcome rather than process orientated and fail to see the benefit of tasks which do not appear to<br />
have a direct correlation with the achievement of their immediate goal. Peer assessment is an attempt<br />
to re-engage them with the process of <strong>learning</strong>, but there may be more fundamental issues which<br />
need to be addressed with students before they value the potential benefits rather than being taskfocused<br />
alone. Zepke and Leach suggest that higher education needs to adapt to the changing<br />
expectations of students by negotiating "a strategic approach to <strong>learning</strong> in which students choose<br />
which approach to adopt in any given situation" (2010: 173). The implication of this is that while peer<br />
assessment may be a tool that can be used to exercise engagement, it may only be effective when<br />
students have first embarked upon a commitment to <strong>learning</strong> and see peer assessment as a positive<br />
part of that process. The issue in question then is not their understanding of <strong>learning</strong> requirements but<br />
their motivation to learn. So the challenge for tutors is how to incentivise approaches to <strong>learning</strong>. This<br />
needs to be addressed at point of entry onto a programme, prior to students embarking upon<br />
assessed work, for without this commitment students are unlikely to reflect on the peer assessment<br />
process as a <strong>learning</strong> experience and will judge it only as a task. The dilemma is whether<br />
inexperienced students who are operating in a very different environment in the workplace can<br />
continuously shift between two such contrasting sets of expectations.<br />
For part-time students in employment it can no longer be assumed that they are engaged with a<br />
concept of <strong>learning</strong> in its own right. It may have become inextricably linked with achievement and<br />
career progression to the extent that they measure their <strong>learning</strong> through their employment status. It<br />
is with this in mind that the course team will progress to meet the challenge of extricating the two with<br />
a view to introducing students to the joys of <strong>learning</strong> and the fulfillment that is achieved through the<br />
acquisition of new knowledge and the development of wisdom.<br />
453
8. Conclusion<br />
Linda Martin et al.<br />
The pilot project aimed to developing greater engagement and as a result enhance student academic<br />
achievement. Early findings suggest that where students have engaged with the process they have<br />
enjoyed and benefitted from the interaction. There are also indications that their confidence has<br />
increased and they have appreciated being able to see the work of other students as a means of<br />
locating the quality of their own work.<br />
The project will continue into a second phase but will only be used with level 1 students in order to<br />
address concerns that unreliability of marking may impact on a student’s overall degree classification.<br />
It will also only be used in two of the four possible modules at level 1 so the students can focus on the<br />
quality of their assessments rather than it becoming an administrative burden. Other peer assessment<br />
tools will be considered for use and formative self assessment will be introduced at level two to<br />
observe whether students have developed skills which are beneficial to their own <strong>learning</strong> and<br />
development.<br />
The pilot has provided a valuable insight into how students approach <strong>learning</strong> and assessment which<br />
has wider implications for programme development and as the pilot continues, further consideration<br />
will be given to the relationship between work and study which underpins Foundation Degrees.<br />
References<br />
Bamberger, P.A., Erev, I., Kimmel, M. & Oref-Chen, T. (2005) Peer assessment, individual performance and<br />
contribution to group processes: the impact of rater anonymity, Group and Organization Management, 30,<br />
(4): 344-377<br />
Bass, R. (2010) The rise of social <strong>learning</strong> as a bridge to expertise presented at The Pedagogical Research in<br />
Higher Education Conference, Liverpool Hope University, 25 th October.<br />
Brew, A. (1999) Towards autonomous assessment: using self-assessment and peer assessment, in S. Brown<br />
and A. Glasner (eds) Assessment Matters in Higher Education: choosing and using diverse approaches,<br />
Buckingham: Society for Research into Higher Education & Open University Press.<br />
Cho, K. & C. D. Schunn (2003) Validity and Reliability of Peer Assessments with a Missing Data Estimation<br />
Technique, Proceedings of ED-Media, Honolulu, Hawaii, USA.<br />
Coates, H (2005) The value of student engagement for higher education quality assurance, Quality in Higher<br />
Education, 11 (1): 25-36.<br />
Coates, H. (2007) A model of online and general campus based student engagement, Assessment and<br />
Evaluation in Higher Education, 32 (2): 121-141.<br />
Coleman, I (2006) A Study using Formative Assessment Feedback in Physiology and Pharmacology to<br />
Encourage Engagement (see Case Studies section of www.open.ac.uk/fast)<br />
DeNisi, A.S. & Stevens, G. (1991) Profiles of performance, performance evaluation and personnel decisions,<br />
Academy of Management Review, 24: 592-602.<br />
Fredericks, J. A. Blumenfeld, P. C. and Paris, A. H. (2004) School engagement: potential of the concept, state of<br />
the evidence, Review of Educational Research, 74 (1): 59-109.<br />
Gellin, A. (2003) The effect of undergraduate student involvement on critical thinking: A Meta- Analysis of the<br />
Literature, 1991 – 2000, Journal of College Student Development, 44 (6): 746-762.<br />
Harper, S. R. & Quaye, S. J. (2009) Student Engagement in Higher Education, London, Routledge.<br />
Henri, F. (1992) Computer conferencing and content analysis, in A. R. Kaye, Collaborative Learning Through<br />
Computer Conferencing, Berlin, Springer-Verlag.<br />
Jankowska, M. & Atlay, M. (2008) Use of creative space in enhancing students’ engagement, Innovations in<br />
Education and Teaching International, 45 (3): 271-279.<br />
Kelly, D., Baxter,J. & Anderson, A. (2010), Engaging first-year students through online collaborative<br />
assessments, Journal of Computer Assisted Learning, 26: 535–548.<br />
Krause, K-L. (2005) Engaged, inert or otherwise occupied? Deconstructing the 21 st century undergraduate<br />
student, Keynote paper presented at the Sharing Scholarship in Learning and Teaching: Engaging Students<br />
Symposium, James Cook University, Townsville, 21-22 September.<br />
Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K. and Hayek, J. C. (2007) Piecing Together the Student<br />
Success Puzzle: Research, Propositions and Recommendations, ASHE Higher Education Report, 32, 5,<br />
San Francisco, Jossey- Bass.<br />
Laird, T. & Kuh, G. (2005) Student experiences with information technology and their relationship to other<br />
aspects of student engagement, Research in Higher Education, 46, (2): 211-33.<br />
Lindblom-ylänne, S., Pihlajamäki, H. & Kotkas, T. (2006) Self-, peer- and teacher-assessment of student essays,<br />
Active Learning in Higher Education, 7, 1: 51-62.<br />
Low Pay Commission (2011) National Minimum Wage Low Pay Commission Report 2011, accessed at:<br />
http://www.lowpay.gov.uk/lowpay/report/pdf/Revised_Report_PDF_with_April_date.PDF, [last accessed 20<br />
June 2011]<br />
454
Linda Martin et al.<br />
Mansell, J, (2011) Bristol care home: a failure on every level, The Guardian, accessed at:<br />
http://www.guardian.co.uk/commentisfree/2011/jun/01/bristol-care-home-failure-every-level, [last accessed<br />
20 June 2011]<br />
Meyer, J. H. F. and Ward, S. C. and Latreille, P. (2009) Threshold concepts and meta<strong>learning</strong> capacity,<br />
International Review of Economics Education., 8 (1), 132-154<br />
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education,<br />
Assessment and Evaluation in Higher Education, 35 (5): 501-517.<br />
Normann, R. (2000) Service Management: strategy and leadership in service business, 3 rd edition, Chichester:<br />
Wiley.<br />
McInnes, C. (2003) New realities of the student experience: How should universities respond? Paper presented<br />
at the European Association for Institutional Research 25 th Annual Conference, Limerick.<br />
Par, D. & Joordens, S. (2008) Peering into large lectures: examining peer and expert mark agreement using<br />
peerScholar, an online peer assessment tool, Journal of Computer Assisted Learning, 24, 6, 526-540.<br />
Pascarda, E. T. & Terenzini, P. T. (2005) How College Affects Students: a third decade of research, vol 2, San<br />
Francisco: Jossey-Bass.<br />
Race, P. (1998) Practical Pointers in Peer Assessment, in S. Brown (ed) Peer Assessment in Practice, (SEDA<br />
paper 102) Birmingham: SEDA.<br />
Race, P (2001) A Briefing Document on Self, Peer and group Assessment, LTSN Generic Centre.<br />
Reilly, R.R. & Chao, G. (1982) Validity and fairness of some alternative employee selection procedures,<br />
Personnel Psychology, 78: 113-123<br />
Sadler, D. R. (2010) Beyond feedback: developing student capability in complex appraisal, Assessment and<br />
Evaluation in Higher Education, 35 (5): 535-550.<br />
Salmon, G. (2000) E-moderating, London: Kogan Page.<br />
Skills for Care (2008) The Sector Qualification Strategy - Implications for employers, accessed at<br />
www.skillsforcare.org.uk%2Fnmsruntime%2Fsaveasdialog.aspx%3FlID%3D208%26sID%3D145&rct=j&q=<br />
The%20Sector%20Qualification%20Strategy%20-<br />
%20Implications%20for%20employers&ei=_mz_TZqaFo2yhAfMltSjCw&usg=AFQjCNFio4Er59rDUbiJ75uN<br />
1TudQVoz8g&sig2=Ql1r5FMnGqIZjgmAfov1LQ&cad=rja, [last accessed 20 June 2011]<br />
Tross, S. A., Harper, J. P., Osherr, L. W. and Kneidinger, L. M. (2000) Not just the usual cast of characteristics:<br />
using personality to predict college performance and retention, Journal of College Student Development, 41<br />
(3): 325-336.<br />
van der Pol, J, van den Berg, B, Admiraal, W, & Simons, P (2008) The nature, reception, and use of online peer<br />
feedback in higher education, Computers & Education, 51, 4: 1804-1817.<br />
Vickerman, P (2009) Student perspectives on formative peer assessment: an attempt to deepen <strong>learning</strong>?,<br />
Assessment & Evaluation in Higher Education, 34 (5): 221- 230.<br />
Yang, Y. (2010) Students’ reflection on online self-correction and peer review to improve writing', Computers &<br />
Education, 55, (3): 1202-1210.<br />
Yorke, M. (2006) Student engagement: deep, surface or strategic? Keynote address delivered at the 9 th Pacific<br />
Rim First Year in Higher Education Conference: Engaging Students. Griffith University, Gold Coast,<br />
Australia 12-14 July<br />
Zepke. N. & Leach, L. (2010) Improving student engagement: ten proposals for action, Active Learning in Higher<br />
Education, 11, (3): 167-177.<br />
455
Personalized e-Feedback and ICT<br />
Maria-Jesus Martinez-Argüelles 1 , Josep-Maria Batalla-Busquets 1 , Patricia<br />
Noguera-Guerra 1 and Ernest Pons-Fanals 2<br />
1 Economia i Empresa, Universitat Oberta de Catalunya, Barcelona, Spain<br />
2 Departament Econometria, Estadística i Economia Espanyola, School of<br />
Economics, Universitat de Barcelona, Barcelona, Spain<br />
mmartinezarg@uoc.edu<br />
jbatalla@uoc.edu<br />
pnoguera@uoc.edu<br />
epons@ub.edu<br />
Abstract: In the context of the adaptation of the new undergraduate, graduate and postgraduate degrees to the<br />
European Higher Education Area (EHEA), the student centered <strong>learning</strong> (SCL) is an approach which aims to<br />
overcome some of the problems inherent to more traditional forms of education by focusing on the learner and<br />
their needs, rather than being centred around the teacher’s input. Thus the student becomes the protagonist of<br />
his <strong>learning</strong> process by means of doing <strong>learning</strong> exercises or activities. So, these <strong>learning</strong> exercises or activities<br />
constitute the axis the student’s <strong>learning</strong> process. As far as they are evaluated and corrected in a systematic and<br />
rigorous form, they turn into the most direct and automatic channel that allows the student to get clear and<br />
concrete information on the foreseen aims’ and competences’ attainment. The aim of this article is to analyse the<br />
effectiveness and the efficiency of giving a more personalized feedback by means of ICT to the <strong>learning</strong><br />
exercises or activities done by the students. The way to achieve it is to offer the tutors a diversity of multimedia<br />
tools (text, oral, video, etc...) that facilitates this task. The objective is to reduce the time of preparing the<br />
feedback and thus to make it easier as well as to achieve a higher impact. The work has been undertaken in a<br />
virtual classroom where the tutor is provided with a kit of ICT tools that allows him to send to each student a<br />
personalized comment on his exercises or activities. To make this comment, different complementary alternative<br />
forms to written feedback have been explored in order to achieve greater motivation and greater impact. This<br />
alternative forms have been chosen taking into account the personality of the tutors participating in the project as<br />
well as their origins and profession paying special attention to their abilities in transmitting knowledge. That's why<br />
one of the main proposals is trying to get best communicative capabilities of each one. The results show that the<br />
students find personalized feedback very positive, because it makes <strong>learning</strong> easier and thus more motivating. In<br />
order to implement personalized feedback in an efficient way from the tutor’s point of view, several actions should<br />
be undertaken such as technical training in communication and pedagogy, modifying the way of correcting the<br />
exercises or activities and also adapting the dimension of the classroom by taking into account that the number of<br />
students is a key element for this tutorial model.<br />
Keywords: personal e-feedback, ICT, student centered <strong>learning</strong>, improvements in students rendability<br />
1. Introduction<br />
In the asynchronous eLearning contexts of Higher Education, based on written language, the<br />
feedback provided by the teacher is one of the basic elements in the students' <strong>learning</strong> process. The<br />
student centred <strong>learning</strong> (SCL) approach aims to overcome some of the problems inherent to more<br />
traditional forms of education by focusing on the learner and their needs, rather than being centred<br />
around the teacher’s input. Thus the student becomes the protagonist of his/her <strong>learning</strong> process by<br />
means of fulfilling <strong>learning</strong> exercises or activities.<br />
This study is made at the Universitat Oberta de Catalunya (UOC) The UOC is a totally virtual<br />
university. One of the basic characteristics of its educative pattern is to encourage a range of<br />
continuous evaluation activities to every subject that is part of the diverse <strong>learning</strong> programs. The<br />
underlying idea is that the student attains the competencies inherent to each subject by carrying out<br />
the different proposed activities. These continuous evaluation activities are called Continuous<br />
Evaluation Tests (PAC, for its acronym in Catalan). We call them tests as they also serve to assess<br />
how the students are performing on a continued basis and to check if they attain progressively the<br />
<strong>learning</strong> aims that are previewed.<br />
These PACs are, therefore, the axis of the student's work. Furthermore, as they are evaluated<br />
systematically and rigorously, they become the most direct and automatic channel for the student to<br />
receive a clear information on whether he/she is attaining the previewed goals or not, as well as the<br />
mistakes he/she is making and what he/she is lacking.<br />
456
Maria-Jesus Martinez-Argüelles et al.<br />
Nevertheless, one of the aspects of the UOC's <strong>learning</strong> model that the students have missed most in<br />
recent years is the absence of a personalized feedback to the continuous evaluation activities they<br />
have submitted [Martinez et al. 2010].<br />
This work is organized as follows: in the first part we present the theoretical framework that served us<br />
to think and design the pilot test. In the second part we describe the pilot test's implementation and<br />
we announce the first results. We close with some conclusions and the future lines of investigation<br />
opened out of this first experience.<br />
2. Methodology<br />
Using feedback in the pedagogical process has been broadly studied. Existing literature shows the<br />
difficulties arisen both from the teacher's perspective and the student's perspective, giving or receiving<br />
feedback in the <strong>learning</strong> process in an eLearning environment. Such difficulties are linked basically to<br />
the high ratio students-teacher in the virtual classrooms, that increases the follow-up task of the<br />
<strong>learning</strong> that the teacher could do (see [Buchanan, 2010] and [Ley, 1999]). Other existent difficulties<br />
are related to the contents of the feedback, that is to say, how and which feedback is provided to the<br />
student so as for him/her to make the best of it in his/her <strong>learning</strong> process, and to make sure that it<br />
has been the appropriate feedback [Gibbs et al., 2004]. This issue has not been too studied until now<br />
and even less in eLearning environments. The present work intends to explore how and which is the<br />
feedback to be provided, in order to improve its efficiency and efficacy in virtual classrooms with a<br />
high number of students. With this aim, different multimedia technologies are used (video, audio,<br />
screen captures, etc.) as a complement or substitute to written language. This aspect has not been<br />
studied yet and is the main contribution of this article.<br />
Taking as starting point the definition of the concept of feedback contained in works by Narciss and<br />
her collaborators ([Narciss, 2004, 2008], [Narciss et al., 2004a, 2004b, 2006]), the aim of this article is<br />
focused on its semantic and structural dimension.<br />
The semantic dimension of feedback refers to the contents it transmits, and has been studied in nonvirtual<br />
environments by several authors such as [Kluger et al., 1996], [Kramarski et al., 2001],<br />
[Kulhavy et al.,1989], [Mason et al., 2001] and [Narciss, 2004]. In virtual environments it has been<br />
tackled by [Espasa, 2008], [Espasa, 2010], [Espasa et al., 2010], [Alvarez et al.] and [Guasch et al.,<br />
2010]. Because of the objectives and characteristics of our study, we use the taxonomy elaborated by<br />
this author, which includes four semantic sub-dimensions of feedback (from the lesser to the major<br />
degree of complexity) that are not exclusive amongst them.<br />
Such sub-dimensions are the following:<br />
Sub-dimension 1 – Error identification and correction: the feedback informs about the correction<br />
of errors made.<br />
Sub-dimension 2 – Correct answer: the feedback shows the correct answer or gives the right<br />
solution.<br />
Sub-dimension 3 – Task improvement: the feedback recommends contents and strategies in<br />
order to improve the work, or the test that has been submitted.<br />
Sub-dimension 4 – In-depth information: the feedback suggests in-depth information in order to<br />
continue to move forward in <strong>learning</strong> in the future.<br />
Out of these four sub-dimensions, our analysis will focus on the first three, (the ones inherent to<br />
graduate studies). The last one is not used as it corresponds to postgraduate studies. Together with<br />
[Kulhavy et al., 1996], [Mory, 2004] and [Mason et al., 2001], we think that, in order for the feedback<br />
to succeed, it is necessary for it to contain both sub-dimensions 1 and 2 (in which the teacher<br />
provides a judgement on whether the answer is correct or incorrect) and sub-dimension 3 (that offers<br />
the student guidelines towards the correct answer).<br />
As regards the structural dimension of feedback, it refers to the presentation or the feedback's shape<br />
in a concrete context. Considering contributions made by [Narciss et al., 2006], others such as<br />
[Hyland, 2001, 2003] and [Espasa, 2008] in a eLearning environment of higher education such as the<br />
university we are analysing, the main elements that define the feedback's structural dimension are:<br />
457
Maria-Jesus Martinez-Argüelles et al.<br />
Agents that take part in the feedback processes: One of the elements inherent to feedback are<br />
the members that take part of its processes. This research has been developed by using a<br />
technological platform that integrates among others two applications that make it possible to<br />
provide the student with feedback. The first one is the continuous evaluation registry (RAC, for its<br />
acronym in Catalan), a space in the virtual classroom where the teacher inserts the marks of the<br />
continuous evaluation and facilitates the personalized feedback (which is sent to a private space<br />
only shared by teacher and learner). The second one is the calendar and it can be found in the<br />
planning space of the virtual classroom. There, the student can accede to the solutions of the<br />
continuous evaluation tests (PAC), prepared by the teacher and published one day after the<br />
students have submitted their solutions.<br />
Space of virtual communication where the feedback processes take place: Another component is<br />
the virtual space where the feedback takes place, be it public or private. In this study we analyse<br />
the effects of feedback in a private space (from now on personalized feedback). Authors such as<br />
[Rice et al., 1994] have proved that personalized feedback (private spaces) has advantages over<br />
feedback in public spaces because it fits better to the needs and characteristics of each student.<br />
Moment of feedback: To provide immediate or differed feedback has been the object of<br />
discussion in several researches on this issue. It is remarkable the proposal made by [Dempsey<br />
et al., 1988], that define some categories used in this work. The first one is immediate feedback (it<br />
provides the student with information on the correct answer) and in this context it corresponds to<br />
the PACs' solutions, published just one day after the students submit their answers. The second<br />
one is differed feedback (where the errors made are commented on, some contents are<br />
recommended as well as strategies to improve the process of <strong>learning</strong>), that gives added value to<br />
the solution of the PAC.<br />
Extension and form of the feedback: Another special feature of feedback in a virtual environment<br />
is its extension. It is tightly linked to the semantic dimension of feedback and to the kind of<br />
knowledge on which the feedback is given (depending on whether it is more abstract or<br />
conceptual or more applied and procedural). In order for the feedback to be effective it must give<br />
enough information to university students so as for them to attain the aims of <strong>learning</strong> previewed.<br />
The feedback can have different shapes: text and/or technological multimedia (audio and video<br />
recordings, screenshots with audio as well as the possibility to upload compressed videos). The<br />
student receives it in the RAC as an attached file or in the message itself.<br />
Following Narciss and her collaborators [Narciss 2004, 2008], [Narciss et al., 2004a, 2004b, 2006], in<br />
this article we also state that there are other dimensions apart from the semantic and the structural<br />
dimension with influence on the feedback processes, as in the case of the pedagogical design.<br />
Following we explain the pedagogical framework in which this study has been carried out.<br />
2.1 Pedagogical design<br />
One of the defining aspects of this work is the context in which it is carried out. It is an eLearning<br />
environment where the process of teaching and <strong>learning</strong> is conveyed through a technological<br />
platform. This work studies the case of the Universitat Oberta de Catalunya (UOC). This university<br />
was created in 1995 and from the very beginning has developed its activities in a virtual campus. In<br />
the UOC's model of education, the agents of the educative activity do not share the same space and<br />
time coordinates. This flexibility that involves that both the student and the teacher enjoy the <strong>learning</strong><br />
environment from any space and moment (asynchronous communication) needs the educative<br />
activity to be based upon communication exchanges between both of them.<br />
The UOC's pedagogical model puts the student in the centre of the teaching and <strong>learning</strong> process<br />
(student centred <strong>learning</strong>, SCL) carried out in a virtual classroom. This virtual classroom has four<br />
different areas: planning, communication, resources and evaluation. The evaluation area gathers all<br />
aspects related to the evaluation process of a given subject. This study is focused in one of the<br />
spaces of the evaluation area: the Continuous Evaluation Registry (RAC for its acronym in Catalan)<br />
that is where the personalized feedbacks take place. This personalized feedback consists of<br />
introducing marks by the teacher, and/or personalized inputs (sub-dimension 1, error identification<br />
and correction, and sub-dimension 3, task improvement). Despite the fact that sub-dimension 2 is not<br />
directly dealt with from the RAC (nor it is a direct part of the feedback), some guiding solutions to<br />
evaluated activities are given once the submission term is over. Furthermore, when necessary, part of<br />
the feedback uses the solutions as a support.<br />
458
Maria-Jesus Martinez-Argüelles et al.<br />
During the teaching and <strong>learning</strong> process, the student counts with the tutor's and the teacher's direct<br />
support. The latter is the one that teaches in the virtual classroom, accompanies the student in the<br />
development of the teaching and <strong>learning</strong> process acting both as a facilitator of the educative activity<br />
and as expert on the subject he/she is teaching.<br />
The UOC's evaluation system is based upon the combination of an educational and time-continuous<br />
evaluation, with a final evaluation. The continuous evaluation is based on a series of proposed<br />
evaluation tests -continuous evaluation test (PAC)- that are carried out on a time-continuous basis.<br />
To this point, the theoretical basis of this study and the context in which it has been developed have<br />
been described. It follows the empirical analysis and its results.<br />
3. Implementation<br />
It is in the context of research on processes of feedback with the aim to promote construction of<br />
knowledge and regulation of <strong>learning</strong>, still rather incipient in eLearning environments, where this study<br />
takes shape. Thus, attaining the objectives proposed in the context of this article will allow us to get<br />
results at two different levels: on the one hand, efficiency for the teachers and, on the other hand,<br />
efficacy for the students (performance). In this context, considering costs and benefits of<br />
implementing personalized feedback, it is necessary to question its usefulness, in order to extend it to<br />
other subjects or studies. With the aim to analyse the impact that has the incorporation of<br />
personalized feedback to the solution of activities and correction of exercises during the continuous<br />
evaluation process, and to make available several technological methodologies as well as their<br />
feasibility, diverse pilot tests have been designed according to the following criteria: groups of<br />
reduced dimensions (maximum 35 students) per subject have been chosen in order to ensure the<br />
teacher's adequate dimensioned task. Subjects have been selected according to the sort of<br />
knowledge (structural dimension of feedback) and diverse methodology, apart from considering<br />
teachers with different aptitudes, assessing the action's transversality and its possible extension to all<br />
of the subjects in the degree. The involved subjects, all of them from the Degree in Business<br />
Administration, were: Introduction to business (01.500), Statistics essentials (01.501), Behaviour of<br />
economic aggregates (01.508), Economic structure (01.510) and Introduction to accountability<br />
(01.520). As pointed out, there was a will to find subjects covering different fields of the degree. From<br />
applied economy (Economic structure and Behaviour of economic aggregates) to more<br />
methodological subjects (Statistics essentials), including accountancy subjects (Introduction to<br />
accountability) and business organization subjects (Introduction to business) (see detail in table 1).<br />
Table 1: Summary charter of forecasts made in each subject when carrying out the pilot test<br />
Subject<br />
Introduction<br />
to<br />
accountability<br />
Statistics<br />
essentials<br />
Economic<br />
structure<br />
Behaviour of<br />
economic<br />
aggregates<br />
Introduction<br />
to business<br />
In all the<br />
activities?<br />
Yes, with<br />
different levels<br />
of intensity<br />
Yes, with<br />
different levels<br />
of intensity<br />
Yes, with<br />
different levels<br />
of intensity<br />
Yes, with<br />
different levels<br />
of intensity<br />
Yes, with<br />
different levels<br />
of intensity<br />
To whom?<br />
PAC1 to everyone. To<br />
the rest, depending on<br />
the results (D, C- and<br />
C+ enough to<br />
everyone) and to the<br />
rest depending on the<br />
errors and evolution<br />
Depending on marks<br />
categories: C+, C- and<br />
D.<br />
PAC1 to everyone. To<br />
the rest, depending on<br />
the results and needs.<br />
To everyone with a<br />
weak mark and<br />
especially to those<br />
who fail<br />
PAC1 to everyone, to<br />
the rest depending of<br />
results and needs<br />
Level of<br />
personalization<br />
Individual<br />
Depending on<br />
the type of error<br />
made<br />
Individual<br />
How? Proposal of feedback<br />
PAC1/PAC2:<br />
audio and/or<br />
textual. The rest<br />
video or screenshots<br />
The most difficult<br />
(PAC2/PAC3):Ca<br />
ptures and video.<br />
To the rest<br />
text/audio.<br />
Video and/or<br />
audio<br />
Depending on<br />
Captures/video/te<br />
the type of error<br />
xt/audio<br />
made<br />
459<br />
Individual Video/text/audio<br />
To correct errors and<br />
guide/improve the<br />
<strong>learning</strong> process<br />
To correct errors and<br />
guide/improve the<br />
<strong>learning</strong> process<br />
Firstly, to motivate.<br />
Secondly, to correct<br />
errors and<br />
guide/improve the<br />
<strong>learning</strong> process<br />
To correct errors and<br />
re-orient/improve the<br />
<strong>learning</strong> process<br />
To correct errors and<br />
re-orient/improve the<br />
<strong>learning</strong> process and<br />
to motivate
Source: Own elaboration from surveys to teachers.<br />
Maria-Jesus Martinez-Argüelles et al.<br />
There has also been a will to find subjects with different evaluation models. Whereas some of them<br />
had five activities, four of which are compulsory (Economic structure, Behaviour of economic<br />
aggregates and Business introduction), others have five compulsory activities (Statistics essentials)<br />
and even others (Introduction to accountability) has five continuous evaluation activities with the<br />
obligation to submit the last one and three more out of the other four activities).<br />
The design of the pilot test has a double dimension. A technical part focussed on contributing to the<br />
maximum efficiency when making the teacher's development of feedback possible, and a teaching<br />
part that focusses on how to get a major impact on the student out of this intervention. In order to<br />
assess the action's impact it was rendered necessary to introduce monitoring classrooms (in order to<br />
grant the major degree of equalization possible in the monitoring classroom, it was made over the<br />
same subject, in the same semester, guided by the same teacher and on the same dimension, except<br />
for the subject Statistics essentials).<br />
The design of the technical part was made taking into account different possibilities of feasible<br />
multimedia feedback according to capabilities of the technological platform (audio, video, commented<br />
screen-shots...). In order to make it possible, some technical improvements have been implemented<br />
to the RAC, with the aim to facilitate elaboration of a personalized feedback. In fact, technological<br />
possibilities in a technological platform when applying these techniques, is a limitation to be taken into<br />
account when making it extensive to the rest of the degree.<br />
As regards the teaching part, the discussion on each of the feedback options' suitability and the<br />
possibility for it to be applied to one or several activities took place between the Teacher Responsible<br />
for the Subject (PRA for its acronym in Catalan) and the teacher. Where the multimedia feedback was<br />
not implemented, the option has been the written feedback (individual or individualized group). Having<br />
in mind these components, plus each teacher's abilities and concerns, a pilot test for each subject<br />
was defined. As common elements, individualised feedback was made in all activities of all subjects,<br />
though with different levels of intensity. In all cases, students have been given an answer to the<br />
generic PAC, in line with what is being done in the rest of the subjects in the Economy and business<br />
Studies (sub-dimension 2 in the previous part).<br />
Both the multimedia options adopted to carry out the personalized feedback and the number of targetstudents,<br />
are according to each subject and result from the planning made by the teacher and the<br />
PRA. Whereas in some cases, the option was a generalized personalized feedback in the first<br />
activities, in other cases, feedback was focussed on those persons that have not passed the activity<br />
or passed it with difficulties, in order to encourage them to go ahead as well as for them to try and<br />
correct the errors made so as to better face next activities, and also for them to be able to set up<br />
concepts that will allow them to pass the Final Test without difficulties (working sub-dimensions 1 and<br />
3 of the previous part). In other cases the option was for a more generic one, comprising students<br />
with similar typologies, whether because they made the same error when solving the activity or they<br />
had a similar final result in the activity. Another of the options adopted was the generalized<br />
personalized feedback (to all of the students), independently of their mark in order to correct errors of<br />
those who had a worse result and encourage those who had a good result to maintain their level.<br />
As already pointed out, the pilot test was adapted to each teacher's abilities and to each activity's<br />
specific needs. As said, the first activity in each subject had a generalized personalized written<br />
feedback in line with the evaluation criteria of the Economy and Business Studies degree. From the<br />
second activity, implementation of the multimedia feedback started in a generalized manner in all the<br />
subjects. In Economic Structure, feedback was made with audio oriented to those students that did<br />
not pass the subject, that made important errors or that did not submit it. From the third activity, audio<br />
was complemented by video in search of a generalization of messages so as they can be used with<br />
more than one student. In Statistics Essentials the option was a personalized feedback based on<br />
commented screen-shots this option was chosen because the subject's format allows a very efficient<br />
exploitation of this tool. Resolution of problems in the context of statistics requires certain approaches<br />
and mathematical developments that the commented screen capture dynamics makes it easier to<br />
understand (it is very useful in elements such tables reading, statistics calculations solving from a<br />
statistics program, etc.) In this sense, this feedback has not been generalized to the whole range of<br />
activities of the subject but was centred on those activities where major profits were foreseen. As<br />
460
Maria-Jesus Martinez-Argüelles et al.<br />
regards the personalized feedback's addressee, this was chosen according to errors made while<br />
carrying out the activity or because he/she did not pass the activity. This criteria intends to minimize<br />
the number of answers to be elaborated by the teacher, but involves a lesser degree of<br />
personalization. In order to fulfil this lack the multimedia feedback was complemented by written<br />
feedback (in some occasions, reiteration of errors made by the student can provoke two multimedia<br />
feedback entries). The cost of implementing such option is high but can be reduced in time for<br />
different reasons: improvement of the teacher's abilities, generalized use of answers to concrete<br />
problems and accumulation of examples and exercises solutions for the future (to be used in future<br />
feedback or as complementary tools to be used in the <strong>learning</strong> process).<br />
The pilot test carried out in the subject Behaviour of economic aggregates shows clearly the increase<br />
of the amount of work that involves the personalized feedback, even if it is written (this excess work<br />
was valued as double the work, though there have been attempts to save time with common answer<br />
strategies). The strategy followed here varied according to results obtained by students and to the<br />
test's features. In PAC 2 audio feedback was implemented when searching a major impact (the PAC<br />
was not passed, the mark could have improved in relation with the previous PAC, …) whereas in the<br />
rest of cases the option was written feedback (good results in PACs, non-submission of two of the<br />
PACs,...) For the rest of PACs this has been the criteria followed when considering which feedback<br />
should be implemented. Furthermore, video was introduced in cases where a rather common error<br />
was made. In this case, technological abilities also become a key factor to be taken into account.<br />
Here, the assessment made by the tutor is very positive due to the degree of proximity between<br />
student and teacher that this technique allows. In this case, instead of facilitating understanding of<br />
certain concepts, as happened in subjects such as Statistics Essentials, the most valued element is<br />
improvement in the student's answer due to the impact that personalized feedback had on him/her.<br />
Nonetheless, again it is remarkable the importance that dimension has when offering a correct<br />
personalized feedback.<br />
In Introduction to Accountability there was a general written answer to everybody in the first PAC,<br />
then audio personalized feedback to those that submitted the second PAC and written to those that<br />
had not done so. Video was used in the rest of activities. In this case, the only discrimination was<br />
between students who has submitted the activity and those who had not. The degree of performance<br />
of students in the classroom where the personalized feedback took place was higher.<br />
In Business Introduction, feedback was also performed according to the mark, in text for the first<br />
activity. From then on, the option was to incorporate audio / video feedback depending on the mark<br />
obtained after the second activity, with a clear intention to improve the result in those cases where<br />
they had not been positive. Together with these messages, written feedback was used when it was<br />
necessary to complement some information. In this case, the teacher assessed that video or audio<br />
feedback messages did not bring significant quality improvement with respect to the personalized<br />
feedback in text. What is really considered as critic in this case is the possibility to perform<br />
personalized feedback with all the students. The option to extend the use of such tools in the <strong>learning</strong><br />
process as a complement in cases where there are important doubts about the contents and the<br />
teacher can clarify them, is raised.<br />
Once all pilot tests have been presented, it is time to draw up a balance. The benefits of implementing<br />
multimedia personalized feedback go in two aspects. The first one shows how useful is such<br />
methodology in cases that it leads to improving understanding and <strong>learning</strong> of the subject, from the<br />
correction of errors and the extension of contents. In this sense, it must be taken into account that the<br />
fact of considering personalized and adapted to each student's needs feedback, is in itself a highly<br />
positive elements that is worth counting on. The second aspect to be considered is the impact that<br />
multimedia personalized feedback has on the student, due precisely to the fact that it has been done<br />
in audio or in video. Some teachers pointed out the positive assessment of students on this option<br />
and, therefore, it must be understood as an element to be taken into account in the future.<br />
The costs involved in the adoption of this methodology are high and are directly related to the<br />
dimension. Teachers must acquire technical and communication skills that need to be developed.<br />
This is the reason why the need of training the teachers before implementing this option is suggested.<br />
Time invested in preparing generalized and personalized feedback, both written and multimedia, is<br />
quite high and, thus, it is necessary to assess how effective is to incorporate it. In fig. 1 we see that,<br />
as an average, some 20% of the total time invested by the teacher, can be directly attributed to<br />
461
Maria-Jesus Martinez-Argüelles et al.<br />
generation of personalized feedback. Moreover, it must be considered that this is directly related to<br />
the size of the group. Over 40 students, timely-costs in the elaboration of feedback go far beyond the<br />
time teachers must dedicate to it. This forces us to opt for several strategies. Reducing the number of<br />
people in the classroom leading to an optimal size could be an example, or also cost-reducing<br />
strategies can be raised, starting with the creation of generic messages (oriented to solve concrete<br />
doubts and errors which are common to several students).<br />
50,0<br />
45,0<br />
40,0<br />
35,0<br />
30,0<br />
25,0<br />
20,0<br />
15,0<br />
10,0<br />
5,0<br />
0,0<br />
Training, Multimedia personalised feedback Correction Multimedia implementation Personalized feedback<br />
01.520 01.500 01.508 01.501 01.510<br />
Figure 1: Hours destined to correction and personalized feedback, by type of work (ource: Own<br />
elaboration from surveys to teachers).<br />
One element that has not been taken into account when implementing the pilot test and could add<br />
some significant distortion to the analysis is the fact that in the subject Statistics Essentials there was,<br />
at the same time, another pilot test incorporating important changes to the structure of the student's<br />
classroom being carried out. This fact reduced our frame of reference to work with when making the<br />
comparison. That is to say, whereas comparison between classrooms can be analysed with no fear,<br />
comparative analysis with previous semesters is difficult because they would not be strictly<br />
comparable. Moreover, in this case, the teacher did not manage two small-sized groups but only one.<br />
4. Conclusions<br />
Implementation of multimedia personalized feedback arises as a need to improve the student's<br />
<strong>learning</strong> capacity and performance, making the best out of generalization and spreading of multimedia<br />
tools, beyond the feedback in text. From this element and after developing a set of pilot tests in<br />
several subjects of the Degree of Business Administration the following conclusions can be drawn:<br />
Firstly, with the aim to make multimedia feedback more efficient and to make the best of its options,<br />
teachers need specific training both technological and pedagogical, as well as to develop<br />
communication abilities through video. Secondly, it has been proved that personalized feedback<br />
requires an important time investment by the teacher, therefore it is necessary to plan the intensity<br />
and frequency in which inputs will take place both regarding students and PACs. Thirdly, the<br />
effectiveness of feedback and accuracy of using one or another channel, one or another type of<br />
feedback depends a lot on the typology of the subject and the teachers' communication and<br />
technological abilities. Fourthly, it can be stated that personalized feedback brings a closer<br />
relationship between learner and teacher, this having favourable and encouraging effects on <strong>learning</strong><br />
on the students. It is also certain that, on the other hand, this more personalized relationship between<br />
learner and teacher is to the detriment of the relationship classroom-group. Fifthly, it is apparent that<br />
personalization of feedback changes the way PACs are corrected and, therefore, lastly we can<br />
conclude that using other ways to give feedback different from the written one, does not reduce the<br />
462
Maria-Jesus Martinez-Argüelles et al.<br />
teachers' dedication and makes it hard to be implemented generally in groups of 70 students with the<br />
current contractual relationship of dedication.<br />
These conclusions must be compatible with the fact that students judge very positively the<br />
personalized feedback and has a facilitating as well as motivating effect on <strong>learning</strong>. Nevertheless,<br />
according to what the teacher suggests, the most important element is what is said rather than the<br />
tool that is used to say it.<br />
On the other hand, in order for this feedback to be implemented by teachers, a technical training on<br />
communication and pedagogy is needed. This personalization requires additional dedication and new<br />
ways to correct PACs that make it difficult to be extrapolated to a 70-student classroom-group,<br />
particularly taking into account that the longer this personalized feedback takes place, the more<br />
demanding students become and the teacher-student interaction increases.<br />
Finally, from this study several lines of future investigation arise. Data analysis regarding academic<br />
performance between pilot classrooms and monitoring classrooms is a key element to be valued<br />
when considering that the academic performance of the over effort that personalized feedback<br />
means. In this sense, it must also be considered, when assessing its possible spreading to the rest of<br />
subjects within the context, whether or not different types of subjects correspond to different types of<br />
feedback.<br />
Acknowledgements<br />
This article is the result of a collective work resulting from the project “TOWARDS AN<br />
IMPROVEMENT OF THE E-FEEDBACK”, funded by AGAUR (Catalan Goverment), leaded by María<br />
Jesús Martínez Argüelles (2010MQD00145), We are grateful to the work done by the rest of members<br />
of the project: Marc Badia-Miró, Carolina Hintzmann, Dolors Plana-Erta, Muriel Garreta Domingo,<br />
David Trelles Bertran and Antoni Mangas. We also want to thank the collaboration of Anna Espasa<br />
Roca when improving the methodological work around feedback. Lastly, we want to thank the<br />
economic support given by the Economy and Business Department of the Universitat Oberta de<br />
Catalunya.<br />
References<br />
Alvarez, and., Espasa, A. & Guasch, T. (in press). “The value of feedback in improving collaborative writing<br />
assignments in an online <strong>learning</strong> environment”. Studies in Higher Education.<br />
Buchanan, T. (2000). “The efficacy of a World-Wide Web mediated formative assessment”. Journal of Computer<br />
Assisted Learning, 16, 193-200.<br />
Dempsey, J.V. & Wager, S.U. (1988). “A taxonomy for the timing of feedback in computer-based instruction”.<br />
Educational Technology, 28(10), 20–25.<br />
Espasa, A. & Meneses, J. (2010). “Analysing feedback processes in an online teaching and <strong>learning</strong><br />
environment: An exploratory study”. Higher Education, 59-3, 277-292.<br />
Espasa, A. (2008). “El Feedback en el marc de la regulació de l’aprenentatge: caracterització i anàlisi en un<br />
entorn formatiu en línia”. Doctoral Thesis. On line, available<br />
at:http://www.tdx.cat/browse?value=Espasa+Roca%2C+Anna&type=author<br />
Espasa, A. (2010). “Temporal and assessment dimension: characterisation of feedback after assignments”.<br />
eLearn Center Research Paper Series, Issue 1. Time factor in eLearning and assessment.<br />
http://elcrps.uoc.edu/ojs/index.php/elcrps/article/view/issue1-espasa<br />
Gibbs, G. & Simpson, C. (2004). “Conditions under which assessment supports students’ <strong>learning</strong>”. Learning and<br />
Teaching in Higher Education, 1, 3-31.<br />
Guasch, T.; Espasa, A. & Álvarez, and.(2010). “Formative e-feedback in collaborative writing assignments: the<br />
effect of the process and time”. eLearn Center Research Paper Series, Issue 1. Time factor in eLearning<br />
and assessment. http://elcrps.uoc.edu/ojs/index.php/elcrps/article/view/issue1-guasch-espasaalvarez/issue1-guasch-espasa-alvarez<br />
Hyland, F. (2001). “Providing Effective Support: Investigating feedback to distance language learners”. Open<br />
Learning, 16(3), 233-247.<br />
Hyland, F. (2003). “Focusing on form: Student engagement with teacher feedback”. System, 31, 217-230.<br />
Kluger, A.N. & DeNisi, A. (1996). “The effects of feedback interventions on performance: A historical review, a<br />
meta-analysis, and a preliminary feedback intervention theory”. Psychological Bulletin, 119(2), 254-284.<br />
Kramarski, B. & Zeichner, O. (2001). “Using technology to enchance mathematical reasoning: Effects of feedback<br />
and self-regulation <strong>learning</strong>”. Educational Media International, 38 (2-3), 77-82.<br />
Kulhavy, R.W. & Stock, W.A. (1989). “Feedback in written instruction: the place of response certitude”.<br />
Educational Pyschology Review, 1(4), 279-308.<br />
Ley, K. (1999). “Providing feedback to distance students”. Campus-Wide Information Systems, 16(2), 63-69.<br />
463
Maria-Jesus Martinez-Argüelles et al.<br />
Martínez, M.J.; Juan, A.A.; Castan, J. (2010). “Using the Critical Incident Technique to Identify Factors of Service<br />
Quality in Online Higher Education”. International Journal of Information Systems in the Service Sector,<br />
2(4), 57-72.<br />
Mason, J. & Brunning, R. (2001). Providing feedback in computer-based instruction: what the research tell us.<br />
University of Nebraska-Lincoln: http://dwb.unl.edu/Edit/MB/MasonBruning.html<br />
Mory, E. H. (2004). “Feedback research revisited”. A D. Jonassen (Ed.), Handbook of research on Educational<br />
Communications and Technology. Mahwah, NJ: Lawrence Erlbaum, 745-785.<br />
Narciss, S. (2004). “The impact of informative tutoring feedback and self-efficacy on motivation and achievement<br />
in concept <strong>learning</strong>”. Experimental Psychology, 51(3), 214-228.<br />
Narciss, S. (2008). “Feedback strategies for interactive <strong>learning</strong> tasks”. A J.M. Spector, M.D. Merrill, J. Narciss,<br />
S., Körndle, H., Reimann, G. & Müller, C. (2004a). Feedback-seeking and feedback efficiency in web-based<br />
<strong>learning</strong>– How do they relate to task and learner characteristics? A. P. Gerjets, P. A. Kirschner, J. Elen, &<br />
R. Joiner (Eds.), Instructional design for effective and enjoyable computer- supported <strong>learning</strong>. Proceedings<br />
of the first joint meeting of the EARLI SIGs Instructional Design and Learning and Instruction with<br />
Computers (pp. 377-388). Tuebingen: Knowledge Media Research Center.<br />
Narciss, S. & Huth, K. (2004b). How to design informative tutoring feedback for multimedia <strong>learning</strong>. A H. M.<br />
Niegemann, R. Brünken & D. Leutner (Eds.), Instructional Design for Multimedia Learning (pp. 181-195).<br />
Münster: Waxmann.<br />
Narciss, S. & Huth, K. (2006). Fostering achievement and motivation with bug-related tutoring feedback in a<br />
computer-based training for written subtraction. Learning and Instruction, 16(4), 310-322.<br />
Van Merriënboer & M. P. Driscoll (Eds.). Handbook of Research on Educational Communications and<br />
Technology (Aect). New Jersey (EUA): Lawrence Erlbaum.<br />
Rice, M., Mousley, J & Davis, R. (1994). “Improving student feedack in distance education: a research report”. A<br />
T. Evans & D. Murphy (Eds.), Research in distance education (pp. 52-62). Geelong (Austràlia): Deaking<br />
University Press.<br />
464
Evaluation of Multimedia Tools and e-Feedback in Virtual<br />
Learning Environments<br />
Maria-Jesús Martínez-Argüelles 1 , Marc Badia-Miro 2 , Carolina Hintzmann 1 and<br />
Dolors Plana-Erta 1<br />
1<br />
Economia i Empresa, Universitat Oberta de Catalunya, Barcelona, Spain<br />
2<br />
Departament Història Econòmica, Universitat de Barcelona, Barcelona, Spain<br />
mmartinezarg@uoc.edu<br />
mbadia@ub.edu<br />
chintzmann@uoc.edu<br />
dplana@uoc.edu<br />
Abstract: There is a high consensus that one of the key elements to ensure knowledge attainment in a virtual<br />
university context is the existence of regulatory processes of <strong>learning</strong>, which allows students to regularly evaluate<br />
their <strong>learning</strong> process. In asynchronous <strong>learning</strong> environments one of the usual strategies to facilitate this<br />
regulation is the use of feedback mechanisms between the student and tutor. But in the environment of large<br />
groups of students written personalized feedback may be too laborious for the tutor. In this paper we present a<br />
pilot project focusing on the context of Degree in Business Administration from the Universitat Oberta de<br />
Catalunya. We have explored ICT tools to give feedback (audio, video and screenshots). The aim is to improve<br />
the effectiveness and efficiency of the <strong>learning</strong> regulatory process. During one semester the project was<br />
implemented by taking a virtual classroom with a reduced number of students in specific subjects using<br />
alternative ways to written feedback. In order to evaluate the effects of using these alternative multimedia tools,<br />
the results obtained have been compared with the other benchmark groups taking into account the following<br />
aspects: academic results, assessment of students, evaluation of tutors, and hours of dedication of tutors. The<br />
implementation of the pilot test and evaluation of results have allowed to draw some conclusions about the<br />
conditions under which the use of multimedia tools in feedback may in fact help the adjustment process in virtual<br />
<strong>learning</strong>.<br />
Keywords: feedback, virtual <strong>learning</strong>, ICT, personal feedback in large groups<br />
1. Introduction<br />
In the framework of the implementation of new degrees to the European Higher Education Area<br />
(EHEA), the student becomes the protagonist of his/her <strong>learning</strong> process, which carries out by<br />
fulfilling <strong>learning</strong> activities. In order to ensure that the student really attains the competencies<br />
associated to each subject, the function of accompaniment and personalised relationship between<br />
tutor and learner becomes a priority.<br />
The UOC is a totally virtual university. One of the basic characteristics of its educative pattern is to<br />
encourage a range of continuous evaluation activities to every subject that is part of the diverse<br />
<strong>learning</strong> programs. The underlying idea is that the student attains the competencies inherent to each<br />
subject by carrying out the different proposed activities. These continuous evaluation activities are<br />
called Continuous Evaluation Tests (PAC, for its acronym in Catalan). We call them tests as they also<br />
serve to assess how the students are performing on a continued basis and to check if they attain<br />
progressively the <strong>learning</strong> aims that are previewed.<br />
These PACs are, therefore, the axis of the student’s work. Furthermore, as they are evaluated<br />
systematically and rigorously, they become the most direct and automatic channel for the student to<br />
receive a clear information on whether he/she is attaining the foreseen goals or not, as well as the<br />
mistakes he/she is making and what he/she is lacking.<br />
Nevertheless, one of the aspects of the UOC’s <strong>learning</strong> model that the students have missed most in<br />
recent years is the absence of a personalised feedback to the continuous evaluation activities they<br />
have submitted (Martínez, M.J.; et al (2010)). Taking as starting point the definition of the concept of<br />
feedback given by Susanne Narciss in her multiple scientific works (Narciss, S. (2004, 2008) and<br />
Narciss, S. & Huth, K. (2004, 2006)), this study is focussed on the semantic and structural dimension<br />
of feedback.<br />
465
1.1 Semantic dimension<br />
Maria-Jesús Martínez-Argüelles et al.<br />
The semantic dimension of feedback refers to the contents it transmits, and has been studied in nonvirtual<br />
environments by several authors such as Kluger, A.N. & DeNisi, A. (1996), Kramarski, B. &<br />
Zeichner, O. (2001), Kulhavy, R.W. & Stock, W.A. (1989), Mason, J. & Brunning, R. (2001) and<br />
Narciss, S. (2004). In virtual environments it has been tackled by Espasa, A. (2008). Because of the<br />
objectives and characteristics of our study, we use the taxonomy elaborated by this author, which<br />
includes four semantic sub-dimensions of feedback (from the lesser to the major degree of<br />
complexity) that are not exclusive amongst them. So, such sub-dimensions are:<br />
S1. Error identification and correction: the feedback informs about the correction of errors made.<br />
S2. Correct answer: the feedback shows the correct answer or gives the right solution.<br />
S3. Task improvement: the feedback recommends contents and strategies in order to improve the<br />
work, or the test that has been submitted.<br />
S4. In-depth information: the feedback suggests in-depth information in order to continue to move<br />
forward in <strong>learning</strong> in the future.<br />
Our analysis will focus on the first three sub-dimensions (error identification and correction, correct<br />
answer and task improvement) as they are the most inherent to initial subjects in the framework of<br />
graduate studies. So in our analysis feedback made by the tutor includes the following subdimensions:<br />
1) Guidelines to the answer: once the term to submit the continuous evaluation activities is over, a<br />
correct indicative answer is provided to the whole group of students in the classroom. Thus, students<br />
can carry out a self-evaluation activity, comparing this solution to the contents of the activity they<br />
submitted. This category of feedback fully coincides with sub-dimension 2 of the feedback’s semantic<br />
dimension.<br />
2) Specific feedback: Each tutor has in the virtual classroom a tool (continuous evaluation registry)<br />
that allows the sending of (written) comments to each student. But often, groups are of 70 students or<br />
more and there is the commitment to correct and comment on the test in a term no longer than a<br />
week. So, this makes a personalised input to every student very costly.<br />
The solution we have adopted is twofold. On one hand, there is a feedback according to the mark<br />
obtained. All students that have got the same mark receive in the private space an identical comment<br />
on their tests.. Although, this is not a fully personalized feedback; obviously we can assimilate this<br />
kind of feedback to the third semantic sub-dimension.<br />
On the other hand, it is also common to all subjects to send to the public space of the virtual<br />
classroom a more generic comment on the most common mistakes done in the evaluation test as well<br />
as some advices in order to improve the <strong>learning</strong> process (feedback S1).<br />
3) Answer to doubts: Moreover, each tutor answers the doubts raised by the students on the activities<br />
and contents they refer to. This could be labelled as a reactive, non-proactive feedback. The aim of<br />
this kind of input is to improve the work to be done by students, coinciding with the third subdimension<br />
previously described.<br />
1.2 Structural dimension<br />
As regards the structural dimension of feedback, it refers to the presentation or form of the feedback<br />
provided in a given context. Inspired on contributions made by Narciss, S. & Huth, K. (2004), Hyland,<br />
F. (2001, 2003) and Espasa, A. (2008) in a higher degree e-<strong>learning</strong> context as it is the case of the<br />
Universitat Oberta de Catalunya, the main elements that define the feedback's structural dimension<br />
are:<br />
Space of virtual communication where the processes of feedback take place: One of the elements<br />
inherent to feedback are the spaces where its processes take place. This study has been<br />
developed with the use of a technological platform that integrates among others two tools that<br />
466
Maria-Jesús Martínez-Argüelles et al.<br />
serve to provide the student with feedback. The first one is the continuous evaluation registry, a<br />
space in the virtual classroom where the tutor inserts the marks of the continuous evaluation and<br />
facilitates the personalised feedback (which is sent to a private space only shared by tutor and<br />
learner). The second one is the calendar and it can be found in the planning space of the virtual<br />
classroom. There, the student can accede to the solutions of the continuous evaluation tests,<br />
prepared by the tutor and published one day after the students have submitted their solutions.<br />
Publicness or privacy of the feedback’s processes: Another component is the virtual space where<br />
the feedback takes place, be it public or private. In this study we analyse the effects of feedback<br />
in a private space (from now on, personalised feedback). Authors such as Rice, M., et al. (1994)<br />
have proved that personalised feedback (private spaces) has advantages over feedback in public<br />
spaces because it fits better to the needs and characteristics of each student.<br />
Moment of feedback: To provide immediate or differed feedback has been the object of<br />
discussion in several researches on this issue. It is remarkable the proposal made by Dempsey,<br />
J.V. & Wager, S.U. (1988), that define some categories used in this work. The first one is<br />
immediate feedback (it provides the student with information on the correct answer) and in this<br />
context it corresponds to the continuous evaluation tests, that are published just one day after the<br />
students submit their answers. The second one is deferred feedback (where the errors made are<br />
commented on, some contents are recommended as well as strategies to improve the <strong>learning</strong><br />
process), that provides value added to the solution of the continuous evaluation test.<br />
Extension and form of the feedback: Another special feature of feedback in a virtual environment<br />
is its extension. It is tightly linked to the semantic dimension of feedback and to the kind of<br />
knowledge evaluated (depending on whether it is more abstract or conceptual or more applied<br />
and procedural). To guarantee an effective feedback, it must give enough information to students,<br />
so that they attain the foreseen <strong>learning</strong> aims. The feedback can have different shapes: text<br />
and/or technological multimedia (audio and video recordings, screen-shots with audio as well as<br />
the possibility to upload compressed videos). The student receives it in the continuous evaluation<br />
registry as an attached file or in the message itself.<br />
Furthermore, this input, apart from contributing to improve the students' <strong>learning</strong> process, becomes a<br />
key motivating tool for them not to quit the subjects but to go on with the continuous evaluation<br />
process.<br />
So, to grant UOC's pedagogical model's quality the implementation of personalized inputs becomes a<br />
key factor.<br />
On the contrary, elaborating personalised inputs of four or more continuous evaluation activities by<br />
the tutors, that have classrooms with 70 students or more, is often a non-sustainable task due to the<br />
time it requires particularly taking into account that the estimate and demanded dedication of tutors is<br />
maximum of two hours per day.<br />
In this project we analyse the efficiency, of different forms of feedback alternative to written text. The<br />
analysis have been done both from the the student's viewpoint and the tutor’s point of view, but .this<br />
work is focussed on the assessment from the student’s point of view .<br />
The pilot test consists of exploring new ways of feedback with the aim to make it closer, more efficient<br />
and more motivation-generating to the student. One of the UOC's features is the diversity of tutors<br />
that take part in it. Diversity understood as different origins, professions, and particularly, as refers to<br />
abilities in transmitting knowledge. In this sense, we propose to pay special attention to this diversity,<br />
trying to make each one's best communicating skills arise. Thus, we propose to investigate on the<br />
possibilities to give inputs in different formats that current technologies make available (audio, video,<br />
screen-shots with audio, etc.). We propose to integrate this input in the framework of the continuous<br />
evaluation registry in order to make it even less costly to tutors. Furthermore, we integrate different<br />
tools that facilitate adoption of diverse multimedia responses (video recordings, audio recordings and<br />
the chance to upload compressed videos).<br />
In this context, the pilot test's main objective is to assess the effect of introducing a major<br />
personalization in the feedback received by students through an offer of alternative channels (oral,<br />
video, etc.) to the tutors, in order to facilitate their task.<br />
467
2. Methodology<br />
Maria-Jesús Martínez-Argüelles et al.<br />
With the specific aim already mentioned, a pilot test was designed in order to analyse the results and<br />
the students' assessment on the use of multimedia tools to provide personalised feedback.<br />
The pilot test's design has been double-sided. On the one hand, the technical side that focussed on<br />
making feedback feasible by the tutor, and, on the other, the teaching side that focussed on how to<br />
make it have a major impact on the student.<br />
When implementing the pilot test, these criteria were followed:<br />
Five subjects as different as possible were selected, so the implementation of multimedia<br />
feedback could be evaluated according to the typology of the subject of the Business<br />
Administration Degree.<br />
The pilot test has focussed on one of the subject's classrooms in order for it to check its real<br />
impact, while other monitoring classrooms of the same subject were maintained. Furthermore, a<br />
group of small dimensions was considered.<br />
Previous information from the involved students was searched in order to analyse their profile,<br />
preferences and predisposition to receive a better feedback and the importance they give to its<br />
quantity and quality, among other aspects.<br />
Bearing in mind the diverse technical available options when giving feedback (audio, video, audio<br />
and video or commented screen-shots), we opted for trying different tools in different subjects<br />
taking into account which one suits better to the subject's needs.<br />
Feedback in the pilot test was only implemented in specific evaluation tests, searching to<br />
maximize its usefulness. In the rest of tests the option was for written feedback.<br />
In all cases, feedback is linked to the correction of evaluation tests.<br />
Now we are going to take a look at the technical design of the computer tool that facilitated<br />
incorporation of multimedia feedback.<br />
2.1 Technical part<br />
In order to have available all necessary tools in the continuous evaluation registry, to start the pilot<br />
test, an integration of LANGblog was developed in this application of mark registration. Thus, in a<br />
short space of time the options of audio and video recording have been incorporated to the<br />
continuous evaluation registry.<br />
Despite this agile integration makes the implementation of the pilot test of multimedia feedback<br />
possible, it is not a scalable nor accurate solution in the long term. The objective was to allow a quick<br />
and easy sound and video recording as well as its editing and ulterior visualization by the students.<br />
2.2 Teaching part<br />
Teachers responsible of the subjects that make up the pilot test, jointly with the tutors that carried it<br />
out, defined the structure that the feedback should have during the pilot test. To do it, eight items<br />
adapted in every case to the needs inherent to each subject, were defined:<br />
Is feedback necessary in all the activities?<br />
Who must the feedback be addressed to?<br />
What level of personalization should it have? It’s appropriate to group students by marks in order<br />
to send them the same feedback?<br />
Which of the tools must one opt for? Video? Audio? Screen-shots? Etc.<br />
Is it applicable to all classrooms of the subject? Is it a monitoring classroom?<br />
Is feedback complemented with a generic solution?<br />
Which is the specific purpose of feedback?<br />
Bearing these elements in mind, each pilot test was adapted to the main specific features of each<br />
subject. The common elements to all subjects were the ones that refer to the fact that feedback has<br />
been implemented in all activities of all subjects, with a different level of intensity and that the pilot test<br />
468
Maria-Jesús Martínez-Argüelles et al.<br />
was implemented only in Catalan-teaching campus., For all the subjects taking part in the pilot test<br />
there was a monitoring classroom with the same dimension and managed by the same tutor, so that<br />
the comparison between evolution of the students' performance in similar environments was easy,<br />
except for one of the subjects. As we already explained in point 1.1 Semantic dimension.<br />
Then, there is a range of particularities inherent to each subject. Among them, differences in the<br />
number of persons to which feedback was addressed are remarkable. Whereas in some cases,<br />
everybody would receive multimedia feedback during the first continuous evaluation tests and it would<br />
be personalised, in others the option was to focus only in those persons that passed the activity only<br />
tightly or rather, that directly did not pass it. In this case, the aim is to encourage the student to go<br />
ahead with the subject and to be aware of the errors he/she may have made, as well as to be able to<br />
correct and solve them. In this sense, there are also differences when looking at the level of<br />
personalization of feedback. Whereas some subjects opt for it to be personal and individualized, other<br />
prefer them to be more generic and to include students with similar typologies, whether because they<br />
made the same kind of mistake when solving the activity or because they had a similar final result.<br />
Finally, another of the most significant differences between subjects is tools used to elaborate the<br />
feedback. While some chose audio, others chose video or screen-shots<br />
3. Results<br />
Our starting point is that in asynchronous teaching contexts, the feedback provided by the tutor is one<br />
of the basic elements in the students' <strong>learning</strong> process. Thus, one of the first aspects to be analysed<br />
was the perception that students had about the importance and need of feedback linked to the<br />
continuous evaluation tests.<br />
For this reason, a questionnaire was designed. It was distributed among the students at the beginning<br />
of the semester in order to know their opinions on feedback. This gives an a priori view and, then,<br />
after finishing the semester, it can be compared with the results of a second questionnaire in order to<br />
know the students’ perception of feedback received.<br />
The first questionnaire was answered by a total of 45 students out of 165 registered in the involved<br />
groups, which makes a 27,4%. In it when asked to compare the importance of feedback in an on-line<br />
course with an in-person course, the answer was that more importance is given in an on-line course,<br />
independently if it’s they are enrolled for the first time or not at UOC<br />
Furthermore, if we analyse their opinion, also in a range from 1 (least importance) to 5 (more<br />
importance), about several aspects of feedback, we can prove that more importance is given to<br />
quality of feedback and to the degree of personalization rather than to quantity or to the moment they<br />
receive it. But, most of all, importance is given to the fact of receiving feedback from the tutor.<br />
Table 1: Assessment of the importance of diverse aspects of feedback<br />
Woman Man Total<br />
Receiving feedback from the tutor 4.6 4.7 4.7<br />
Amount of feedback received 3.5 3.6 3.6<br />
Quality of feedback received 4.1 3.9 4.0<br />
The moment when feedback is received 3.5 3.7 3.6<br />
Degree of personalization of feedback 4.3* 3.6* 4.0<br />
* Scores statistically different with a significance of 1%.<br />
If we compare average scores given by men and women, a statistical contrast can be made in order<br />
to know if these differences are statistically significant, that is to say, if we can conclude that there are<br />
clear differences between men and women's opinion. Well, the only case where a significant<br />
difference is observed is with relation to the “degree of personalization” of feedback. This means that<br />
women give more importance to the degree of personalization than men.<br />
This analysis can be repeated by making a distinction between students for whom this is their first<br />
semester at the UOC and those for whom it is not, and between students that already have a<br />
university degree and those who have not. In any of the cases, there are no significant differences.<br />
469
Maria-Jesús Martínez-Argüelles et al.<br />
Conclusion, thus, is double: The big importance that all groups of students give to the fact of receiving<br />
feedback from the tutor. And, to a lesser extent, the significant importance given to aspects such as<br />
quality and degree of personalization and, a little less, to quantity and moment.<br />
These conclusions confirm the interest of a pilot test like this, which intends to explore and assess,<br />
precisely, the effect of introducing channels and methods of alternative feedback that are different<br />
from a written message.<br />
This initial assessment can be complemented with the posterior opinion of these students, once the<br />
semester where multimedia tools to give personalised feedback were used is finished. Following is<br />
the comment on the most relevant results for the five subjects (that we called subjects A, B, C, D and<br />
E).<br />
In the case of this second questionnaire a total of 39 students out of the 164 registered in the groups<br />
involved answered, which means a 23,8%. Regarding the 5 subjects, 8 students answered to subject<br />
A, 8 to subject B, 8 to subject C and 15 to subject D. In this case, we have changed the names of the<br />
subjects by “fictitious” names as we try to assess the results of the pilot test and to the task carried<br />
out by the tutors in each subject.<br />
One of the questions made to students is if they “consider that feedback received was excessive,<br />
accurate or insufficient”. In this case, the answers received make it clear that students, in no case,<br />
considered excessive the feedback received. On the contrary, some of them (only women)<br />
considered it was insufficient. Such consideration is interesting if we bear in mind that the aim of the<br />
pilot test was to assess ways of attaining a better feedback.<br />
Table 2: Assessment on the feedback received (number of women)<br />
A B C D Total<br />
Excessive 0 0 0 0 0<br />
Accurate 2 1 4 3 10<br />
Insufficient 2 3 0 3 8<br />
Total 4 4 4 6 18<br />
Table 3: Assessment on the feedback received (number of men)<br />
A B C D Total<br />
Excessive 0 0 0 0 0<br />
Accurate 4 2 3 9 18<br />
Insufficient 0 0 0 0 0<br />
Total 4 2 3 9 18<br />
Another of the key questions for us to assess the students' opinion is the one where they are asked if<br />
they “consider that the degree of personalization of the feedback received was excessive, accurate or<br />
insufficient”. In this case, the students' perception is more or less the same as in the previous<br />
question.<br />
The two tables that follow, where men and women's answers have been separated, prove that none<br />
of the students that answered the questionnaire feel that the degree of personalization of feedback<br />
received is excessive.<br />
Table 4: Assessment on the degree of personalization of feedback received (number of women)<br />
Woman A B C D Total<br />
Excessive 0 0 0 0 0<br />
Accurate 2 1 4 2 9<br />
Insufficient 2 3 0 4 9<br />
Total 4 4 4 6 18<br />
470
Maria-Jesús Martínez-Argüelles et al.<br />
Table 5: Assessment on the degree of personalization of feedback (number of men)<br />
Man A B C D Total<br />
Excessive 0 0 0 0 0<br />
Accurate 2 2 3 8 15<br />
Insufficient 2 0 0 1 3<br />
Total 4 2 3 9 18<br />
Finally, as a way to try and capture the students' point of view on methods used to give feedback, the<br />
following listing shows the answers to the questionnaire's open question where they were asked to<br />
give their opinion on “how to improve the feedback received in the subject”. Reading these opinions<br />
suggests some aspects that, we think to be very interesting as for example:<br />
It seems that even though rarely students give critical opinions about the dynamics in the<br />
classroom, this does not mean that they do not have a critical view about teaching practices that<br />
can be more useful to them. And, when asked, they give interesting answers (at least some of<br />
them).<br />
All the given opinions go in the line of claiming for a major degree of personalization.<br />
Students prove to be very practical and pragmatic in the sense that they value, above all, the<br />
usefulness of feedback received, much more than the type of feedback or the channel (audio,<br />
video, etc.) used.<br />
4. Conclusions<br />
As conclusions from the experience we want to outline the three most relevant ideas that students<br />
evidenced on the importance and need of receiving feedback:<br />
In general, students give more importance to feedback in an on-line <strong>learning</strong> than in an in-person<br />
<strong>learning</strong> context.<br />
Students give more importance to quality of feedback and the degree of personalization rather<br />
than to the amount or the moment they receive it. But, above all, they give importance to the fact<br />
of receiving the feedback from the tutor.<br />
Comparing average scoring given by men and women, there are some clear differences between<br />
men’s and women’s opinion concerning some of the assessed issues. Particularly, women give<br />
more importance to the degree of personalization than men.<br />
And, on the other hand, when assessing the concrete experience with subjects and groups where<br />
feedback based on several channels (audio, video or screen-shots) were used, their opinions show<br />
the following:<br />
In no case, students consider excessive or inaccurate the feedback received. On the contrary,<br />
some of them (in this case only women) considered it was insufficient.<br />
Students value, above all, obtaining clear messages, in which the explanation to their mistakes<br />
are perfectly explained. That is to say, the feedback that, according to the identified dimensions,<br />
aggregates the semantic dimension 2 and 3.<br />
Students show to be very pragmatic as they value, above all, the feedback received, much more<br />
than the type of feedback or the channel (audio, video, etc.) used.<br />
Independently of the channel used, the fact to be able to consult in different moments the<br />
feedback given to a particular activity and, even, that it can be printed out is positively judged by<br />
students.<br />
The average student is usually one that adapts him/herself to working conditions in each subject.<br />
Thus, despite the fact that conditions and tools are improvable, often he/she does not suggest or<br />
claim other options or resources. But, once he/she discovers (or is shown) that there can be other<br />
tools such as videos, personalised messages, particular corrections, etc., he/she gets used to<br />
them and claim them when he/she does not find them.<br />
In conclusion, it seems that students valuate very positively personalised feedback and this has a<br />
facilitating effect on <strong>learning</strong>, as well as on motivation. But, according to opinions expressed by<br />
471
Maria-Jesús Martínez-Argüelles et al.<br />
students it seems that they value more the contents of communication than the tool through which the<br />
message is transmitted.<br />
On the other hand, the tutors, in order to implement this feedback, need technical, communication<br />
and pedagogical training. Preparing personalized feedback requires additional dedication and new<br />
ways to correct continuous evaluation tests which make it difficult to extrapolate it to a<br />
group/classroom of 70 students. Particularly, if we bear in mind that the more personalized feedback<br />
is given to students, the more demanding they become, so a door opens to increasing interaction<br />
between tutor and learner.<br />
Acknowledgements<br />
This article is the result of a collective work resulting from the project (2010MQD00145) “TOWARDS<br />
AN IMPROVEMENT OF THE E-FEEDBACK”, funded by AGAUR. We are grateful to the work done<br />
by the rest of members of the project: J.M. Batalla-Busquets, P. Noguera-Guerra, Ernest Pons, Muriel<br />
Garreta Domingo, David Trelles Bertran and Antoni Mangas. We also want to thank the collaboration<br />
of Anna Espasa Roca when improving the methodological work around feedback.<br />
References<br />
Dempsey, J.V. & Wager, S.U. (1988). “A taxonomy for the timing of feedback in computer-based instruction”.<br />
Educational Technology, Vol 28, No. 10. pp 20–25.<br />
Espasa, A. (2008). “El Feedback en el marc de la regulació de l’aprenentatge: caracterització i anàlisi en un<br />
entorn formatiu en línia”. Doctoral Thesis. On line,<br />
http://www.tdx.cat/browse?value=Espasa+Roca%2C+Anna&type=author<br />
Hyland, F. (2001). “Providing Effective Support: Investigating feedback to distance language learners”. Open<br />
Learning, Vol 16, No. 3, pp 233-247.<br />
Hyland, F. (2003). “Focusing on form: Student engagement with teacher feedback”. System, Vol 31, 217-230.<br />
Kluger, A.N. & DeNisi, A. (1996). “The effects of feedback interventions on performance: A historical review, a<br />
meta-analysis, and a preliminary feedback intervention theory”. Psychological Bulletin, Vol Vol 119, No. 2,<br />
pp 254-284.<br />
Kramarski, B. & Zeichner, O. (2001). “Using technology to enchance mathematical reasoning: Effects of feedback<br />
and self-regulation <strong>learning</strong>”. Educational Media International, Vol 38, No. 2-3, pp 77-82.<br />
Kulhavy, R.W. & Stock, W.A. (1989). “Feedback in written instruction: the place of response certitude”.<br />
Educational Pyschology Review, Vol , No. 4, pp 279-308.<br />
Martínez, M.J.; Juan, A.A. & Castan, J. (2010). “Using the Critical Incident Technique to Identify Factors of<br />
Service Quality in Online Higher Education”. International Journal of Information Systems in the Service<br />
Sector, Vol 2, No. 4, pp 57-72.<br />
Mason, J. & Brunning, R. (2001). Providing feedback in computer-based instruction: what the research tell us.<br />
University of Nebraska-Lincoln: http://dwb.unl.edu/Edit/MB/MasonBruning.html<br />
Narciss, S. (2004). “The impact of informative tutoring feedback and self-efficacy on motivation and achievement<br />
in concept <strong>learning</strong>”. Experimental Psychology, Vol 5, No. 3, pp 214-228.<br />
Narciss, S. (2008). “Feedback strategies for interactive <strong>learning</strong> tasks”. A J.M. Spector, M.D. Merrill, J. Van<br />
Merriënboer & M. P. Driscoll (Eds.). Handbook of Research on Educational Communications and<br />
Technology (Aect). Lawrence Erlbaum, New Jersey (EUA).<br />
Narciss, S. & Huth, K. (2004). How to design informative tutoring feedback for multimedia <strong>learning</strong>. A H. M.<br />
Niegemann, R. Brünken & D. Leutner (Eds.), Instructional Design for Multimedia Learning (pp. 181-195).<br />
Münster: Waxmann.<br />
Narciss, S. & Huth, K. (2006). Fostering achievement and motivation with bug-related tutoring feedback in a<br />
computer-based training for written subtraction. Learning and Instruction, Vol 16, No. 4, pp 310-322.<br />
Rice, M., Mousley, J & Davis, R. (1994). “Improving student feedack in distance education: a research report”. A<br />
T. Evans & D. Murphy (Eds.), Research in distance education (pp. 52-62). Deaking University Press<br />
Geelong, Australia.<br />
472
Cyberbullying: A Workplace Virus<br />
David Mathew<br />
University of Bedfordshire, Luton, UK<br />
David.Mathew@beds.ac.uk<br />
Abstract: This paper explores workplace cyberbullying in an education institution in the south of England, in<br />
which declarations of zero tolerance towards bullying masked the reality that it was silently condoned as a means<br />
of controlling staff. As with face-to-face bullying, cyberbullying is a matter of impact and not necessarily of intent;<br />
and here we contemplate the role of the bully and the role of the victim, while viewing an example through a lens<br />
of control theory. The following questions – Is there a need for bullying in the workplace? Does it serve a<br />
function? Does bullying help contain workplace anxiety as well as create it? – are posed. The case of a man in<br />
his mid-thirties, who was systematically bullied by his manager for eighteen months, is presented. Here I<br />
examine the social structures that the bullying enforced (and destroyed) and examine what the subject learned<br />
from the manager's behaviour. In my commentary on this case study I refer to Wilfred Bion's work on workgroup<br />
anxiety. In Group Psychology and the Analysis of the Ego, Freud explores group formation and the giving up of<br />
individual ideals for the group ideal. I argue that something analogous happens in a workplace environment in<br />
which bullying is rife and in which a scapegoat must be found, even if there is no corresponding misdemeanor for<br />
which he must be punished. I examine the need for homogeneity when it comes to bullying, as well as the issues<br />
of power; transference; the defence against paranoid anxieties; and what happens when a manager is troubled<br />
by others' intelligence. Finally, in the second half of the paper, I extrapolate a future of cyberbullying. The name<br />
of the company in question has been made anonymous and throughout this paper is referred to only as 'the<br />
Institution'. Similarly, the name given to the victim – Rob – has been invented for the sake of anonymity, at his<br />
request.<br />
Keywords: cyberbullying, bullying, anxiety, control<br />
1. The poisoned department<br />
At the end of March 2011, the comedy/satire website The Daily Mash produced a faux-news story<br />
entitled ‘Mob seeks new thing to be angry about’. In this article it emerges that, around the United<br />
Kingdom, enraged people 'may be forced to return to their slightly depressing lives unless they can<br />
find a new cause' to rebel against in a violent manner.'The angry mob,' the story continues, 'who had<br />
been shouting a lot outside a building they believed was the site of a trial but was actually the remains<br />
of a Courts Furniture Superstore, have been enjoying being definitely the goodies for once.'<br />
It is interesting to note this use of for once. Bullies who work in crowds – whether their own tags would<br />
be anarchists, agitators, even protesters – are well aware that they are not 'the goodies' in the eyes of<br />
others: by preying on weaker targets they are clearly in the wrong – again, in the eyes of others –<br />
whatever their original motives. But are they in the wrong in their own eyes? And what implications<br />
does self-awareness, or the lack of it, have for the twenty-first century's development of a bullying<br />
culture, namely the rise of cyberbullying? More specifically still, what implications are there for the<br />
victim and the perpetrator of cyberbullying in the workplace?<br />
It might come as a shock to some people that organisations exist in which cyberbullying is not only<br />
accepted, it is silently encouraged as a tried and tested means of controlling staff. When we think of<br />
industries in which bullying is rife, perhaps we think of industries in which a strict and enforced<br />
hierarchical structure is in place: the Army, for example; the Police Force, the Prison Service, or the<br />
Navy. Whether these perceptions are justified or not, these industries are more likely to be thought of<br />
as places were bullying occurs. However, there are industries in which, for all their proclamations of<br />
fairness for all and zero tolerance towards bullying and harassment, a general air of malice and fear is<br />
not only allowed to exist, but is encouraged. The Institution referred to herein is an example in the<br />
education industry. The workplace operates on a system of strict hierarchy, in which management<br />
and the lower-paid education staff rarely communicate beyond the necessary; in which creative<br />
thinking and inititaitve is firmly suppressed. As John Steiner writes in Seeing and Being Seen (2011):<br />
'A tolerance of difference is necessary for development and for creativity, but difference can also<br />
provoke envy, and it is often when this is attached to injustice that the destructiveness becomes so<br />
magnified' (p.12). It is this triangulation – of envy, injustice and destructiveness – that has raised the<br />
bullying bar to new heights of creative cruelty.<br />
Although cyberbullying is in its infancy, it is growing up fast, and effective techniques are maturing<br />
with it. At places like the Institution, where it is at best ignored and at worst condoned, it pays to keep<br />
473
David Mathew<br />
members of staff frightened. Far from reducing them to unproductive emotional jelly, cyberbullying is<br />
used as a means of lowering staff morale and robbing employees of belief in themselves and in their<br />
skills, which results in the employee working harder to prove his place. Counterintuitively, perhaps,<br />
the damage works in a fashion that increases the propensity for taking on extra tasks: to begin with, at<br />
least. After a while, the pressure builds, and the victim either crumples or tries to achieve against<br />
(deliberately) impossible goals as way of securing his manager's receding favour. The victim's<br />
inevitable submission to psychological impacts such as stress, anxiety, a failure of concentration, are<br />
used as spurs to induce more work. Physiological impacts such as heart rate increases, blood<br />
pressure changes, sweating, shortness of breath, headaches, and so on, are regarded as clear signs<br />
that the victim was not the equal of the challenge in the first place.<br />
Traditionally cyberbullying has taken the form of threats and intimidation, impersonation, stalking,<br />
defamation of character, rejection, and the unauthorised (and unwanted) publication of private<br />
material. The workplace cyberbully will not have such tactics at his or her disposal. Something more<br />
subtle, and arguably more insidious, must be employed. And while an email written in an<br />
exaggeratedly formal style that is completely out of character might not sound like cyberbullying, if the<br />
style of such a message is repeated fifteen times a day, from a manager who sits in the same office<br />
as the victim, and if the messages are copied to most of the senior management team, the cumulative<br />
impact constitutes a case of bullying. For although it is fair to say that bullying in general is a matter of<br />
impact and not intent (i.e. if I think I am being bullied then I am being bullied, irrespective of your<br />
original intentions), the case in question relied on such techniques to such an extent that the victim<br />
believed that nearly everyone had been turned against him through fear of the manager.<br />
Increasing anxiety makes the victim feel isolated among his peers. Inadvertently (perhaps), the<br />
bystanders of cyberbullying become perpetrators themselves, their collective misdemeanours<br />
compounded of ignorance and inaction. Before long it is entirely plausible that these accessories will<br />
enjoy the scapegoating of one individual as much as the originator of the offensive material does. The<br />
fact that scapegoating is generally interpreted as a defence against psychological distress in groups is<br />
therefore pertinent: in a failing team in a ‘poisoned department’, the respite from group anxiety takes<br />
the form of making fun of a figure of fresh hatred, whether or not he has done anything to upset<br />
anybody. His perceived tolerance of the accumulated ‘jokes’ and rebukes via email (for example) are<br />
sufficient provocation to continue. As Wilfred Bion (1961) writes in Experiences in Groups: ‘there is<br />
no way in which the individual can, in a group, “do nothing” – not even by doing nothing’ (p118).<br />
This is one way of paraphrasing part of Sigmund Freud’s Group Psychology and the Analysis of the<br />
Ego (1921), in which Freud explores group formation and the giving up of individual ego ideals for the<br />
group ideal. With reference to the emotional homogeneity of cyberbullying, one might argue that<br />
something analogous is taking place. Referencing Bion himself, in ‘The unconscious at work in groups<br />
and teams’ (1994), Jon Stokes describes such a ‘basic assumption dependency’ team (Bion 1961)<br />
perfectly: ‘there is little capability to bear frustration… and quick solutions are favoured… members<br />
have lost their capacity to stay in touch with reality and its demands. Other external realities are also<br />
ignored or denied… instead of seeking information, the group closes itself off from the outside world<br />
and retreats into paranoia. A questioning attitude is impossible; any who dare to do so are regarded<br />
as either foolish, mad or heretical’ (p22-23). Thus the question might be begged at this point: is there<br />
actually a need for bullying in the workplace? Perhaps the fact that cyberbullying becomes pandemic<br />
says more about the group than it does the bully (or the victim). In this case, does cyberbullying serve<br />
a function? Perhaps it helps contain workplace anxiety as well as create it? Even if these hypotheses<br />
are correct, however, they do not help the victims. And this being said, this is a good point to<br />
introduce Rob, who is the victim in the following case.<br />
2. The wolf who cried boy?<br />
Rob is a thirty-five year old professional man who was subjected to eighteen months of systematic<br />
workplace bullying at an institution for further education. Before the bullying began, he had spent a<br />
contented year at the Institution; the change in the working dynamics between Rob and his older<br />
female manager changed gradually, from positive to negative, and followed no impetus of which Rob<br />
is aware. Not once in the past had Rob experienced problems with a manager. Rob is a hard worker<br />
and prides himself of being thought of favourably in the workplace; and while the manager had a<br />
reputation of being troubled by the greater intelligence of others, Rob had always assumed that<br />
although he was better educated, he and his manager had had a reasonable working relationship.<br />
However, as time progressed, the feedback that Rob received from his manager was increasingly<br />
474
David Mathew<br />
negative, to the point that not a single thing that Rob completed remained unscathed from the<br />
manager’s weak editing skills; and the shift in his manager’s attitudes towards him were confusing.<br />
Although the two parties shared an office of twenty employees, and sat no more than five metres from<br />
one another, the manager stopped talking to Rob directly, even in team meetings, unless she<br />
specifically had to. If Rob asked a question of the manager while in company, the manager either<br />
brushed it away or immediately asked Rob a question in return, sometimes the very same question.<br />
The manager made fun of Rob when the chance arose, for example referring to his childless status<br />
with the comment: ‘He hasn’t even started yet’ when she was discussing children with another<br />
manager… Rob became nervous of approaching the manager’s desk: the manager would sometimes<br />
make him wait a minute while finishing off an email or a piece of work, pretending that it was<br />
inconceivable that she could be interuupted at this crucial moment, even though others had<br />
interrupted her in the previous minute. Rob was the only member of the team not to be invited to the<br />
pub for occasional drinks after work, or to the Christmas dinner; the only member of the team not to<br />
be invited to share an infrequent lunch together in the canteen.<br />
When the manager stopped addressing Rob directly and started using email for every<br />
communication, Rob was baffled and started to feel anxious. At this point it did not occur to him that<br />
this constituted cyberbullying, but he was aware that it was peculiar behaviour (by anyone’s<br />
standards) and that it was behaviour that had been reserved for him alone. Before long, every<br />
message that the manager sent to Rob was also copied to most of the senior management team;<br />
what was worse than the feeling that this action created that he was being ‘watched’ for reasons that<br />
he did not understand was the sensation that he experienced that the contents of the messages were<br />
becoming increasingly (and deliberately) unclear. Rob believes that the emails were written in such a<br />
vague way that he would have no choice but to ask for clarification of the task that he had been given,<br />
or even of the topic in question. The clarification would then be seen in any subsequent emails by the<br />
same people that were observing the proceedings.<br />
After months of this sort of intensive scrutiny, interspersed by entire weeks in which the manager<br />
refused to communicate with Rob in any fashion at all, leaving several of his projects in the air, the<br />
resultant effect on Rob’s nerves and emotional condition was severe. In the meantime, Rob’s<br />
applications for the same training that other members had been accepted for were turned down;<br />
although other team members were allowed flexible working conditions, Rob was told that he must<br />
report in at 9 a.m. and finish at 5 p.m. – even though the department had long since had a guideline of<br />
‘early in, early home’ or ‘late in, late to leave’ for people with external responsibilities. Up to this point,<br />
Rob had always favoured an arrival at work an hour earlier than his colleagues as he had a long<br />
commute home. With this new ‘rule’ that was imposed only on him, his working day stretched by an<br />
extra three hours as he was no longer able to catch his usual train home and had to wait some time<br />
for the next one. After ten months of bullying, Rob went off sick.<br />
The treatment drove him to a recourse to medication and a prolonged spell away from work (during<br />
which the subject saw more than one message that confirmed that he would not be returning to work,<br />
even though he had stated nothing for the kind). He admits to feeling anxious that the department was<br />
discussing him; unbeknownst to Rob, his colleagues had been told by his manager not to ‘bother’ him<br />
with any messages of goodwill or any Get Well Soon cards. A regular appointment with a counselling<br />
service was swiftly arranged via Rob’s G.P. The counsellor, who had worked for this particular NHS<br />
Trust, later stated that Rob’s was one of the worst examples of bullying that she had ever seen. And<br />
yet, throughout all of this, Rob clung to the vanishing hope that things would eventually get better<br />
without any form of legal intervention.<br />
It was around this time that Rob started getting texts on his mobile phone from a number that he did<br />
not recognise. The messages were unambiguous but nevertheless perplexing: ‘Failure’, ‘Lonely?’,<br />
‘What did you want NOT to talk to me about?’ At first he deleted them, still attempting to convince<br />
himself that he was looking at the whole matter incorrectly, and that things would get better soon. He<br />
put the texts down to a simple coincidence, and his manager’s former and current silence down to her<br />
own pressures in the workplace. (Not once did the manager call Rob to ask after his welfare. He was<br />
off sick for nearly a third of a year.) As the texts kept coming, sometimes after midnight, Rob began to<br />
suspect that not all was well psychologically and mentally with his manager, and though he knew that<br />
steps can be taken to trace a mobile owner’s details, he also knew that the mobile operator can only<br />
475
David Mathew<br />
disclose this information to the police, and Rob was reluctant to involve the police. He now wishes<br />
that he had involved the police, and much earlier on.<br />
It is somewhat ironic that a piece of work that Rob had authored earlier, during the year in which<br />
everyone seemed happy, had been on the subject of bullying. Thus it was that Rob had some<br />
recourse to information that he never thought that he would use practically; now that his health had<br />
been affected (headaches, anxiety attacks, vomiting) he looked back on this work and at some of the<br />
references therein. He was alarmed to note how little of what he’d written could be used in his own<br />
case. For example, victims of cyberabuse are advised to block emails or remove names from<br />
contacts; but how is this possible if one’s bully is also one’s manager? Similarly, when a YouTube<br />
‘joke’ had been sent to everyone in the department as a link, with the single line in the body of the<br />
email that read ‘One for Rob…’ and the film contained content that addressed something personal to<br />
Rob, the victim was unsure of how to proceed. After all, everyone had been sent the joke film. How<br />
was he meant to say that he alone had found it offensive without appearing as a crybaby?<br />
Eventually Rob screwed up the courage to approach a Union representative. Next, on his eventual<br />
return to work, he reported the cyberbullying to his manager’s manager, even though he suspected<br />
that the latter would be futile. So it proved: the manager’s manager merely told Rob that his<br />
perception was wrong and that what he thought was happening was not happening at all. It was all in<br />
his mind: his tormentor didn't mean him any harm, and so on. Rob was flatly informed that<br />
cyberbullying did not exist in the Institution for the simple reason that it was not acceptable. This was<br />
the extent of the mitigation: it does not exist because we say that it not allowed to exist. (Surely this is<br />
the workplace equivalent of a child holding his hands over his own eyes and assuming that no one<br />
can see him because he is now in darkness. It is workplace equivalent of putting fingers in one's ears<br />
and chanting: Can't here you! Can't hear you!)<br />
There followed months of meetings (while the counselling continued) and Rob was eventually forced<br />
out of the Institution with a payment of several thousands of pounds and a copy of a contract that he<br />
was obliged to sign that said that he accepted this payment as a redundancy package. Needless to<br />
say, the contract said nothing about bullying: Rob, or rather the man whose real name I must not<br />
disclose, had effectively been gagged on the subject, legally, for ever. If the title of one of Slavoj<br />
Žižek’s many papers – namely, ‘Is this digital democracy, or a new tyranny of cyberspace?' – was not<br />
applicable enough in this instance, then let us take a glance at what the author has to say. ‘Our social<br />
identity, the person we assume to be in our social intercourse, is already a "mask" that involves the<br />
repression of our inadmissible impulses,’ Žižek’ writes. ‘The fact that I perceive my virtual self-image<br />
as mere play thus allows me to suspend the usual hindrances which prevent me from realising my<br />
"dark half" in real life. My electronic id is given wing’ (web article).<br />
3. Unpatrolled boundaries<br />
When cases of cyberbullying are of a severity to make the news (ref the Houghton case,<br />
NetFamilyNews 2009) there is often a reference to the anonymity (or attempted anonymity) or the<br />
perpetrator. The idea is for the audience to recognise the sly and insidious fashion with which the<br />
bully reached his/her victim. Arguably, however, the case where the bully is completely upfront and<br />
'open' is worse. In cases such as this, it becomes a matter of misdirection and stealth. The bully<br />
knows enough about the victim to be able to hurt him by using languages/images that might be<br />
regarded (by others outside the dualogue) as innocuous enough. This serves, of course, to make<br />
matters worse. If the bully is hurt by words/images that might be innocent in an uncontextualised<br />
situation, the bully is thought to be 'too sensitive' or 'not tough enough'. It represents another slur on<br />
the bully's character, and now a wider audience is allowed to join in.<br />
The workplace bully is an abuser of power, whether this power is actual (where the bully might have<br />
influence over someone’s career, for example) or supposed (where the bullying has been left alone<br />
because it is easier to accept than the challenge to its existence would be). Either way, relationships<br />
in the workplace should be governed by the same sort of boundaries that survive in any other<br />
relationship; and where cyberbullying has been ignored we see a clear example of unpatrolled<br />
boundaries. No one is checking. According to Wenger (2000), members of a community are bound<br />
together by a collective understanding of what their community is about; they hold each other<br />
accountable to his joint enterprise. In the absence of a possibility to maim or kill physically, the<br />
workplace bully’s only real ‘weapon’ is the use of technology, and the only real target the victim's<br />
476
David Mathew<br />
emotional world: a world that has opened up for everybody who is interested in exploring it, since the<br />
mass market evolution of the Internet and of social networking in cyberspace.<br />
Social control theory would suggest that instead of looking forfactors that make people reveal criminal<br />
tendencies, we should try to explain why people do not become criminal. In Causes of Delinquency<br />
(1969),Travis Hirschi revealed four personal characteristics – attachment to others, a belief in the<br />
moral validity of rules, the commitment to achievement, and an involvement in conventional activities<br />
– the possession of which might keep a person away from criminal intent. But what if the attachment<br />
to others is the attachment to likeminded individuals in a bullying group? What if the ‘rules’ are not<br />
understood by everybody (in the way that the victim will fail to understand them)? What if<br />
‘achievement’ is only scored via the downfall of others (or one in particular)? And so on. There is no<br />
doubt that Hirschi’s research has been studied for decades, but there is equally no doubt that he was<br />
writing before the advent of the Internet, and that times change. Violence has changed for ever.<br />
4. Future dissonance<br />
Neimeyer (1993) and Steenbarger (1991 ) suggest that anyone is capable of creating a variable future<br />
based on one’s own history and experience, one’s perceptions of the moment, and personal values.<br />
So a pertinent question, perhaps, is not 'Whence Cyberbullying?' or even 'Why Cyberbullying?':<br />
perhaps we should be asking: 'Where Next, Cyberbullying?' When the achievements have already run<br />
to financial disaster due to staff dissatisfaction, and to employee suicide – even murder – what is left<br />
for cyberbullying to work towards, and how will it set about this challenge?<br />
A cyberbully who exhibits the efficacy of the one highlighted in this paper does not arrive fully formed:<br />
effective cyberbullying is an iterative approach. The treading of a fine line between cruelty-and-notgetting-caught<br />
and misfired-slander-and-getting-caught takes practice and resolve. The individual<br />
concerned has almost certainly executed similar hate crimes in the past; it is likely that he or she will<br />
execute more in the future, unless the organisation takes a robust stand against the habit. Quite<br />
possibly the bully was a victim of bullying in the past, or began the practice as early as during the<br />
school years. The bully uses the psychoanalytic device of transference – the replacement of early<br />
feelings for someone onto someone new, years later, and cyberbullying thereby becomes a bully’s<br />
defence against paranoid anxieties. Or to put it another way, in the findings of Loeber and Hay<br />
(1997), there is much to learn about contemporary aggression in studies of the aggressor’s<br />
adolescence. As Steiner (1993, p4) would have it: ‘Traumatic experiences with violence or neglect in<br />
the environment leads to the internalisation of violent disturbed objects which at the same time serve<br />
as suitable receptacles for the projection of the individual's own destructiveness.’<br />
While we may hypothesise that cyberbullying constitutes a resolution of issues of physical space, or a<br />
delineation of one's struggle with personal boundaries, we must not forget (or ignore) the painful truth<br />
that some people simply enjoy being unpleasant. It might not have been a uniform that was worn (in<br />
this case), and the 'weapon' might not have fired shells, but cyberbullying is every bit as distressing as<br />
a case of (say) a sergeant showing up a constable's faults in front of others. The context has<br />
changed; the consequences and the motives have not. What is certain is that the bully will continue<br />
to derive personal satisfaction (and arguably ersatz sexual gratificationo) from the attempts to wound<br />
and vilify unless the emotional imbalances and lacks that he or she endures are directly addressed.<br />
The bully fears that someone else can function better or is more capable of carrying out the job at<br />
hand; the quest for power is the by-product of individual impotence, and the incompetence that the<br />
bully shows in the workplace is often camouflaged by the actions that are aimed at the victim. Without<br />
a change in a failing department that addresses these shortcomings head on, the bully thrives and<br />
continues with identical behaviours, unable to change. The analogy in the following should be simple<br />
enough to grasp, from Ricky Emanuel’s seminal paper on psychological retreats, 'A Void':<br />
‘The baboon smells the watermelon seeds and puts its hand through the hole. By<br />
grabbing the seeds in its fist, it cannot get its fist out of the hole and remains trapped<br />
there until the Bushmen come and capture it. It cannot think that releasing the<br />
attachment to the seeds will result in its freedom. It is something like this that I am<br />
discussing, in relation to becoming fixed or attached to one’s sense of identity or even<br />
true self’ (pX).<br />
However, we must not disregard the human being's simple reluctance to let go of a situation – and an<br />
identity – once it has been established, regardless of how unpleasant this situation and identity might<br />
be. In his seminal paper on the psychoanalytic 'Void', Ricky Emanuel ( explains as follows: 'The<br />
477
David Mathew<br />
baboon smells the watermelon seeds and puts its hand through the hole. By grabbing the seeds in its<br />
fist, it cannot get its fist out of the hole and remains trapped there until the Bushmen come and<br />
capture it. It cannot think that releasing the attachment to the seeds will result in its freedom. It is<br />
something like this that I am discussing, in relation to becoming fixed or attached to one’s sense of<br />
identity or even true self.’<br />
At the start of this paper I wrote that it 'might come as a shock' to learn that there were institutions in<br />
which cyberbullying was tacitly endorsed. Please note my use of the word might for I used it<br />
deliberately. What I could have said was 'It will come as a shock' but when I typed this I had a failure<br />
of confidence in my sentence. Would it come as a shock, I wondered, to consider cyberbullying in the<br />
workplace? I wasn't sure so I opted for might. I am still not sure, the truth be told; but if what has been<br />
said has not come as any sort of shock (or shall we compromise with 'unpleasant surprise'?) then<br />
questions are inevitably begged. The logical sequiturs are at least twofold. If information such as this<br />
does not come as a shock, first why does it not come as a shock; and second, where have you seen<br />
such behaviour yourself? A third question, on which I would like to conclude, is this:<br />
What did you do about it?<br />
References<br />
Bion, W.R. (1961) Experiences in Groups. East Sussex/New York: Routledge.<br />
Daily Mash (2011) Mob seeks new thing to be angry about. http://www.thedailymash.co.uk/news/society/mobseeks-new-thing-to-be-angry-about-201103303673/.<br />
Accessed on 30 March 2011.<br />
Emanuel, R. (2001) ‘A Void – An Exploration of Defences Against Sensing Nothingness’ in Int. J. Psychoanal. 82,<br />
1069.<br />
Freud, S. (1921) Group Psychology and the Analysis of the Ego. SE 18. The Standard Edition of the Complete<br />
Psychological Works of Sigmund Freud. Trans. James Strachey. London: Hogarth, 1953-74.<br />
Hirschi, T. (1969) Causes of delinquency.Berkeley: Unversity of California Press.<br />
Loeber, R., & Hay, D. (1997). Key issues in the development of aggression and violence from childhood to early<br />
adulthood. Annual Review of Psychology, 48, 371-410.<br />
Neimeyer, R.A. (1993) , An appraisal of constructivist psychotherapies. Journal of Consulting and Clinical<br />
Psychology , 61(2), 221-234.<br />
NetFamilyNews. (2009) First UK teen to be jailed for cyberbullying. http://www.netfamilynews.org/?cat=5526.<br />
Accessed 31 May 2011.<br />
Olweus, D. (1993) Bully/victim problems among schoolchildren: Long-term consequences and an effective<br />
intervention program. In S. Hodgins, Mental disorder and crime (pp. 317-349). Thousand Oaks, CA: Sage.<br />
Parkes, K. R. (1991) Locus of control as moderator explanation for additive versus interactive findings in the<br />
demand-discretion model of work stress? British Journal of Psychology, 82, 291 - 312.<br />
Salmivalli, C., Lagerspetz, K., Bjorkqvist, K., Osterman, K., & Kaukiainen, A. (1996). Bullying as a group process:<br />
Participant roles and their relations to social status within the group. Aggressive Behavior, 22, 1–15.<br />
Steenbarger, B.N. (1991) . All the world is not a stage. Journal of Counselling , 70(Nov-Dec), 228-296.<br />
Steiner, J. (1993). Psychic Retreats. London: Routledge.<br />
Steiner, J. (2011) Seeing and Being Seen. East Sussex/New York: Routledge.<br />
Stokes, J. (1994) The unconscious at work in groups and teams. In Obholzer, A. Roberts, V.Z. The Unconscious<br />
at Work. East Sussex/New York: Routledge, 1994.<br />
Symington, N. (1994) Emotion and Spirit. London: Karnac.<br />
Wenger, E. (2000) Communities of Practice and Social Learning Systems. Organization 7(2): 225-246<br />
Žižek, S. (2006) Is this digital democracy, or a new tyranny of cyberspace? In: Guardian. December 30, 2006.<br />
478
Learning in Smart Environments – From Here to There<br />
Peter Mikulecky<br />
University of Hradec Kralove, Hradec Kralove, Czech Republic<br />
peter.mikulecky@uhk.cz<br />
Abstract: Recently, ten years after the Information Society Technologies Advisory Group (ISTAG) Report<br />
(Ducatel et al 2001) was issued with several visions of smart environments for various purposes, we could<br />
already evaluate the results achieved. One of the original scenarios by ISTAG was the Scenario 4: Annette and<br />
Solomon in the Ambient for Social Learning. The vision presented in the scenario started an intensive research<br />
resulting in a number of projects aiming at design and utilization of various smart <strong>learning</strong> environments. In the<br />
paper, which is based on our ongoing research, we wish to remind several recent important attempts in the area<br />
of smart environments designed for <strong>learning</strong>.<br />
Keywords: smart environments, ambient intelligence, ubiquitous <strong>learning</strong>, scenarios<br />
1. Introduction<br />
The famous ISTAG Report started from 2001 a decade of various research initiatives covered by the<br />
recently very successful area of Ambient Intelligence (AmI). The AmI vision can be applied to a<br />
number of diverse application environments, varying from homes, offices, or cars to homes for the<br />
elderly and hospitals. It refers to a wide range of human emotional and intellectual needs, from<br />
comfort, pleasure, and entertainment to safety, security, and health. AmI scenarios provide examples<br />
of such environments. They are included in the ISTAG report (Ducatel et al 2001) that introduced four<br />
scenarios of the future development in information society.<br />
One of the ISTAG scenarios – Scenario 4: Annette and Solomon in the Ambient for Social Learning –<br />
brought a vision of a smart <strong>learning</strong> environment, based on a position that <strong>learning</strong> is a social<br />
process. According to the original description in the ISTAG report, the Ambient for Social Learning<br />
(ASL) is an environment that supports and upgrades the roles of all the actors in the <strong>learning</strong> process,<br />
starting with the roles of the mentor and the students as most concerned parties. The systems that<br />
make up the ASL are capable of creating challenging and interacting <strong>learning</strong> situations that are<br />
codesigned by the mentor and students in real-time. Students are important producers of <strong>learning</strong><br />
material and create input for the <strong>learning</strong> ‘situations’ of others. In other words, the ASL is both an<br />
environment for generating new knowledge for <strong>learning</strong> and a ‘place’ for <strong>learning</strong> about <strong>learning</strong><br />
(Ducatel et al 2001, p. 7).<br />
Recently, ten years after the ISTAG Report was issued with several similar visions of smart<br />
environments for various purposes, we could already evaluate the results achieved. An intensive<br />
research has been provided since that time with a number of interesting results. However, recent<br />
experience with new achievements in the area of smart classrooms and other similar smart<br />
environments seems not to be always satisfactory.<br />
Smart environments for <strong>learning</strong>, as a result of intensive research in the area of Ambient Intelligence,<br />
deserve also attention of the large community oriented on e-<strong>learning</strong> and technology enhanced<br />
<strong>learning</strong>. Smart environments could be naturally considered a new degree of computer enhanced<br />
<strong>learning</strong>, with a number of new facilities. The area of Ambient Intelligence can be studied from several<br />
perspectives. As (Bureš, Čech, and Mls 2009) pointed out, besides its technological perspective,<br />
social perspective, or ethical perspective, we can also identify an educational perspective. The<br />
educational perspective deals with problems and challenges related to proper education in relevant<br />
AmI areas.<br />
In the paper we intend to map the recent state of the art in the area of smart environments designed<br />
for <strong>learning</strong>. On the basis of our ongoing research we wish to present a number of open problems and<br />
possible ways to their solutions, as well as a critical view on the research of smart environments for<br />
<strong>learning</strong>.<br />
2. Context-aware ubiquitous <strong>learning</strong><br />
The context aware and ubiquitous <strong>learning</strong> as being naturally close to the educational perspective of<br />
AmI as well as to the idea of smart <strong>learning</strong> environments, was defined and studied by several<br />
authors. (Winters et al. 2005) pointed out that ubiquitous computing has tremendous potential for<br />
479
Peter Mikulecky<br />
framing <strong>learning</strong>, particularly in informal and socially constructed contexts. To reach this potential<br />
according to (Winters et al. 2005) it is necessary for the current desktop-focus development of<br />
technology in education to be challenged through the design, development and testing of new<br />
ubiquitous prototypes for <strong>learning</strong>.<br />
For instance, according to (Yang et al 2008), context-aware and ubiquitous <strong>learning</strong> is a computer<br />
supported <strong>learning</strong> paradigm for identifying learners' surrounding context and social situation to<br />
provide integrated, interoperable, pervasive, and seamless <strong>learning</strong> experiences. The objective of<br />
context-aware and ubiquitous <strong>learning</strong> is to enhance Web-based <strong>learning</strong> a step further from <strong>learning</strong><br />
at anytime and anywhere to <strong>learning</strong> enabled at the right time and the right place using right<br />
resources and right collaborators. Alternatively, according to (Hwang et al 2009) context-aware<br />
ubiquitous <strong>learning</strong> is an innovative approach that integrates wireless, mobile, and context-awareness<br />
technologies to detect the situation of learners in the real world and provide adaptive support or<br />
guidance accordingly.<br />
Already mentioned authors (Yang et al 2008) summarize the characteristics of context-aware and<br />
ubiquitous <strong>learning</strong> in the following eight aspects: mobility, location awareness, interoperability,<br />
seamlessness, situation awareness, social awareness, adaptability, and pervasiveness. More detailed<br />
descriptions of these aspects are as follows:<br />
Mobility: The continuousness of computing while learners move from one position to another.<br />
Location awareness: The identification of learners’ locations.<br />
Interoperability: The interoperable operation between different standards of <strong>learning</strong> resources,<br />
services, and platforms.<br />
Seamlessness: The provision of everlasting service sessions under any connection with any<br />
device.<br />
Situation awareness: The detection of learners’ various situated scenarios, and the knowledge of<br />
what learners are doing with whom at what time and where.<br />
Social awareness: The awareness of learners’ social relationship, including what do they know?<br />
What are they doing at a moment? What are their knowledge competence and social familiarity?<br />
Adaptability: The adjustability of <strong>learning</strong> materials and services depending on learners’<br />
accessibility, preferences, and need at a moment.<br />
Pervasiveness: The provision of intuitive and transparent way of accessing <strong>learning</strong> materials and<br />
services, predicting what learners need before their explicit expressions.<br />
On the other hand, (Bomsdorf 2005) considered ubiquitous <strong>learning</strong> as the next step in performing e<strong>learning</strong><br />
and by some authors it was expected to lead to an educational paradigm shift, or to new<br />
ways of <strong>learning</strong>. The potential of ubiquitous <strong>learning</strong> results from the enhanced possibilities of<br />
accessing <strong>learning</strong> content and computer-supported collaborative <strong>learning</strong> environments at the right<br />
time, at the right place, and in the right form. Furthermore, and this is close to the ideas of AmI<br />
presented by (Ducalet et al. 2001), it enables seamless combination of virtual environments and<br />
physical spaces. Ubiquitous computing leads to ubiquitous <strong>learning</strong> that allows embedding of<br />
individual <strong>learning</strong> activities into everyday life. As Bomsdorf (2005) mentioned, early applications of<br />
ubiquitous <strong>learning</strong> were tourist and museum guides, by means of which a visitor gets information<br />
based on his current position, e.g., facts about a painting he is standing in front of. In most ubiquitous<br />
<strong>learning</strong> approaches the physical environment is directly (semantically) related to <strong>learning</strong> objectives<br />
and activities (e.g., the museum visitor gets information each time based on his current location). As it<br />
was already stressed, the fundamental issue in a ubiquitous <strong>learning</strong> environment is how to provide<br />
learners with the right material at the right time in the right way. Context aware adaptation is therefore<br />
indispensable to all kinds of <strong>learning</strong> activities in ubiquitous <strong>learning</strong> environments.<br />
In addition to the context-aware ubiquitous <strong>learning</strong> characteristics by (Yang et al 2008) mentioned<br />
earlier in this chapter, (Hwang et al 2008) formulated the potential criteria of a context-aware<br />
ubiquitous <strong>learning</strong> environment as follows:<br />
It is context-aware; that is, the learner’s situation or the situation of the real-world environment in<br />
which the learner is located can be sensed, implying that the system is able to conduct the<br />
<strong>learning</strong> activities in the real world.<br />
480
Peter Mikulecky<br />
It is able to offer more adaptive supports to the learners by taking into account their <strong>learning</strong><br />
behaviours and contexts in both the cyber world and the real world.<br />
It can actively provide personalized supports or hints to the learners in the right way, in the right<br />
place, and at the right time, based on the personal and environmental contexts in the real world,<br />
as well as the profile and <strong>learning</strong> portfolio of the learner.<br />
It enables seamless <strong>learning</strong> from place to place within the predefined area.<br />
It is able to adapt the subject content to meet the functions of various mobile devices.<br />
As (Hwang et al 2008) pointed out, researchers have different views of the term “ubiquitous <strong>learning</strong>”<br />
till now. A popular view is “anywhere and anytime <strong>learning</strong>”, which is a very broad-sense definition of<br />
ubiquitous <strong>learning</strong>. With this definition, any <strong>learning</strong> environment that allows students to access<br />
<strong>learning</strong> content in any location at any time can be called a ubiquitous <strong>learning</strong> environment, no<br />
matter whether wireless communications or mobile devices are employed or not. From this viewpoint,<br />
the mobile <strong>learning</strong> environment which allows students to access <strong>learning</strong> content via mobile devices<br />
with wireless communications is a special case of the broad-sense definition of ubiquitous <strong>learning</strong>.<br />
However, if we took into account the ISTAG scenario Annette and Solomon in the Ambient for Social<br />
Learning that could serve as an ideal case for a smart <strong>learning</strong> environment, which was undoubtedly<br />
context-aware as well as ubiquitous at the same time, the popular view of “anywhere and anytime<br />
<strong>learning</strong>” should be considered as impractically broad. As (ElBishouty et al 2010) pointed out, the<br />
challenge in the information-rich world is not to provide information at anytime and at anywhere but to<br />
say the right thing at the right time in the right way to the right person.<br />
This approach is supported also by (Yang 2006, p. 188), stating that a ubiquitous <strong>learning</strong><br />
environment provides an interoperable, pervasive, and seamless <strong>learning</strong> architecture to connect,<br />
integrate, and share three major dimensions of <strong>learning</strong> resources: <strong>learning</strong> collaborators, <strong>learning</strong><br />
contents, and <strong>learning</strong> services. Therefore ubiquitous <strong>learning</strong> is characterized by providing intuitive<br />
ways for identifying right <strong>learning</strong> collaborators, right <strong>learning</strong> contents and right <strong>learning</strong> services in<br />
the right place at the right time.<br />
The main characteristics of ubiquitous <strong>learning</strong> are permanency, accessibility, immediacy,<br />
interactivity, and situating of instructional activities. (ElBishouty et al 2010) mentioned that the<br />
ubiquitous environment should be personalized according to the learner’s situation. They defined<br />
personalization as the way in which information and services can be tailored in a specific way to<br />
match the unique and specific needs of an individual user. While a learner is doing <strong>learning</strong> task or<br />
activity, she usually looks for some knowledge. In a ubiquitous <strong>learning</strong> environment, as (ElBishouty<br />
et al. 2010) stressed, it is very difficult for a learner to know who has this knowledge even though they<br />
are at the same place. In this case, the learner needs to be aware of the other learners’ interests that<br />
match his request.<br />
There are only a few studies that have attempted to induce the educational affordances of contextaware<br />
ubiquitous <strong>learning</strong> environment. (Liu and Chu 2009) devoted attention to the problem of what<br />
educational affordances can be provided by a context-aware ubiquitous <strong>learning</strong> environment. They<br />
proposed a system named EULER that can provide eight educational affordances: knowledge<br />
construction, apply, synthesis, evaluation, interactivity, collaborative <strong>learning</strong>, game-based <strong>learning</strong>,<br />
and context-aware <strong>learning</strong>. Moreover, they stressed that ubiquitous <strong>learning</strong> provides context-aware<br />
information and self-<strong>learning</strong> opportunities for learners. Therefore, it not only enables students to<br />
achieve <strong>learning</strong> goals anytime and anywhere, it is also cultivating their ability to explore new<br />
knowledge and solve problems. This should be considered to be one of most important characteristics<br />
of ubiquitous <strong>learning</strong>.<br />
Interesting ideas about <strong>learning</strong> in smart environments can be found in (Winters et al. 2005).<br />
According to them, <strong>learning</strong> is no longer viewed only as a form of delivered instruction, undertaken<br />
within the confines of traditional educational environments. It is now understood as a social process<br />
that happens at a time and place of the learner's choosing, continuing throughout one’s life. It is<br />
collaborative, evolving and informed by a process of self-paced development.<br />
(Winters et al. 2005) define a smart environment as any space where ubiquitous technology informs<br />
the <strong>learning</strong> process in an unobtrusive, social or collaborative manner. Thus a smart environment can<br />
be an ‘aware’ room or building, capable of understanding something about the context of its<br />
481
Peter Mikulecky<br />
inhabitants or workers; it can be a digitally enhanced outdoor space – park, cityscape or rural<br />
environment; or it can be the environment created when peoples’ meetings or interactions are<br />
augmented by wearable devices. These ideas are very close to that of original Scenario 4: Annette<br />
and Solomon in the Ambient for Social Learning from the ISTAG Report (Ducatel et al 2001).<br />
3. Examples of ubiquitous <strong>learning</strong> environments<br />
There is a number of interesting attempts to propose ubiquitous <strong>learning</strong> environments of the type<br />
discussed in previous chapters. For instance (Chen, Kinshuk, Wei, and Yang 2008) proposed a<br />
wireless communication based network called GroupNet. It is a Group Area Network that is proposed<br />
on the basis of P2P wireless network connection to fit with this type of mobile scenario. GroupNet<br />
consists of a set of interconnecting handheld devices with wireless access, carried by a group of<br />
people within proximity.<br />
GroupNet works with wireless modules of the handheld devices to achieve the best of ubiquitous<br />
networking. Ubiquitous networks enable secure access to data from everywhere on multiple devices<br />
to achieve the ubiquitous <strong>learning</strong> environment. The ubiquitous <strong>learning</strong> environment can connect,<br />
integrate and share <strong>learning</strong> resources in the right place at the right time by an interoperable,<br />
pervasive and seamless <strong>learning</strong> architecture. P2P networking used in GroupNet is one approach of<br />
creating ubiquitous networks for supporting ubiquitous <strong>learning</strong>.<br />
Another interesting proposal of intelligent <strong>learning</strong> environments published (Mhiri and Ratté 2009).<br />
They proposed an intelligent environment for human <strong>learning</strong> (the AARTIC project) that assists<br />
software engineering students in their assignments. The system resolve real problems: for the<br />
students, too much time to complete each assignment, for the teacher, too many students to offer any<br />
personalized help. Moreover, because students find themselves in a precarious situation (the<br />
concepts are new and complex), they rely on old primary reflexes: zero collaboration or planification.<br />
The proposed system aims to help the student in the understanding of concepts by suggesting<br />
examples. Two pedagogical agents compose the adaptive aspect of the system. The first monitors<br />
students’ activities in the environment. The second allows the teacher to observe the performance of<br />
each student and of the class as a whole. The environment also emphasizes collaboration.<br />
(ElBishouty et al 2010) present a model of personalized collaborative ubiquitous <strong>learning</strong> environment<br />
in order to support learners doing <strong>learning</strong> tasks or activities. It utilizes RFID tags to detect the<br />
surrounding physical objects and provides personalized recommendations based on the detected<br />
objects. It provides the learner with social knowledge awareness map for the peer helpers. The map<br />
visualizes the learners’ surrounding environmental objects, peer helpers and the strength of the<br />
relation in the social network perspective. The learner can contact, interact, and collaborate with the<br />
peer helpers to address the <strong>learning</strong> goal.<br />
Another example of a different approach towards intelligent <strong>learning</strong> environments presented (Winters<br />
et al. 2005). They defined an intelligent environment as any space where ubiquitous technology<br />
informs the <strong>learning</strong> process in an unobtrusive, social or collaborative manner. In their paper, two<br />
ubiquitous devices for use in such an environment were presented: the Experience Recorder and the<br />
iBand. The Experience Recorder is an embedded system that records the paths travelled by users –<br />
i.e. trails – in a particular place, for example at a museum or trade fair. It then recreates this visit in<br />
digital form, for example as a personalised website, enhanced for <strong>learning</strong>. The iBand is a wearable<br />
bracelet-like device that exchanges information about its users and their relationships during a<br />
handshake. (Winters et al 2005) stressed that the challenge of ubiquitous computing was to design<br />
and build systems for augmenting human capabilities rather than to replace them. In the context of<br />
<strong>learning</strong>, any ubiquitous computing tool cannot be viewed as deskilling the user. It must encourage<br />
skills development in a manner in which the learner is comfortable and engaged with. We cannot<br />
agree more.<br />
The last example of a smart environment is the system ISABEL described in (Garruzzo et al., 2007).<br />
The ISABEL is a new sophisticated multi-agent e-<strong>learning</strong> system, where the basic idea is in<br />
partitioning the students in clusters of students that have similar profiles, where each cluster is<br />
managed by a tutor agent. When a student visits an e-<strong>learning</strong> site using a given device (say, a<br />
notebook, or a smart phone), a teacher agent associated with the site collaborates with some tutor<br />
agents associated with the student, in order to provide him with useful recommendations. Generally,<br />
these systems use a profile of the student to represents his interests and preferences, and often<br />
482
Peter Mikulecky<br />
exploit software agents in order to construct such a profile. More in particular, each student is<br />
associated to a software agent which monitors his Web activities, and when the student accesses an<br />
e-<strong>learning</strong> site, his agent exploits the student’s profile interacting with the site. In this interaction, the<br />
site can use both content-based and collaborative filtering techniques to provide recommendations to<br />
the student’s agent by adapting the site presentation.<br />
4. On the way to the Annette and Solomon scenario<br />
Quoting (Weiser 1991), the most profound technologies are those that disappear. They weave<br />
themselves into the fabric of everyday life until they are indistinguishable from it. As a matter of fact,<br />
ubiquitous <strong>learning</strong> could be a good example of such a technology.<br />
Without any doubts any AmI application bringing new ideas and approaches into educational process<br />
at every level of education deserves a special attention. One of educational applications is the Smart<br />
Classroom project (Shi et al. 2010). It aims to build a real-time interactive classroom with teleeducation<br />
experience by bringing pervasive computing technologies into traditional distance <strong>learning</strong>.<br />
The goal of Smart Classroom project is to narrow the gap between the teacher’s experience in teleeducation<br />
and that in the traditional classroom education, by means of integrating these two currently<br />
separated education environments together. The used approach was to move the user interface of a<br />
real-time tele-education system from the desktop into the 3D space of an augmented classroom<br />
(called Smart Classroom) so that in this classroom the teacher could interact with the remote students<br />
with multiple natural modalities just like interacting with the local students.<br />
A more general overview of the AmI possibilities in education brings our recently published book<br />
chapter (Mikulecky et al. 2011). The objective of the paper is to identify and analyze key aspects and<br />
possibilities of AmI applications in educational processes and institutions (universities), as well as to<br />
present a couple of possible visions for these applications. A number of related problems are<br />
discussed there as well, namely agent-based AmI application architectures. Results of a brief survey<br />
among optional users of these applications are presented as well. The conclusion of this research<br />
was that introduction of Ambient Intelligence in educational institutions is possible and can bring us<br />
new experiences utilizable in further development of AmI applications.<br />
The scenario Annette and Solomon was considered in the time of its origin as a long term future.<br />
However, we presented a lot of examples and arguments in favour of the idea, that the scenario can<br />
be nowadays implemented, as the relevant technology has matured enough.<br />
5. Conclusions<br />
Taking advantage of AmI technologies educational institutions can become real “knowledge centres”,<br />
which are formed by intelligent applications, devices, or technologies. Such a technologically intensive<br />
system, whose significant attribute is an intelligent interaction with its users, can support following<br />
activities:<br />
Plan classroom instructions in a way that increases individual attention in critical areas,<br />
Identifying training needs for teachers,<br />
Aim to develop competence (opposed to awarding certificates, which cannot be an end in itself),<br />
Enable students to learn in a way that promotes interest in <strong>learning</strong> & continue <strong>learning</strong> in<br />
environments other than the institute which imparts formal education etc.<br />
The idea of enhancing educational environment by suitably chosen AmI solutions has been already<br />
sketched e.g. in (Mikulecky 2009) and for the case of e-<strong>learning</strong> education more elaborated in a<br />
number of sources we mentioned (see e.g. (Bomsdorf 2005), (Bureš, Čech, and Mls 2009), (Mhiri and<br />
Ratté 2009) and others). In this paper, we presented a whole range (from here to there) examples of<br />
smart environments, which certainly can be implemented and used by any modern university which<br />
uses nowadays information and communication technology. The basic principle of any further<br />
development lies, apart of its technological basis, also in understanding and using new approaches,<br />
new ways how these technologies can be utilized. We are deeply convinced, that AmI solutions are,<br />
among other recent approaches, the right way of achieving a new quality of educational process,<br />
which will be both motivating and effective for anyone.<br />
483
Acknowledgements<br />
Peter Mikulecky<br />
The research has been partially supported by the Czech Grant Foundation, grant No. P403/10/1310.<br />
References<br />
Bomsdorf, B. (2005) “Adaptation of Learning Spaces: Supporting Ubiquitous Learning in Higher Distance<br />
Education.”, Mobile Computing and Ambient Intelligence: The Challenge of Multimedia, Dagstuhl Seminar<br />
Proc. 05181, available on http://drops.dagstuhl.de/opus/volltexte/2005/371.<br />
Bureš, V., Čech, P., Mls, K. (2009) “Educational Possibilities in the Development of the Ambient Intelligence<br />
Concept”, Problems of Education in the 21 st Century, Vol. 13, No. 13, pp 25-31.<br />
Chen, N.-S., Kinshuk, Wei, C.-W., & Yang, S. J. H. (2008). „Designing a Self-contained Group Area Network for<br />
Ubiquitous Learning”. Educational Technology & Society, Vol. 11, No. 2, pp 16-26.<br />
Costello, R., Mundy, D.P. (2009) “The Adaptive Intelligent Personalised Learning Environment”, IEEE ICALT<br />
2009, IEEE Press, pp 606-610.<br />
Ducatel, K. et al. (2001) Scenarios for Ambient Intelligence in 2010: Final report. Seville, Spain, Information<br />
Society Technologies Advisory Group. Retrieved June 7, 2011, from<br />
ftp://ftp.cordis.lu/pub/ist/docs/istagscenarios2010.pdf.<br />
El-Bishouty, M. M., Ogata, H., Rahman, S., & Yano, Y. (2010). “Social Knowledge Awareness Map for Computer<br />
Supported Ubiquitous Learning Environment”. Educational Technology & Society, Vol. 13, No. 4, pp 27–37.<br />
Garruzzo, S., Rosaci, D., Sarné, G.M.L. (2007) ISABEL: A Multi Agent e-Learning System That Supports Multiple<br />
Devices. In: IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IEEE, New York, pp.<br />
485-488.<br />
Hwang, G.-J., Tsai, C.-C., & Yang, S. J. H. (2008) “Criteria, Strategies and Research Issues of Context-Aware<br />
Ubiquitous Learning”. Educational Technology & Society, Vol. 11, No. 2, pp 81-91.<br />
Hwang, G.-J., Yang, T.-C., Tsai, C.-C., Yang, S.J.H. (2009) “A context-aware ubiquitous <strong>learning</strong> environment for<br />
conducting complex science experiments”. Computers & Education, Vol. 53, pp 402-413.<br />
Liu, T.-Y., Chu, Y.-L. (2009) “Educational Affordances of Ubiquitous Learning Environment for Natural Science<br />
Learning”. IEEE ICALT 2009, IEEE Press, pp 290-292.<br />
Mhiri, F., Ratté, S. (2009) “AARTIC: Development of an Intelligent Environment of Human Learning”. ITiCSE 09,<br />
Paris, ACM Press, p 359.<br />
Mikulecký, P. (2009) “Remarks on Ubiquitous Intelligent Supportive Spaces”. Recent Advances in Applied<br />
Mathematics and Computational and Information Sciences, Proc. of the 15th American Conference on<br />
Applied Mathematics and Proc. of the International Conference on Comp. and Information Sciences (Eds.<br />
Jegdic, K., Simeonov, P., Zafiris, V.), Vols. I and II, pp 523-528, WSEAS Press, Athens.<br />
Mikulecký, P., Olševičová, K., Bureš, V., Mls, K. (2011) “Possibilities of Ambient Intelligence and Smart<br />
Environments in Educational Institutions.” In: Handbook of Research on Ambient Intelligence and Smart<br />
Environments: Trends and Perspectives (Mastrogiovanni, F., Nak-Young Chong, eds.), IGI Global, Chap.<br />
29, Information Science Reference (IGI Global), pp 620-639.<br />
Shi, Y., Qin, W., Suo, Y., Xiao, X. (2010) “Smart Classroom: Bringing Pervasive Computing into Distance<br />
Learning”, In: Handbook of Ambient Intelligence and Smart Environments, Springer Science+Business<br />
Media, pp. 881-910.<br />
Weiser, M. (1991). “The computer for the twenty-first century”. Scientific American, September, 94-104.<br />
Winters, N., Walker, K., Rousos, D. (2005) “Facilitating Learning in an Intelligent Environment”. The IEE<br />
International Workshop on Intelligent Environments. Institute of Electrical Engineers, London, pp. 74-79.<br />
Yang, S.J.H. (2006) “Context Aware Ubiquitous Learning Environments for Peer-to-Peer Collaborative Learning”.<br />
Educational Technology & Society, Vol. 9, No. 2, pp 188-201.<br />
Yang, S. J. H., Okamoto, T., & Tseng, S.-S. (2008). „Context-Aware and Ubiquitous Learning (Guest Editorial)”.<br />
Educational Technology & Society, Vol. 11, No. 2, pp 1-2.<br />
484
Using Courseware for More Than Courses: You May<br />
Already Hold the Lease on a Versatile Virtual Meeting<br />
Space<br />
Karen Hughes Miller 1 and Linda Leake 2<br />
1<br />
University of Louisville School of Medicine, USA<br />
2<br />
University of Louisville Delphi Center for Teaching and Learning<br />
Louisville Kentucky, USA<br />
Karen.miller@louisville.edu<br />
Laleak01@louisville.edu<br />
Abstract: The University of Louisville (Kentucky, USA) is a large, urban institution on three separate campuses<br />
that includes undergraduate, graduate, and professional schools such as education, business, medicine, law, and<br />
engineering. The institution also offers staff/faculty development opportunities, corporate <strong>learning</strong>, and life-long<br />
<strong>learning</strong> for the community as a whole. Blackboard is our online course platform and every section of every<br />
course (face-to-face, online, and blended) is automatically issued a course site to be used at the discretion of the<br />
instructor. A few years ago, those of us who teach online realized that the “organizations” option in Blackboard<br />
had far more potential for student, faculty, and staff support than we were using. The “organizations” option<br />
provides a virtual meeting space that can be used to facilitate discussions, share documents, administer events,<br />
conduct quizzes and surveys, broadcast e-mails, link to media, wikis, and blogs, and track the type and<br />
frequency of activities by each participant. While none of these capabilities are unique to BlackBoard, what is<br />
unique is that in BlackBoard these capabilities are bundled so users have access to all capabilities in a<br />
controlled (password protected) environment. The two critical differences between “organization” sites and<br />
traditional “course” sites are (1) once an organization leader has been designated, the leader controls who may<br />
log into the site and the role they play; and (2) organization sites are not time-bound by the academic calendar.<br />
Admission to traditional course sites are limited by administrators to include only faculty assigned to the course<br />
and students enrolled in the course and the site usually expires at the end of the term, semester, or quarter. In<br />
Blackboard, organization sites have all of the functionality of course sites without the two major limitations;<br />
therefore we could use our imaginations, and either apply the functions as they were intended or repurpose them<br />
to meet other needs. The purpose of this paper is to share five brief case studies that illustrate the range of uses<br />
to which we have applied the BlackBoard organizations option to support students, faculty, and staff in hopes<br />
that these ideas may be transferable to other institutions. Each case shows at least one unique application.<br />
Although there are a few technical limitations to Blackboard organization sites, in general the benefits outweigh<br />
the limitations. We argue that when an institution uses fewer types of platforms (or other software packages);<br />
they reduce training time for developers and users and maximize the value delivered by the <strong>learning</strong> platform<br />
provider. Using a platform provided thru central administration also reduces costs to individual schools,<br />
departments, and programs as they need not purchase additional software to support their unique needs.<br />
Organizational <strong>learning</strong> also increases as departments learn from each other’s examples of how the<br />
organizations option can be applied.<br />
Keywords: non-traditional applications of <strong>learning</strong> platforms; student, faculty, staff support<br />
1. Case Study #1: Supporting a large interdisciplinary student <strong>learning</strong> event<br />
on a health sciences campus<br />
As the population of the USA becomes more racially and ethnically diverse, students in the health<br />
professions need more instruction to increase their cultural competency. Whether they plan to practice<br />
medicine or dentistry, they will not be able to deliver successful healthcare if they do not understand<br />
their patient’s cultural and/or religious frame-of-reference relating to health and wellness (Bonder,<br />
Martin, and Miracle, 2001; Mott, 2003). For the past five years, a half-day workshop has been offered<br />
on the University of Louisville Health Sciences Campus (U of L HSC) on Increasing Cultural<br />
Competence. The workshop is designed much like a professional conference with plenary sessions<br />
and breakout sessions that learners may select according to their interests. First year students from<br />
medicine, dentistry, and dental hygiene along with first year dental residents and first year audiology<br />
students are required to attend. To justify the instructional time and the budget dedicated to this<br />
annual event, it has been important to measure <strong>learning</strong> outcomes. For the past several years,<br />
students have completed pre- and post- attitude and knowledge surveys to ensure that they are<br />
benefiting from the instruction.<br />
A Blackboard organization is created each year to manage enrollment, session selection, learner<br />
satisfaction, and share background information and bios of the various speakers. The process begins<br />
485
Karen Hughes Miller and Linda Leake<br />
with the Schools of Medicine and Dentistry providing the list of eligible students and their User IDs to<br />
the event organizers. Students are then entered as “members” of the organization. (For some years,<br />
this has been as many as 400 learners, but still seems to be within the capacity for a Blackboard<br />
organization site to function well.) Using the site’s e-mail function, learners are contacted with an<br />
invitation to log in and select their preferences of presentations and discussions. Reminder e-mails<br />
are sent to ensure the presentation selections are made, and learners are assigned groups according<br />
to the sessions they select. The conference agenda is posted as a printable document, as are print-ityourself<br />
name badges, meal tickets, bus passes (when sessions are held off campus), and other<br />
conference essentials. Some speakers also enjoy sharing their PowerPoints or links to their web sites<br />
when the event has concluded so students can use it as an ongoing reference.<br />
Originally, students who came prepared with all their materials pre-printed were invited to the “fast”<br />
registration lines and students who came without were in the “slow” lines, but preparation seems to<br />
have improved over the years. Currently students’ ID badges are scanned electronically as they enter<br />
and this list can be compared to the “roster” from the Grade Center to ensure each student has<br />
participated.<br />
Several assessment strategies have been applied over the past five years including learner<br />
satisfaction surveys and content quizzes for specific sessions. Because we knew which sessions<br />
specific learners planned to attend, we could direct content quizzes to specific groups. For program<br />
evaluation purposes, the Grade Center provides a good record of which sessions were most attended<br />
and which session’s content quizzes scored the highest.<br />
Over the past five years there have been several staff members assigned to manage the site;<br />
therefore, several variations of Blackboard’s technical capabilities have been applied. But for each<br />
event, the use of Blackboard to administer this large face-to-face event for students from different<br />
schools and departments has been uniformly successful. Because there are so many Blackboard<br />
users on campus, there are many individuals who can offer advice and support to a new staff member<br />
assigned to this task. This would not be the case if we had elected to create an event website using<br />
software that required unique training. Students, because they are familiar with using Blackboard<br />
for their coursework, have no hesitation logging on to find the documents they need or to reply to<br />
quizzes and surveys. The course “banner” for the organization site is the same design as the cover for<br />
campus flyers and other publicity in order to help learners connect the site and the event, and all<br />
language in the site is designed to be as inclusive as possible to reflect the overall theme.<br />
The major limitation to using a Blackboard organization for this event (or for any other purpose) is<br />
that while quiz results are linked to specific learners and reported to the leader (instructor) as discreet<br />
data; survey results are anonymous but only available as aggregate data thus limiting the statistical<br />
analysis that can be applied. So, if you prefer having survey data as discrete data for in-depth<br />
analysis, you must create the survey as a quiz and sacrifice anonymity. You can certainly assure<br />
learners that their identities will be kept confidential, but that is somewhat different from true<br />
anonymity.<br />
2. Case #2: Preparing advanced students to be effective tutors<br />
Peer tutoring has become an important component of student support on many college campuses,<br />
but those who study this trend have realized that simply being knowledgeable with the content of a<br />
course does not equate to being able to tutor others successfully in that content. Teaching/training<br />
skills such as explaining content and responding to questions (Roscoe and Chi 2007, 2008) are skills<br />
that can be taught to tutors resulting in measurable improvements in their success.<br />
U of L’s undergraduate tutoring program REACH (Resources for <strong>Academic</strong> Achievement) has been<br />
internationally recognized. The program’s web site was awarded Third Place in the annual 2010<br />
Learning Support Centers in Higher Education (LSCHE) Website Excellence Award; and in 2009,<br />
REACH received both a Program of Excellence and the Tutor of Excellence by the Association for the<br />
Tutoring Professions (ATP). REACH recently received its three-year certification renewal from the<br />
College Reading and Learning Association (CRLA), and is one of the only about 260 Tutor Training<br />
programs worldwide to gain certification from the CRLA.<br />
At the beginning of each academic year, REACH holds two days of tutor training. The first day is<br />
spent orienting the tutors to the mission and goals of REACH, reviewing <strong>learning</strong> styles, and providing<br />
486
Karen Hughes Miller and Linda Leake<br />
general tutoring basics. The second day provides campus speakers and breakout sessions, where<br />
each center meets as a group and discusses their specific area and how to best meet the needs of<br />
the students who take advantage of that particular service. These two days not only provide<br />
information about tutoring with REACH, but also allow an opportunity for the tutors to begin to develop<br />
relationships with one another. An additional day of tutor training is held at the beginning of each<br />
spring semester. This one-day event continues the information from the two-day training. During the<br />
first part of the day, campus updates are discussed and tutoring basics are covered. Then, tutors<br />
break out into small groups for center-specific information for the second part of the day.<br />
In addition to attending the two-day training, all tutors are required to use the REACH Blackboard<br />
Tutor Training course (developed as an “organization”) to complete modules on Diversity Training and<br />
on Sexual Harassment. These modules are completed by each new tutor and then graded by REACH<br />
staff. When the modules are completed successfully, the tutor becomes eligible for the first level of<br />
certification and a slight pay increase. The complete Blackboard Tutor Training course is required<br />
for tutors who were unable to attend the face-to-face sessions or who were hired after this training<br />
took place. Modules on the Blackboard course cover the topics and basic information from training<br />
so that tutors will be given the same information as those who attended the new tutor training.<br />
Online tutor training has several advantages. As in all online <strong>learning</strong>, the Blackboard site provides<br />
a place for learners to return to specific content if they feel they need a “refresher” on certain topics.<br />
Having the modules online also standardizes the training being given to tutors from semester to<br />
semester, and provides a way to document the completion of specific modules that can earn the<br />
various levels of certification and the differential pay rates. The combination of face-to-face and online<br />
training creates a blended <strong>learning</strong> model for tutor training, and because the U of L access to<br />
Blackboard is thru the central University web portal, they are already situated to access the Virtual<br />
Math Center and the Virtual Writing Center when the students they are tutoring need specialized<br />
assistance. When necessary, tutors may be given guest privileges into a specific course site in<br />
Blackboard to assist with linked Supplemental Instruction (SI) study sessions.<br />
3. Case #3: Keeping medical faculty, students, and staff up to date with<br />
patient privacy training<br />
A very simple but effective application of a Blackboard organization site is keeping medical and<br />
nursing faculty, students, and staff up-to-date with their training and certification on patient privacy.<br />
The US statute, The Health Insurance Portability and Accountability Act of 1996 (HIPAA), includes<br />
several objectives relating to health insurance availability and medical record keeping. The aspects<br />
most relevant in a healthcare teaching and research environment are: (1) the basics of patient<br />
privacy, (2) HIPAA information security fundamentals, and (3) privacy when patients are the subjects<br />
of research. The U of L Privacy Office has developed Blackboard training courses and quizzes for<br />
each of these three essential areas, and, over the past three years, there have been more than 2,000<br />
individuals enrolled to take one or more courses. Because not every employee or student is required<br />
to take every course, but passing the required courses is their legal obligation, individual record<br />
keeping is very important. Blackboard uses the University’s User ID system; therefore it has<br />
reduced the complexity for the Privacy Office to track users by name. In Blackboard, users can not<br />
have more than one “record” of performance as can happen with some other systems.<br />
When someone’s certification in a specific topic is due for renewal, the course name will appear on<br />
their own personal Blackboard menu under Organizations. It is then their responsibility to take the<br />
course and pass the online quiz. Modules and quizzes are asynchronous and untimed so participants<br />
can work at their own pace. As with all Blackboard courses and organizations, grades are recorded<br />
in the central Gradebook, but participants are also encouraged to print out their own record of<br />
performance. One “cross check” to this system is that HIPAA training is verified when a faculty<br />
member or student is named as a researcher in any clinical study presented for approval to the<br />
Institutional Review Board (IRB).<br />
Regulatory training modules like this are usually developed by committees of content experts and in<br />
this case, the Privacy Office was able to develop its own pool of questions to meet their specific<br />
needs. The “question pool” feature in Blackboard allows the order of questions presented to any<br />
given learner to be randomized thus increasing the reliability of the score as a measure of their actual<br />
knowledge. While managing this training and testing process this is still a labor intensive activity,<br />
using a Blackboard organization site to deliver and monitor training gives the Privacy Office two key<br />
487
Karen Hughes Miller and Linda Leake<br />
advantages: the first is control over content and presentation since they did not need to rely on an<br />
external vendor/developer; and the second is control of learner identification since University User IDs<br />
are unique and learners are not likely to be confused or to have more than one record - an issue that<br />
had occurred when using other software.<br />
4. Case #4: Presenting online staff development with complex content<br />
One of the newest and most technologically advanced applications of a Blackboard organization<br />
site (with several course modules still under development) is an employee certification program on<br />
university business practices. As budgets for higher education become more stringent and regulation<br />
of incoming funds from research and other projects becomes more complex, there is increased<br />
demand on the skills of individual clerks and business managers within college and university<br />
departments. At U of L, the role of Unit Business Manager (UBM) has become highly specialized and<br />
complex so even those employees with degrees in accounting may not have the specific knowledge<br />
necessary to navigate the system. The Unit Business Training (UBT) curriculum was developed in<br />
consultation with senior unit business managers and university leaders. The course content is created<br />
with input from university subject matter experts and designed by the Delphi Center for Teaching and<br />
Learning. Employees with certain job titles and technical permissions within the financial system are<br />
required to complete the certification program.<br />
To become a certified UBM, an employee must successfully complete 18 hours of general electives<br />
focusing on soft-skills and technical training. Almost all content and testing is delivered online using a<br />
Blackboard organization with face-to-face sessions to enhance <strong>learning</strong>. Employees enrolled in a<br />
course must pass a quiz on each module with a score of 85% or higher with no more than three<br />
attempts permitted. (Participants who are not able to pass the test after 2 attempts receive additional<br />
coaching from the subject matter experts before they attempt the quiz again.) There is also a final quiz<br />
where a score of 85% or higher is required. Experienced employees may “test out” of a course by<br />
obtaining a score of 85% or higher on a comprehensive test in the course subject area. Both final<br />
course tests and the “test-out” examinations are administered outside of Blackboard. A single<br />
Blackboard organization site is used to house all courses (designed as modules) (see Figure 1) so<br />
all employee grades are recorded in one central Gradebook.<br />
Figure 1: The announcements screen of unit business training<br />
After completing a prerequisite introductory course called University 101, the required core courses<br />
focus on two general areas: “financial” systems and “people” systems. Examples of financial systems<br />
488
Karen Hughes Miller and Linda Leake<br />
courses include Financial Basics, Procuring and Paying for Goods and Services, and Sponsored<br />
Program Administration Core (for managing extramural funded research). Examples of people<br />
systems courses (HR) include Hiring and Compensation, Benefits, and Personnel Actions. Specialty<br />
electives, while once envisioned as part of the core curriculum, have recently been re-defined as a<br />
continuing education requirement to maintain certification. Examples of specialty electives include<br />
Advanced Sponsored Program Management, Revenue-Based Budgets, and Student Administration.<br />
The most challenging component of developing online courses in business or finance is the same<br />
regardless of the course being for academic credit or for staff development; in general, it is more time<br />
consuming to develop numbers-based content than it is to develop language based content for online<br />
<strong>learning</strong> (Chao and Miller 2002). However, some of the newer development tools located in<br />
Blackboard’s text boxes including MathML, WebEQ, and adding QuickTime, audio content,<br />
Flash/shockwave, and Mashups. All of these multi-media capabilities expand the range of options<br />
for sharing content. With these multi-media options, materials can be created that allow the<br />
participants to actually “walk through” or rehearse the process to fill out a needed form or process in<br />
real-time. Many software programs can be packaged in a format that can be uploaded into<br />
Blackboard to allow for more capabilities for interactivity to help the participants of the UBT program<br />
retain more knowledge of the related information. The information can then be available as a<br />
reference or resource for all the participants.<br />
5. Case #5: Blended orientation and graduate advising<br />
U of L’s Raymond A. Kent School of Social Work found a way to provide a blended approach to<br />
orientation for incoming students to their Master of Science in Social Work (MSSW) program as well<br />
as a way to communicate important advising information for all MSSW students. For ease of use, the<br />
Kent School organization site is divided into separate sections; one for Orientation and one for<br />
Advising. When administrators realized that using a Blackboard organization to support Orientation<br />
was actually saving approximately four hours in the required face-to-face sessions, they were able to<br />
reduce the sessions by half a day. As far as the Advising section of the site, it has saved countless<br />
advising staff hours answering basic advising questions via e-mail.<br />
The Orientation section of the site includes required readings and information on which students are<br />
quizzed using Blackboard’s testing feature. There are also optional readings and information as<br />
well as the information about the half-day face-to-face orientation session. The discussion board is<br />
used to create a dialogue among the new students on practical issues such as car pooling, looking for<br />
someone to share an apartment, as well as areas (or “forums”) to generate ideas for the<br />
organization’s upcoming priorities. The information on which students are quizzed is taken from the<br />
Kent School Catalog, and 8 out of 10 correct replies are required for successful completion.<br />
Since each student in the MSSW program has a specific course plan and specializations, the<br />
Orientation and Advising Staff have developed a truly unique application of the Blackboard Grade<br />
Center. To help students stay organized, each student’s My Grades page reminds them not only of<br />
which items they need to complete but also (after their first advising appointment) the name of their<br />
advisor, the program plan that the student has selected, and their specialization. If a student has been<br />
admitted with conditional status, the Grade Center also contains columns that list any pre-requisite<br />
courses the student may need to complete as well as if any documentation is needed such as<br />
transcripts for the student’s files. This innovative use of the Grade Center is possible because<br />
columns in the Grade Center accept text as well as numerals. Column content does not need to be<br />
included in a calculated “score”, so any type of relevant information can be included in column format.<br />
While the “master” Grade Center includes a complete spread sheet of student data, individual<br />
student’s My Grades page (see Figure 2) displays only their personal records.<br />
The Advising section of the site includes a list of curriculum plans for the current year, information on<br />
what to expect if pre-requisites are needed, a list of important dates, and the Kent School Phone<br />
Directory. Having all of this information easily at hand helps students get their questions answered<br />
without having to e-mail the advising staff. The Catalog, the <strong>Academic</strong> Calendar for the current three<br />
semesters, and the MSSW Student Handbook are also included under Advising. Once a student is<br />
assigned to an advisor, they are enrolled in a “Group” for that advisor within the organization. Groups<br />
include between 10 and 15 students. The Groups function in Blackboard allows each advisor to use<br />
tools such as Group E-mail, Group Discussion Board, and File Exchange to communicate with each<br />
of the students assigned to the advisor.<br />
489
Karen Hughes Miller and Linda Leake<br />
Figure 2: A “My Grades” page used as a personal advising record<br />
In addition to these innovative applications, the site contains information more typical of an<br />
“organization” site such as information about The Kent School Student Association (KSSA). This<br />
provides students the opportunity to network with peers and participate in activities each year that reenforce<br />
building a <strong>learning</strong> community within the school. Other important information at the site<br />
includes master syllabi for each class in a given semester, graduation information, and links to other<br />
campus sites including “Parking” for parking passes and “Bookstore” so that students can order books<br />
in advance.<br />
This site has become a single resource for all of the MSSW students to find the majority of information<br />
with regard to the school, program, and networking opportunities available. It has made the sharing of<br />
information much easier and has saved time and resources that Kent School can now use for other<br />
endeavors.<br />
6. Conclusion<br />
Blackboard organization sites have far more potential than simply supporting extracurricular student<br />
activities. If we consider each of Blackboard’s technical capabilities as platform (or software)<br />
capabilities rather than being limited by the current “label” or how they are typically used, we can see<br />
there are multiple opportunities to use an online organization to support faculty, student, and staff<br />
training and communications needs. As noted by Gautreau and Ahmed (2008), Blackboard is a far<br />
more robust platform than just a traditional <strong>learning</strong> management system. At U of L, we currently<br />
support 663 Blackboard organization sites and receive an average 20 requests per month to create<br />
new sites. The range of applications is still growing.<br />
The Delphi Center for Teaching and Learning under the auspices of the Office of the Provost supports<br />
<strong>learning</strong> technology needs on all three University campuses. They have simplified the process for<br />
faculty members and departments to be granted Blackboard organizational sites so it is done online<br />
and at no cost to the department. Student groups may also be given sites under the guidance of a<br />
faculty advisor. When a site has served its purpose, it may be archived, or portions of the site copied<br />
490
Karen Hughes Miller and Linda Leake<br />
into another site that is now filling that purpose. As with all <strong>learning</strong> platforms or other interactive<br />
software, there are some limitations, but we believe that overall the advantages make this a<br />
worthwhile endeavor (see Table1).<br />
Table 1: Positive and negative aspects of using the Blackboard organizations function for nontraditional<br />
applications<br />
Positives:<br />
The Blackboard platform includes a rich variety of<br />
functions bundled into one proprietary (password<br />
protected) venue.<br />
Most institutions’ contracts already include some form<br />
of “organization” application.<br />
Faculty, students, and staff are usually familiar with<br />
using the online platform and these skills can be<br />
applied to “organization” applications.<br />
Training and technical support can be the same for<br />
regular course application and organization<br />
applications.<br />
Success breeds success and other departments learn<br />
from example and develop new applications.<br />
Negatives:<br />
Users’ browser compatibility: occasional issues with<br />
newer browsers (Firefox 4 and Internet Explorer 9).<br />
Needing to obtain user IDs (passwords) for non<br />
college or university personnel.<br />
Not being able to access “discreet” data on<br />
participant replies when using the survey function<br />
(this is viable only in the “quiz” function and<br />
anonymity is sacrificed).<br />
Although all online <strong>learning</strong> platforms have become<br />
more user-friendly over the years, faculty and staff<br />
time must still be dedicated to site development and<br />
site maintenance.<br />
The term “thinking outside the box” is certainly overused, but in this case, if we can think outside the<br />
idea of a “course site” and imagine to what other purposes we can put all of that online functionality,<br />
we may find we have a wonderful new set of tools already at our fingertips.<br />
Acknowledgments<br />
Sincere thanks to Ronald M. Welch, Kentucky Area Health Education Centers (AHEC) for review of<br />
Case Study #1; to Cathy Liest, Executive Director of REACH, for review of Case Study #2; to Gwen<br />
Holland, Senior Compliance Analyst, U of L Privacy Office, for review of case #3; to Dr. Virginia<br />
Weber Denny, Assistant Director for Professional Development, and John Spence, Program Manager<br />
for Professional Development, Delphi Center for Teaching and Learning, for review of Case #4; and to<br />
Angela Napier, MSSW, ABD, Coordinator of <strong>Academic</strong> Affairs Graduate Programs, and Dr. Pamela<br />
Yankeelov, Associate Dean, Student Services, Raymond A. Kent School of Social Work, for review of<br />
case #5.<br />
References<br />
Bonder, B., Martin, L. and Miracle, A. (2001) “Achieving Cultural Competence: the Challenge for Clients and<br />
Healthcare Workers in a Multicultural Society. Generations, Vol 25, No. 1, pp 35-42.<br />
Chao, Y-C. and Miller, G. (2002) “Effective Math Online Learning (EMOL), Master Project Proposal.” Learning<br />
Design & Technology Program. Stanford University, 18 January, 2002. Advisor: Professor Jim Greeno.<br />
[online] http://ldt.stanford.edu/~chaoyc/project/emolproposal.pdf<br />
Gautreau, C. and Ahmed, S. (2008). “Blackboard Management and Professional Development Strategies to<br />
Augment Teaching and Learning”, MERLOT Journal of Online Learning and Teaching. Vol 4, No. 3. [online]<br />
http://jolt.merlot.org/vol4no3/gautreau_0908.pdf<br />
Mott, W.J. (2003) “Developing a Culturally Competent Workforce: A Diversity Program in progress.” Journal of<br />
Healthcare Management, Vol 48, No. 5 (Sep/Oct) pp 337-342.<br />
Roscoe, R.D. and Chi, M.T.H. (2007) “Understanding Tutor Learning: Knowledge-Building and Knowledge-Telling<br />
in Peer Tutor’s Explanations and Questions”, Review of Educational Research, Vol 77, No. 4, pp 543-574.<br />
Roscoe, R.D. and Chi, M.T.H. (2008) “Tutor <strong>learning</strong>: the Role of Explaining and Responding to Questions”,<br />
Instructional Science, Vol 36, pp 321-350.<br />
US Department of Health and Human Services (2003). Health Information Privacy: Research. [online]<br />
http://www.hhs.gov/ocr/privacy/hipaa/understanding/special/research/index.html<br />
491
An Analysis of Collaborative Learning as a Prevalent Instructional<br />
Strategy of South Africa Government eLearning<br />
Practices<br />
Peter Mkhize, Magda Huisman and Sam Lubbe<br />
University of South Africa, South Africa<br />
mkhizpl@unisa.ac.za<br />
madga.huisman@nwu.ac.za<br />
Sam.Lubbe@nwu.ac.za<br />
Abstract: A skills shortage seem to be engulfing the South African workplace in all economic sectors (Statistics<br />
South Africa 2010). In turn, increasing unemployment rates feed off such lack of critical skills, that cripples the<br />
productive capability of those who are suppose to be economically active. Subsequent to the 2001 parliamentary<br />
decision to migrate the government’s information system (IS) to an open source platform, the State Information<br />
Technology Agency has been faced with the task, among others, to facilitate training during and after the migration<br />
process. According to Department of Public Service and Administration (2006), different instructional delivery<br />
methods will be used – including eLearning, which allows for a lifelong <strong>learning</strong>, collaborative <strong>learning</strong>, flexible,<br />
just-in-time, just-in-context <strong>learning</strong> experience for users who are <strong>learning</strong> the new platform. In this paper, the<br />
researcher will determine the prevalent instructional strategy in relation to current instructional technology available<br />
to support eLearning practices in SITA. In doing so, the authors will employ grounded theory analysis techniques<br />
within a case study conducted in SITA – the custodian of the open source migration project (Charmaz<br />
2006). The grounded theory analysis technique allows the authors to create a relational networks of codes, as<br />
they emerge, for interview transcripts that will be interpreted to determine the instructional value of the instructional<br />
tools used in <strong>learning</strong> practices. In this study, an instructional strategy framework is developed based on<br />
code networks as emerged from grounded theory analysis. The result proves to favour collaborative <strong>learning</strong> as<br />
an efficient and effective instructional strategy to support eLearning practices in the public sector, given multiple<br />
constraints, towards realising a fully-fledged eLearning programme.<br />
Keywords: instructional design, instructional technology, instructional strategy, collaborative <strong>learning</strong>, eLearning<br />
and eLearning delivery modes<br />
1. Introduction<br />
After a parliamentary decision to migrate Government’s Information System (IS) to an open source<br />
platform, the project team decided on user training methods. One of these was eLearning. Meanwhile,<br />
the South African labour market has been engulfed by a skills shortage. It would thus be beneficial for<br />
those interested or struggling with facilitating skills training, to learn from eLearning practices as applied<br />
during the open source migration project.<br />
There are no reports or publications covering successful eLearning experiences in South Africa as<br />
there are reported successes in other countries (Bell 2007). In this paper, the authors investigate<br />
eLearning practices in the public sector, as used to impart skills to employees who are going to use<br />
the new open source systems. This is done by discussing relevant concepts as found in literature,<br />
and conducting an empirical investigation using the research method outlined in the research methodology<br />
section below. After that, a discussion of the results follows, incorporating a proposition emanating<br />
from the results discussion, and the conclusion of the study.<br />
2. Literature review<br />
In this paper, the authors decided to conduct a conceptual literature review, instead of the chronological<br />
review giving a short history of the eLearning experience, and studies that investigate eLearning<br />
practices in the South African context. The following sections unpack the conceptual underpinning of<br />
eLearning practices.<br />
2.1 eLearning in a dynamic socioeconomic environment<br />
Cross and Hamilton (2002) define eLearning as a <strong>learning</strong> process that takes place anywhere, any<br />
time, at the learner’s own pace. Acton and Golden (2003) add that eLearning is just-in-time education<br />
integrated into a high-velocity value chain. However, Asgharkani (2004) asserts that eLearning is<br />
aimed at replacing the old, traditional time/place/content predetermined <strong>learning</strong>, with a just-intime/artwork<br />
place/customised/on demand process of <strong>learning</strong>. This is <strong>learning</strong> made easy. According<br />
to Alsultanny (2006), eLearning programmes should be supported by management, in order to define<br />
492
Peter Mkhize et al.<br />
and align the vision and plan for <strong>learning</strong> in the organisational strategy, so that <strong>learning</strong> can be integrated<br />
into the organisation’s daily activities. The author further suggests that eLearning should be<br />
used to cultivate a culture of <strong>learning</strong> in the morning, and “do” in the afternoon.<br />
Basically, organisational training has been at the heart of organisational development and performance<br />
improvement, used to support skills acquisition and development for employees, and thereby<br />
facilitating the shift from traditional (or old) methods to new methods of doing business (Taran 2006).<br />
According to Alsultanny (2006), such training should be flexible in terms of place, time and pace, in<br />
order to accommodate the different <strong>learning</strong> styles preferred by learners/employees from differing<br />
educational backgrounds. This makes it necessary for an organisation to respond more quickly to the<br />
demands of the knowledge-based global economy with an ever-growing demand for <strong>learning</strong> new<br />
skills (Alsultanny 2006). eLearning could therefore contribute towards a solution to training problems –<br />
which are related to flexibility and access to developmental training programmes, in supporting organisational<br />
growth while sustaining productivity during training.<br />
The <strong>learning</strong> process should therefore no longer be confined to lecture rooms or training rooms, as<br />
long as the student or trainee has access to internet connectivity. This is affirmed by Hunter and Carr<br />
(2002), who also argue that eLearning is easily accessible to those who have time constraints and are<br />
thus unable to attend full-time, class-bound <strong>learning</strong> programmes.<br />
Even though eLearning is gaining in popularity, Li et al. (2003) argue that it dehumanises the <strong>learning</strong><br />
process, because students or trainees do not come into contact with the facilitator, thereby losing a<br />
sense of “belonging”. Meanwhile, Alsultanny (2006) points out the relevance and benefits of using<br />
eLearning in the training programme, where eLearning can be reactionary in dealing with the skills<br />
problems as they occur in a non-linear manner. Training programmes can then be accessible to employees<br />
in times of need, or as the need arises, at the trainee’s own pace. This is in line with the underpinning<br />
principles of eLearning as lifelong <strong>learning</strong>, flexibility of the <strong>learning</strong> process, collaborative<br />
<strong>learning</strong>, and easy access to <strong>learning</strong>. These principles are discussed in the following section.<br />
2.2 Lifelong <strong>learning</strong><br />
El-Khatib et al. (2003) reiterate the fact that the sustenance of the information economy is highly dependent<br />
on lifelong <strong>learning</strong>. Organisations and employees have to keep on <strong>learning</strong> to survive the<br />
revolutionary global competition and the explosion of information technology that incite the need to<br />
acquire new skills, knowledge and training. The prevalence of technology in the dynamic information<br />
economic conditions necessitates change in the workplace; likewise, skills have to be up to date (Bell<br />
2007; Hunter and Carr 2002). This could means that employees have to improve their skills levels<br />
with the latest skills, which are necessary to face the challenges of the new economic conditions and<br />
demands.<br />
Therefore, employees who want to be productive in their area of expertise must have access to training<br />
or <strong>learning</strong> programmes that would allow for just-in-time and just-in-context <strong>learning</strong>, to address<br />
the specific skills shortage. However, employers have to provide such an environment by investing in<br />
eLearning-based training systems that will allow employees to learn as long as they still need to improve<br />
their skills, thereby contributing to corporate intelligence (Cross 2004).<br />
Lifelong <strong>learning</strong> has been a maturing topic over the years in Europe (Bell 2007; Asgharkani 2004), as<br />
many organisations recognise the need to engage in ongoing <strong>learning</strong>. In order to realise the benefits<br />
of lifelong <strong>learning</strong>, organisations utilise eLearning-based training programmes that are flexible<br />
enough to accommodate personalised <strong>learning</strong> styles – unlike traditional training models (Cross and<br />
Hamilton 2002). Fry (2001) affirms that lifelong <strong>learning</strong> is becoming the only way to survive the everchanging<br />
global economy for organisations that want to sustain competitiveness and ascertain employees’<br />
productivity. This is even more relevant where the knowledge and skills lifespan is rapidly<br />
diminishing in the advent of new technology, which is then compelling organisations to change their<br />
production methods (Abdul-Salam et al. 2006; Acton and Golden 2003). Employers have to provide a<br />
lifelong <strong>learning</strong> environment that is timely, accessible and affordable for employees who want to upgrade<br />
their skills and knowledge in order to successfully fulfill their job responsibilities (Wiegel 2002).<br />
The eLearning model makes lifelong <strong>learning</strong> possible, as it is one of the intertwined underpinning<br />
principles of eLearning. Again, lifelong <strong>learning</strong> is more effective in a collaborative <strong>learning</strong> environment<br />
– which is discussed in the following section.<br />
493
2.3 Flexible and accessible <strong>learning</strong><br />
Peter Mkhize et al.<br />
Traditional classrooms have been, and still are, convenient and suitable for <strong>learning</strong> in the residential<br />
institutions. However, traditional <strong>learning</strong> is said to be unresponsive to urgent training needs that<br />
might require just-in-time and just-in-context <strong>learning</strong> aimed at solving specific work problems. In a<br />
flexible <strong>learning</strong> environment, students should be able to set out their own <strong>learning</strong> goals and pursue<br />
their <strong>learning</strong> objectives independently (Acton and Golden 2003). Students should also be able to<br />
learn to construct their own meanings by interpreting and making knowledge claims based on the<br />
<strong>learning</strong> material provided to them – thereby contributing to generation of knowledge.<br />
Asgharkani (2004) emphasises the importance of technology integration into <strong>learning</strong> to support the<br />
flexible <strong>learning</strong> strategy, so that <strong>learning</strong> material can be distributed to students using technology,<br />
and, specifically, web-based technology. The flexible <strong>learning</strong> strategy is technology intensive, because<br />
technology plays an integral role in facilitating asynchronous and synchronous modes of delivering<br />
<strong>learning</strong> material. This makes <strong>learning</strong> material accessible to students who would otherwise not<br />
be able to attend classes in real time because of other daily commitments (Hunter and Carr 2002).<br />
According to Asgharkani (2004), many companies are focusing training investments more towards<br />
online-based <strong>learning</strong> systems than on traditional classroom <strong>learning</strong> – that way, boosting turnaround<br />
time from <strong>learning</strong> to performance of tasks. Flexible <strong>learning</strong> could be facilitated well where students<br />
can collaborate in the development of new meaning, given appropriate instructional technology.<br />
2.4 Collaborative <strong>learning</strong><br />
The advent of internet transfer of information between individuals has been made easy through a multitude<br />
of ICT tools that have enabled interconnectivity between students in distant places, both asynchronously<br />
and synchronously (Hunter and Carr 2002; Abdul-Salam and Mansur 2006). The internet<br />
also enables global communication between students/trainees and experts in their field of specialisation<br />
(Matula 2004). As a result, students learn from each other by exchanging ideas, information and<br />
knowledge in online interactive platforms, both asynchronously and synchronously. The sharing of<br />
information and knowledge between students is done within communities of practice, where specialists<br />
and experts in the field share their experiences and <strong>learning</strong> from each other (Wenger and Snyder<br />
2000).<br />
Collaborative <strong>learning</strong> provides opportunities to critically engage with the <strong>learning</strong> material that is being<br />
discussed within collaborative groups (Drinkwater et al. 2004). Even though some of the eLearning<br />
programmes are not designed to enable a collaborative <strong>learning</strong> strategy, students nevertheless<br />
still enjoy some level of online interaction with the facilitator, experts in the industry, and other students.<br />
Cross and Hamilton (2002) assert that <strong>learning</strong> in the collaborative <strong>learning</strong> environment is<br />
student centred, rather than lecturer centred (which is prevalent in the traditional <strong>learning</strong> model). In<br />
addition to critical engagement, students learn by discovery – where they interrogate <strong>learning</strong> material<br />
to discover implied meaning and interpret it to formulate new valid conclusions.<br />
Leacock (2005) suggests that organisations should create an environment for skills development and<br />
training, that is continuously improved through open sharing of ideas, frequently, and regular evaluation<br />
of processes across different areas of the organisation, by collaborators. The facilitators and<br />
learners/employees engaging in the evaluation of the <strong>learning</strong> process thereby encourage development<br />
and/or subscription to a community of practice where employees can share relevant information<br />
about a technology-driven development in their industry.<br />
The collaborative <strong>learning</strong> environment strengthens situational <strong>learning</strong> that grants students access to<br />
the relevant training programmes addressing particular skills deficiencies directly linked to their work<br />
(Knowles 1984). Williams (2003) confirms that situational <strong>learning</strong> inculcates the “knowing how to act<br />
in practice” rather than “knowing about practice”. This is imperative for a <strong>learning</strong> organisation in the<br />
midst of technological changes in the global marketplace (Senger 1990).<br />
2.5 Student centeredness<br />
Cross (2004) maintains that eLearning systems provide continuous flexible access to <strong>learning</strong> programmes<br />
that are accessible at any time and anywhere, meeting diverse student requirements, as<br />
they have varying preferences as to time and location of <strong>learning</strong> delivery. This requires a student-<br />
494
Peter Mkhize et al.<br />
driven <strong>learning</strong> mechanism that would facilitate <strong>learning</strong> in ways that allows students to drive the pace<br />
and <strong>learning</strong> process (Knowles 1984).<br />
In a fast-pace, changing work environment it important for an organisation to develop training programmes<br />
that are sensitive to employees’ skills needs. Failure to align <strong>learning</strong> programmes with student/employee<br />
needs could lead to rejection of the <strong>learning</strong> programmes (Matlhape and Lessing,<br />
2002). eLearning provides a flexible and student-centred <strong>learning</strong> environment, enabling a responsive<br />
<strong>learning</strong> process in an attempt to close skills gaps that could be caused by changes in the work<br />
environment (Hunter and Carr 2002). In a student-centred <strong>learning</strong> environment, a student would be<br />
able to access <strong>learning</strong> programmes at any time of the year, directly, anywhere that they are situated,<br />
based on the current skills deficiencies (Cross 2004). Cross and Hamilton (2002) assure that studentcentric<br />
<strong>learning</strong> programmes are developmental and enriching, as students gain a competitive edge in<br />
the labour market over those who do not update their skills.<br />
Instructional designers of eLearning experiences need to assess student profiles in order to customise<br />
their offering. Customisation of the eLearning solution would enable students to learn in their own<br />
<strong>learning</strong> style (Hunter and Carr 2002). The student becomes the centre of the <strong>learning</strong> practice, as the<br />
facilitator should be there to facilitate the <strong>learning</strong> process. All the <strong>learning</strong> material is designed<br />
around the student, and they are encouraged to contribute towards the design of <strong>learning</strong> programmes.<br />
Williams (2003) assures that students will be interested in engaging <strong>learning</strong> material to<br />
learn even more abstract concepts on their own, where they have a sense of ownership of the <strong>learning</strong><br />
experience.<br />
Students are encouraged to engage <strong>learning</strong> material and deduct new meaning from <strong>learning</strong> experiences,<br />
which would enable them to learn through discovery (Li et al. 2001). Williams (2003) asserts<br />
that <strong>learning</strong> by discovery increases the sense of ownership of the <strong>learning</strong> process. The facilitator<br />
would not have to push students to learn new concepts, and students will pull all concepts that would<br />
add value to their career and life.<br />
Student centeredness complements collaborative <strong>learning</strong> as both a principal advocate for student<br />
ownership of the <strong>learning</strong> process, and an encouragement to facilitators to create a conducive <strong>learning</strong><br />
environment for discovery of new knowledge. Levinson (2004) suggests that the <strong>learning</strong> programmes<br />
developer should be familiar with the constructivist theory in order to promote student ownership<br />
of the <strong>learning</strong> process, as they could be assisted by the facilitator in discovering new knowledge.<br />
The constructivists’ epistemological perspective promotes co-creation of meaning and knowledge<br />
between the learner and the facilitator of the <strong>learning</strong> experience (Duffy and Cunningham 1996).<br />
In designing the eLearning platform, Li et al. (2003) argue that instructional designers should first<br />
consider or give priority to educational principles over technology infusion, in order to keep the content<br />
in context and relevant, instead of focusing attention on the technological components of eLearning.<br />
3. Research methodology<br />
This is in line with the objective of the study, which is to establish prevalent instructional strategy in<br />
the public sector, and to formulate propositions that can be used as guidelines for instructional design<br />
in the public sector. In doing that, the authors will try to gain an in-depth understanding of eLearning<br />
practice in the public sector, by employing a qualitative method of inquiry (Creswell 2007). The unit of<br />
analysis will be Government officials interested or working with eLearning in different departments. A<br />
grounded theory analysis method will be used to get a dense description of eLearning practice and to<br />
find the most prevalent/appropriate instructional strategy in the public sector (Charmaz 2006). Once<br />
the authors understand the eLearning practices in the public sector, they will be able explain how an<br />
eLearning designer could align different aspects and concepts into designing an effective eLearning<br />
environment.<br />
It is important to note that grounded theory is not applied as the research design in this paper, but as<br />
the method of analysis, because it sets out clearly-defined steps in analysing qualitative data. The<br />
analysis of data entails open coding that allows the researcher to let codes emerge from interview<br />
transcripts, by applying bracketing.<br />
495
3.1 Definition of case study<br />
Peter Mkhize et al.<br />
Case study is one of the government departments that is in charge of open-source migration from<br />
proprietary software. Among other training delivery mechanism is eLearning. This change happens in<br />
the midst of high unemployment rates due to skills shortages in South Africa. This department is also<br />
in charge of end-user training. This is disconcerting, because training initiatives have failed to serve<br />
training needs that existed before the latest requirement of open-source training.<br />
The authors interviewed six members of migration training programmes. They all work for the same<br />
departments, but consult in different departments that have already started to migrate. They are faced<br />
by different challenges, which they shared during interview sessions. The average interview duration<br />
recorded was 43 minutes. Each interview was transcribed and open coded before the subsequent<br />
interview, in order to grasp the exact situation, thereby adapting the interview schedule where necessary.<br />
After open coding comes axial coding – which allows the researcher to categorise open codes<br />
into families. This is a term used in AtlasTI, the qualitative data analysis software used to analyse<br />
data in this study. Subsequent to that is selective coding, where relationships between families are<br />
established; in this study, a network of codes is used to represent such relationships. Code networks<br />
are discussed in the discussion of findings.<br />
4. Discussion of findings<br />
Findings are discussed in relation to categories deduced from codes as they emerged from interview<br />
transcripts. Two major categories emerged during the analysis of data. These categories were formed<br />
by grouping related codes together into families, as is the term used in AtlasTI to describe groups of<br />
codes. Within these categories, codes were interconnected according their relationships with each<br />
other in the family, in order to create a story based on the interconnection (Creswell 2007; Charmaz<br />
2006). Two categories emerged with networked codes as discussed below.<br />
4.1 Management and administration of collaboration<br />
Collaborative <strong>learning</strong> emerged as the prevalent instructional strategy in the public sector. This category<br />
illustrates the setting up of collaborative forums, allows for flexible <strong>learning</strong> between collaborators<br />
as they define the <strong>learning</strong> process and what is to be learnt (Acton and Golden 2003). However,<br />
membership of collaborative forums is controlled, to ensure that everybody in the forum makes a<br />
meaningful contribution, and to facilitate the sharing of ideas around development and adoption of<br />
open sources software in the public sector. Since open sources adoption takes place in the initial<br />
stages of the migration process, most of the communication among participants in the collaboration<br />
forums is kept confidential; therefore, requests to join should be trusted.<br />
I think it’s just about the confidentiality of the information, say you are in government you<br />
typically want to control whoever joins cause you might discuss strategies, communication<br />
strategies, ...in that case you will firstly have the private network ... you might be able<br />
to invite external people but then you would qualify them ...<br />
Even though membership of collaborative forums is closed, there is a need to attract or invite expert<br />
people to fill knowledge and skills gaps in the group of collaborators. Sometimes, social networks are<br />
used to attract prospective collaborators who have something to contribute, as the administrator<br />
would scan the social space to find bloggers who seem to have some level of expertise in open<br />
source – or any other skills that is deemed useful to the group. This makes collaborative <strong>learning</strong><br />
even more accessible to those who are not employees in same department. Knowledge sharing is<br />
then extended to specialised experts.<br />
I would maybe do some research and see...the thinkers in that area and I would extend<br />
an invite to that somebody that writing a blog that’s...or start talking to that network and<br />
say who are the real experts in open source ... Stakeholders onto that network the multiplier<br />
effect, wow ...<br />
Proposition 1: Profile the audience of the intended <strong>learning</strong> programme, then establish instructional<br />
technology that is accessible to all participants in the <strong>learning</strong> or training initiative.<br />
496
Peter Mkhize et al.<br />
Figure 1: Management and administration of collaboration<br />
497
Peter Mkhize et al.<br />
Basically, social networks become an integral part of collaborative <strong>learning</strong>, even for a closed community<br />
of collaborators. Social networks are normally used because of their relaxed and informal <strong>learning</strong><br />
characteristics, where collaborators could learn from each other and advance their current knowledge<br />
and skills (Drinkwater et al. 2004). One of the participants suggested that social networks should be<br />
strategically integrated into a credible educational model instead of loosely infusing technology where<br />
it seems convenient, and then hoping that <strong>learning</strong> will take place.<br />
Social network is the good model to look for, I mean it’s more of a forum environment; it’s<br />
more, what is this? One of these educational models, constructivism...<br />
In formulating an instructional strategy, the instructional designer should avoid the temptation to get<br />
carried away by the potential of instructional technology, and then just transpose traditional instructional<br />
material into technology media. This could deprive learner/trainees of efficient <strong>learning</strong>, and result<br />
in chaos (Buhrmann 2003). One of the participants went on to suggest that there should at least<br />
be one facilitator to direct the <strong>learning</strong> process in the collaborative forum (Abdul-Salam and Mansur<br />
2006).<br />
have a central person that will be able to guide ... it is informal but also <strong>learning</strong> because<br />
the tool allows us to do that in a unified way then we’ll be able to collaborate at different<br />
levels. One way or the other would need to collaborate with other people in the specific<br />
area ...<br />
Proposition 2: Once instructional designers have profiled participants, they should evaluate different<br />
instructional strategies that could be applied in conjunction with the most relevant instructional technology<br />
accessible to most participants.<br />
Even though collaborative <strong>learning</strong> is student centred, there is still a need to monitor collaborative forums,<br />
to avoid chaos (Abdul-Salam and Mansur 2006). Participants suggested that there should be a<br />
facilitator who understands the <strong>learning</strong> objectives that have to be achieved, in order to declare the<br />
<strong>learning</strong> initiative a success. The following section discusses <strong>learning</strong> processes in a collaborative<br />
environment as experienced by participants in this study.<br />
4.2 Learning through discovery<br />
Collaborative <strong>learning</strong> in the public sector allows learners to be grouped, or to group themselves, into<br />
communities of practice where they take charge of their <strong>learning</strong> process (Wenger and Snyder 2000).<br />
This is in line with the principle of student centredness advocated by many authors in literature (Asgarskhani,<br />
2004; Acton and Golden, 2003; Alsultanny, 2006). Figure 2 shows a network of relationships<br />
between codes, that could be interpreted as elements of collaborative <strong>learning</strong>.<br />
Some of the participants in this study argued that instructors and learners should become co-creators<br />
of knowledge in the workplace, as they engage <strong>learning</strong> material according to the skills requirements,<br />
with the assistance of instructional designers (Knowles 1984). Instructional designers should be familiar<br />
with educational principles, such as the constructivist theory, in order to facilitate <strong>learning</strong> in the<br />
workplace and matching instructional technology (Hunter and Carr 2002). In the case of public sector<br />
training, a constructivist approach is used to design a <strong>learning</strong> experience that allows for co-creation<br />
of knowledge and the sharing of ideas within a community of practice (Duffy and Cunningham 1996).<br />
you look at the philosophy behind Moodle eeh it is social constructivism where you cocreate<br />
knowledge, so in one stage you need the instructional designer and knowledge of<br />
teaching presenting online lessons and exams and even on facilitators ...<br />
In communities of practice, the role of a facilitator is to ensure that progression of activities leads to<br />
achievement of <strong>learning</strong> objectives (Leacock 2005). Even though communities of practice are not<br />
formalised, there are groups of people sharing the same interests and expertise, interacting on a<br />
regular basis (Wenger and Snyder 2000). Their interactions are based on experiences and open<br />
source problem solving strategies, as some members of the community have faced some of the open<br />
source migration challenges.<br />
In community of practice would see people who share common interest in a particular<br />
field or something, they will be able to seat and collaborate, discuss, move forward and<br />
look at the development, ... management of knowledge and knowledge gathering that<br />
can be preserved ...<br />
498
Figure 2: Learning through discovery<br />
Peter Mkhize et al.<br />
499
Peter Mkhize et al.<br />
you’ve got your structured <strong>learning</strong> and unstructured <strong>learning</strong> so also contribute to that<br />
unstructured <strong>learning</strong> ...<br />
collaboration in terms of the different roles, to …, and that what’s exciting us, getting together<br />
to work out a solution for the government we’ve had business analysts, we’ve<br />
had the now the content developers ...<br />
Proposition 3: The collaborative <strong>learning</strong> strategy should be encapsulated within the community of<br />
practice where collaborators could share common interests, experiences and/or expertise, thereby<br />
<strong>learning</strong> from each other by using appropriate instructional technology.<br />
The primary objective of the communities of practice is to share experiences and knowledge, thereby<br />
empowering colleagues in business – especially in the specialised business environment (Wenger<br />
and Snyder 2000), which requires collaboration with experienced colleagues (Drinkwater et al. 2004).<br />
Knowledge and skills sharing could be cross functional within the public sector, where interaction is<br />
elevated through the levels of information sharing, to high-level competencies.<br />
we need to engage with a department, so department x whether it’s Arts and Culture or<br />
Home Affairs whatever, the SITA training department can engage with the department<br />
and then setup a development plan that takes people from the basic skills required, basic<br />
skills and basic knowledge through to the competence that you want in the end, ... to<br />
the point where they are fully competent for their job and beyond, moving to the next<br />
level ...<br />
Within communities of practice, <strong>learning</strong> takes place through discovery of the facts and behaviours<br />
that are useful to the improvement of current methods – to the extent that discovery incites changes in<br />
the behaviour of the community (Li et al. 2001). One of the participants went on to say that collaboration<br />
is a culture that has changed ways of doing business in the public sector. It has removed the silo<br />
effect in the public sector, to enable sharing of best practices, so that everyone can learn from one<br />
another.<br />
think the main thing about collaboration in government is the culture is pretty much …so<br />
everybody is busy with their own stuff, fighting their own battles and not knowing that another<br />
department..., so collaboration for us is about creating the tool that can connect<br />
these people so that they can form groups, networks of interests ...<br />
Communities of practice could be interconnected, to allow the transfer of specialised skills from one<br />
group to another, without formal coordinated <strong>learning</strong> programmes. However, this process is conducive<br />
to a self-directed <strong>learning</strong> environment where individual learners know what they want to learn.<br />
Such learners initiate the <strong>learning</strong> process that is directed at the achievement of situation-specific<br />
<strong>learning</strong>.<br />
so if you are in a social network ... there’s more than I think 14 groups already so they<br />
are communities of practice because one the e-government group, ... I can be a member<br />
of the government wide collaboration group as well as the e-government I’m actually a<br />
part of 3 groups one I’m administrators the other one are I’m members ...<br />
Proposition 4: Evaluate different social media to establish appropriateness for specific <strong>learning</strong> objectives,<br />
then adapt one found to be appropriate to instructional design.<br />
Self-directed learners would use any vehicle of knowledge transfer if it would help in achieving a specific<br />
<strong>learning</strong> objective as required by the learner to fulfil their job responsibilities or to improve performance.<br />
Communities of practice would then be useful for employees who want to gain an insight<br />
into a specific knowledge area, and then probe beyond what is currently available in the forum. Learning<br />
would take place without anxiety related to being in a classroom and getting ready to be tested.<br />
Employees could navigate their way into different communities of practice, and then learn collaboratively<br />
from peers.<br />
5. Conclusion<br />
Through grounded theory analysis, two themes emerged – namely: (1) Management and administration<br />
of collaboration, and (2) <strong>learning</strong> through discovery. Collaboration proved to be the prevalent instructional<br />
strategy in the public sector, as all participants agreed that collaborative <strong>learning</strong> would<br />
help in paving the way for massive skills development, both in the public and private sectors. In addition<br />
to that, participant agreed that collaboration would enable industry practitioners to develop the<br />
500
Peter Mkhize et al.<br />
industry by sharing information and knowledge with colleagues, thereby negotiating new meanings.<br />
The authors went on to make a proposition that will serve as a guideline for instructional design in the<br />
public sector.<br />
References<br />
Acton T. and Golden W. (2003). Training the knowledge worker: a descriptive study of training practices in Irish<br />
software companies. Journal of European Industrial Training, 27(3), pp. 137-146.<br />
Alsultanny Y. (2006). eLearning System Overview based on semantic Web. Electronic Journal of eLearning, 4(2),<br />
pp.111-118.<br />
Abdul-Salam Z. A. and Mansur A. (2006). MyGfL: A Lifelong Learning Platform for Malaysian Society. The Electronic<br />
Journal of eLearning, 4(1), pp. 7-14.<br />
Asgharkani M. (2004). The Need for Strategic Foundation for Digital Learning and Knowledge Management Solutions.<br />
Electronic Journal on eLearning pp. 31-42.<br />
Bell J. (2007). ELearning: your flexible development friend? Development and Learning in Organisations, 21(6),<br />
pp. 7-9.<br />
Charmaz J. (2006). Constructing Grounded Theory: A practical guide through qualitative analysis. Thousand<br />
Oaks, CA: Sage.<br />
Creswell J.W. (2007). Qualitative inquiry and research design: Choosing Among Five Approaches. Thousand<br />
Oaks, CA: Sage.<br />
Cross J. (2004). An informal history of eLearning. On the Horizon, 12(3), pp.103-110.<br />
Cross J. and Hamilton I. (2002). The DNA of eLearning, excerpt from Beyond eLearning. Accessed: March 26,<br />
2003 from www.internettime.com.<br />
Department of Public Service and Administration. (2006). Policy on Free and Open Source Software Use for<br />
South African Government. Accessed: 13 May 2008 from<br />
http://unpan1.un.org/intradoc/groups/public/documents/CPSI/UNPAN025432.pdf.<br />
Duffy T. M. and Cunningham D.J. (1996). Constructivism: Implications for the design and delivery of instruction.<br />
In: Jonassen D.H. (ed.) Handbook of Research for Educational Communications and Technology. New<br />
York: Macmillan Library Reference<br />
Drinkwater P.M., Adeline C.M., French S., Papamichail K.N. and Richards T. (2004). Adopting a Web-based Collaborative<br />
Tool to Support The Manchester Method Approach to Learning. Electronic Journal of eLearning,<br />
2(1), pp. 61-68.<br />
Fry K. (2001). ELearning market and providers: some issues and prospects. Education and Training, 43(5), pp.<br />
233-239.<br />
Glaser B. and Strauss A. (1967). The discovery of grounded theory. Chicago: Aldine.<br />
Hoffman E.C. (2002). Information and communications technology, virtual offices and telework. South African<br />
Journal of Information Management, 4(2) pp. 1-27.<br />
Horton W. and Horton K. (2003). ELearning tools and technologies: A consumer's guide for trainers, teachers,<br />
educators and instructional designers. London: Wiley.<br />
Hunter M.G. and Carr P. (2002). Technology in Distance Education: A Global Perspective To Alternative Delivery<br />
Mechanisms. Journal of Global Information Management, 10(2), pp. 52-56.<br />
Knowles M.S. (1984). Andragogy in action. San Francisco: Jossey-Bass.<br />
Leacock T. (2005). Building a sustainable eLearning development culture. The Learning Organization, 12(4), pp.<br />
355-367.<br />
Li R., Cheng S.T. and Willis R.J. (2001). A framework for the implementation of a collaborative flexible <strong>learning</strong><br />
environment for academic institutions”. In: Khosrow-Pour M. (ed.) Managing information technology in a<br />
global economy. Hershey: Idea Group Publishing, pp. 89-93.<br />
McCallin A.M. (2003). Designing a grounded theory study: some practicalities. Nursing in Critical Care, 8(5), pp.<br />
203-208.<br />
Matula S.M. (2004). Peculiarities of the digital divide in sub-Saharan Africa. Electronic library and information<br />
systems, 39(2), pp. 122-138.<br />
Matlhape M.G. and Lessing N. (2002). Employees in Total Quality Management. Acta Commercii, 2, pp. 21-34.<br />
Sambrook S. (2003). ELearning for small organisations. Education and Training, 45(9), pp. 506-516.<br />
Senge P.M. (1990). The Fifth Discipline: The Art and Practice of the Learning organisation. New York: Doubleday.<br />
Taran C. (2006). Enabling SME to deliver synchronous online training – practical guidelines. Campus-Wide Information<br />
Systems, 23(3), pp. 182-195.<br />
Wenger E.C. and Snyder W.M. (2000). Communities of Practice: The Organizational Frontier. Harvard Business<br />
Review, pp. 139-145.<br />
Williams R. (2003). Integrated Distributed Learning with just-in-context Knowledge Management. Electronic Journal<br />
of eLearning, 1(1), pp. 45- 50.<br />
501
Ideas for Using Critical Incidents in Oral Debriefing From a<br />
Business Strategy Simulation Game<br />
Jonathan Moizer and Jonathan Lean<br />
University of Plymouth, UK<br />
jmoizer@plymouth.ac.uk<br />
jlean@plymouth.ac.uk<br />
Abstract: Reflecting on events or experiences can be an important means by which <strong>learning</strong> is achieved.<br />
Specifically, a critical incident is some kind of significant occurrence that can stimulate <strong>learning</strong> through<br />
questioning what has happened and why. Essentially, critical incident analysis is concerned with reflecting on<br />
causes and impacts. An incident can be regarded as critical when the action taken has contributed to an<br />
outcome, either positive or negative. A series of incidents can be reflected upon in a debriefing setting using a<br />
structured approach to help to challenge people’s assumptions and promote personal and professional <strong>learning</strong>.<br />
Critical incidents are a good medium for achieving reflective <strong>learning</strong> on the nature of situational events.<br />
However, there are a number of reasons why it can be difficult to learn from incidents in the real world, and why<br />
therefore, simulation games may facilitate effective <strong>learning</strong> through the application of a critical incident method.<br />
This working paper seeks to present ideas on how critical incidents can be used by instructors in oral debriefing<br />
to stimulate <strong>learning</strong> in a business simulation gaming setting. This is based on a case study of using a total<br />
enterprise game named the Business Strategy Game to teach part-time, post-graduate / post-experience<br />
students. The students, in teams, took on the roles of Directors of a global corporation and played out strategies<br />
and decisions over a simulated period in the life of the company. Following this, students were orally debriefed<br />
and assessed on what they had learnt about strategy making from the critical incidents that surfaced in course of<br />
playing the game. Three sources of critical incidents were introduced during the debriefing, upon which the<br />
students could comment, namely: macro-environment, competitive environment and internal environment of each<br />
simulated company. Preliminary findings from the debriefings has indicated that using a critical incident method<br />
did provide significant opportunity for students to think causally and better reflect on the connections between<br />
actions and consequences under circumstances of gaming complexity. An agenda for a fuller qualitative analysis<br />
of the student debriefings is proposed.<br />
Keywords: critical incident, event, reflection, <strong>learning</strong> simulation game, business strategy game, oral debriefing<br />
1. Introduction<br />
Reflecting on events or experiences can promote <strong>learning</strong> to be achieved. A critical incident is a<br />
significant occurrence that can stimulate <strong>learning</strong> through questioning what has happened and why.<br />
Critical incident analysis is concerned with reflecting on causes of events and their impacts. A series<br />
of incidents can be reflected upon in a debriefing setting using a structured approach to help to<br />
challenge people’s assumptions and encourage <strong>learning</strong>. Although critical incidents can provide<br />
understanding about the nature and repercussions of discontinuities and events, there are a number<br />
of reasons why it can be difficult to learn from incidents in the real world, and why therefore,<br />
simulation games may facilitate effective <strong>learning</strong> through the application of a critical incident method.<br />
The aim of this working paper is to describe how critical incidents can be used within a business<br />
simulation gaming exercise to benefit student <strong>learning</strong>. The objectives are to:<br />
Explain how critical incidents can be harnessed to generate higher order <strong>learning</strong>.<br />
Describe a <strong>learning</strong> and assessment approach using a critical incident method.<br />
2. Background to study<br />
The critical incident technique (method) was originated by Flanagan (1954). It was developed as a set<br />
of procedures for collecting direct observations of human behaviour to assess an individuals’<br />
performance in problem solving situations. It was developed to help assess and classify US air force<br />
flying personnel’s behavioural competence. Since that time, the approach has evolved, and has been<br />
applied to a range of different situations to help develop <strong>learning</strong> and understanding. These include,<br />
amongst other applications, customer service encounters (Kemppainen, 2000; Edvardsson and Roos,<br />
2001; Roos, 2002), strategic innovation decisions (Moenaert et al, 2010), and entrepreneurial <strong>learning</strong><br />
processes (Cope and Watts, 2000; Cope, 2003; Taylor and Thorpe, 2004). Critical incidents have also<br />
been used for teaching and <strong>learning</strong> in higher education, often to help students to reflect on real<br />
workplace incident vignettes (Macfarlane, 2001; 2003) or business case studies (Gulezian, 1981).<br />
502
Jonathan Moizer and Jonathan Lean<br />
The critical incident method has long been used to help people to reflect on their practice and develop<br />
new understandings of the processes they are engaged with (Finch, 2010). Critical incidents are often<br />
unplanned and unanticipated and perceived to have the potential to influence major change. The<br />
<strong>learning</strong> generated from more unusual events can often be more transformational, in that individuals<br />
when faced with critical incidents need to rethink their behaviours, as their habitual ways can prove<br />
ineffective in the face of new conditions (Cope, 2003). Critical incidents provide a means for<br />
individuals to transform their <strong>learning</strong> from a lower-order and more routinised level, to a higher-order<br />
generative level.<br />
A number of structured and unstructured methods are available for collecting data on critical incidents<br />
(Kemppainen, 2000). Flanaghan (1954) originally advocated using interview or observation. More<br />
latterly, others have used more structured survey instruments (e.g. Johnston, 1995; Mallak et al,<br />
2003; Voss, 2009) to unpack critical incidents with individuals. The predominant methods for critical<br />
incident analysis are via interview or focus group. Edvardsson and Roos, (2001) suggest developing<br />
an critical incident interview guide around cause, course and result.<br />
Through applying the critical incident method, data can be gathered through a variety of structured or<br />
unstructured methods and researchers are given considerable latitude in study design. Critical<br />
incident studies are more highly focused on providing solutions to more practical problems than other<br />
qualitative methods that simply seek to describe phenomena. The format devised by Flanagan was<br />
aimed at pinpointing facts and reducing personal opinions, judgements and generalizations<br />
(Kemppainen, 2000).<br />
To understand how a critical incident method can be employed within a simulated gaming<br />
environment to promote reflective <strong>learning</strong>, it is important to first understand the nature of simulation<br />
games and their role in providing experiential <strong>learning</strong> to participants (students).<br />
There is quite an extensive literature dealing with how simulation games can be used as instructional<br />
tools to engage students with active <strong>learning</strong> (see Pasin and Giroux, 2011; and Sitzmann, 2011 for<br />
recent expositions of the simulation and gaming literature). Hainey et al (2011) illustrate the breadth<br />
and depth of disciplinary application of simulation gaming for <strong>learning</strong> from business through to<br />
medicine. Simulation games (SGs) are increasingly computer based. They are highly complex<br />
synthetic artefacts where students are provided with the opportunity to ‘learn by doing’ through<br />
immersion in simulated experiences of the real world (Ben-Zvi, 2010). With regard to business<br />
education, many SGs are characterised by a simulated competitive environment in which rival<br />
companies make periodic decisions, this provides inputs into the software which in turn generates<br />
information which then forms the basis for the next round of decision making (Vos and Brennan,<br />
2011).<br />
The benefits of employing business simulation games to help develop decision-making skills are well<br />
established (see Moizer et al, 2006; Edelheim and Ueda, 2007; Gilgeous and D’Cruz, 1996). SGs can<br />
bring experiential <strong>learning</strong> into the classroom through providing structured environments for <strong>learning</strong><br />
about complex problems (Doyle and Brown, 2000). Such <strong>learning</strong> is often deeper than that achieved<br />
when conventional teaching approaches are adopted, with the opportunity for <strong>learning</strong> to occur at<br />
different levels in Bloom’s taxonomy (Anderson and Lawton, 1988; Ben-Zvi, 2010) as well as allowing<br />
students to pass through all four phases of Kolb’s ‘experiential <strong>learning</strong> cycle’ (Moizer et al, 2004). As<br />
well as being perceived by many students as a more enjoyable way to learn, SGs do provide a means<br />
of integrating subject matter into the mainstream of students’ interests. A SG can be viewed as a<br />
‘<strong>learning</strong> laboratory’ where students are able to practice analytical and decision-making techniques in<br />
preparation for the uncertainties of life (Gilgeous and D’Cruz, 1996). Experimentation can be<br />
undertaken at low cost, where understandings of the nature of cause-and-effect can be promoted and<br />
developed, both under conditions of complexity and within compressed time periods. SGs can<br />
correspond with reality; but at the same time can provide safe environments for students to play out<br />
risks and live with ‘synthetic’ rather than real consequences, i.e. they easily allow <strong>learning</strong> through<br />
mistakes to occur. In the course of playing a SG, students begin to attain a level of ownership which<br />
moves them from passive to active learners. Within the simulation setting, they are presented with<br />
opportunities to put the espoused theories that they have learnt into action, and actually try out new<br />
behaviours that they would not readily attempt in a workplace, often for fear of failure.<br />
503
Jonathan Moizer and Jonathan Lean<br />
Interactive role-play is inherent to the simulation gaming experience. In business SGs, participants<br />
invariably assume prescribed roles of characters from a company; which allows students to move<br />
towards developing an understanding of circumstances and roles that are beyond their immediate<br />
experience (Sutcliffe, 2002). They are given the opportunity to feel what is at stake, albeit in a<br />
synthetic setting. It is hoped that better understanding of roles and relationships, and a greater<br />
awareness of their own activities emerges from a SG role-play experience.<br />
3. Critical incidents to promote <strong>learning</strong> in a simulation game<br />
A review of the critical incident literature reveals various application of the critical incident methods,<br />
particularly as a means to allow individuals’ to reflect on their experiences and change their thinking.<br />
Therefore, it is perhaps surprising that there appears to have been very little research published in the<br />
domain of simulation gaming other than work by Wolfe (1975) who used Flanagan’s critical incident<br />
technique to evaluate the behavioural effectiveness of students participating in a management SG.<br />
Simulation gaming would appear to present as a useful method to enable students to reflect on a<br />
series of simulated events and actions, and therefore, to deepen their understanding of judgement<br />
and decision making.<br />
Critical incidents are a good medium for achieving reflective <strong>learning</strong> on the nature of situational<br />
events. However, there are a number of reasons why it can be difficult to learn from incidents in the<br />
real world, and why therefore, simulation games may facilitate effective <strong>learning</strong> through the<br />
application of a critical incident method.<br />
Firstly, in real environments critical incidents can be infrequent, whereas in a simulation game, a<br />
longer period of simulated time can be compressed into a much shorter period of real playing time, for<br />
example, .a number of simulated years can unfold in a matter of real hours. Hence, it is possible for a<br />
number of incidents to be introduced into the game by the instructor in a planned way, or to allow<br />
them to be generated in an unplanned way through the interactions and complex dynamics of the SG.<br />
Furthermore, the consequences of an event can be seen to emerge and develop in full over an<br />
extended simulation period. This allows the potential for rich <strong>learning</strong> opportunities to present<br />
themselves through critical incidents within a manageable period of play.<br />
Secondly, not only do SGs offer the opportunity to compress real time into simulated time, they also<br />
offer the opportunity for instructors to stop time, and therefore ring-fence space for reflection, both<br />
forensically and in real time within a simulation gaming exercise. This is in contrast to the real world<br />
where day to day operational pressures invariably mean that managers have limited time to think<br />
about and learn from events that have unfolded. As a result, debriefing lends itself well to post-play<br />
critical reflection in a simulation gaming environment. In turn, such debriefing allows the <strong>learning</strong><br />
benefits arising from in-play critical incidents to be harnessed more effectively. Assessment criteria<br />
based upon critical incidents can be introduced to require students to reflect on how an outcome was<br />
governed by a course of action, and what the causal influences were behind such an action or<br />
actions.<br />
Thirdly, through technological advances, computer-based simulation games have become more<br />
authentic. These new higher levels of fidelity and verisimilitude have allowed learners to become<br />
more immersed in the simulated ‘microworlds’ of the games; hence, critical incidents that occur within<br />
the gaming period are felt in an impactful way that is closer to a real world experience. They can be<br />
viewed as more concrete than abstract. Although critical incidents may feel real and can generate<br />
emotional reactions, they do not have negative consequences for the players themselves or any real<br />
organisations; in other words, simulations provide a safe environment for <strong>learning</strong> through critical<br />
incidents.<br />
Although SGs have a higher level of authenticity, they are by definition ‘simplifications of reality’ and<br />
do still work at a level of abstraction relative to the real world. In a real world situation, marked by<br />
problems that are ill-defined, amorphous, messy and complex, it is often difficult to identify a critical<br />
incident, never mind understand its nature, in terms of cause, course and results. In contrast, because<br />
computer based SGs are more rigid (i.e. they comprise of structure, rules and behaviour) and<br />
simplified than real world circumstances, it is easier to identify and separate out different impacting<br />
variables and events within the boundaries of the game. With fewer variables than in the real world for<br />
the students’ to consider, it is easier for the them to actually trace through and establish the causal<br />
influences that are contributing to a course of action and its resultant outcomes; thus making critical<br />
504
Jonathan Moizer and Jonathan Lean<br />
incidents more explicit. This is assisted greatly within simulations by the outputs or results generated<br />
automatically after each playing period of the game. Within some SGs, critical incidents are either predetermined<br />
(pre-programmed) or the instructor has the opportunity, pre-play, to introduce them<br />
(unknown to the player) at key points during game play. In a number of games, there is a facility for<br />
instructors to flag these interventions to aide the learners’ recognition of critical incidents, for example<br />
Airline: A Strategic Management Simulation (Smith and Golden, 2008), Human Resources<br />
Management Simulation (Smith and Golden, 2005) and Corporation: A Global Business Simulation<br />
(2003). This explicit flagging and signposting of critical incidents can aid <strong>learning</strong> that may not be<br />
readily apparent in the real world.<br />
Given the benefits that SGs offer (in terms of authenticity and an ability to manipulate time), they can<br />
provide a potentially powerful mechanism for students to discover and learn about the nature of<br />
decision making through experiencing and reflecting upon critical incidents in a safe, risk-free and<br />
experimental <strong>learning</strong> environment.<br />
4. A <strong>learning</strong> and assessment approach to using a critical incident method in<br />
a simulation game<br />
Within The University of Plymouth Business School, part-time, post-graduate / post-experience<br />
students are taught strategic management as a capstone module of study, dovetailing the various<br />
strands of business and management. As part of an effort to make the subject more engaging and<br />
relevant to their current and future work organisations, simulation gaming was adopted as an<br />
experiential <strong>learning</strong> approach. The Business Strategy Game (Thompson et al, 2010) was selected as<br />
an off-the-shelf total enterprise simulation game that could help address the teaching, <strong>learning</strong> and<br />
assessment imperatives of the subject module.<br />
The Business Strategy Game (BSG) is web-based with 24-7 student and instructor access which<br />
simulates the strategies and decisions of companies serving the global marketplace for athletic<br />
footwear. Student teams (companies) adopt the roles of Directors and compete against other teams<br />
for a predetermined market demand. An instructor is responsible for overseeing the running of the<br />
game and can shape many of the externalities that will govern company decision-making (such as<br />
exchange rates, material prices and shipping costs). The game requires the input of yearly business<br />
decisions, such as volumes of goods to manufacture, shipment volumes, pricing levels and<br />
advertising spend. These decisions are collectively processed on an on-line server program, and the<br />
game then rolls on to another year’s play. Scores based on a number of performance metrics (for<br />
example, profit, market share, capitalisation, sales volume) are determined, resulting in the teams<br />
moving up or down a league table of companies.<br />
Given the wide managerial experience of this particular student cohort, the totality of the BSG<br />
environment does provide resonance and correspondence with their real workplace setting, allowing<br />
them to not only draw on theory, but their own management practice to inform how they strategise for<br />
and operate their simulated companies. This can result in the levels of engagement and<br />
understanding of the game dynamics being higher than that of learners with little or no managerial<br />
experience. Consequently, this does provide good opportunities to extend the real world feel of the<br />
<strong>learning</strong> through to using more innovative methods of assessment of the knowledge and<br />
understanding developed through playing the BSG.<br />
Incidents or events that unfold during the course of playing the BSG are presented as opportunities<br />
for critical reflection. Throughout the course of play, the student teams are confronted by significant<br />
events which provide <strong>learning</strong> opportunities. Such events are not routine, rather critical.<br />
Consequently, they help students, through reflection, to develop high-order skills. In designing the<br />
summative assessment of the simulation gaming exercise, what was required was a method that was<br />
based on reflection and would enable students to identify and discuss, post-game, the critical<br />
incidents that occurred during play. It was decided that a form of debriefing activity would allow critical<br />
reflection to occur.<br />
Reflection can play an important part in constructing knowledge and improving <strong>learning</strong> (Chen et al,<br />
2011). A number of author emphasise using critical reflection or debriefing as an important element of<br />
<strong>learning</strong> with SGs (Garris, Ahlers, and Driskell, 2002; Gredler, 2002). In-game, students may play<br />
without actually reflecting on decision and events, hence the importance for them to be aware of<br />
reflection and how it can be facilitated (Lee and Chen, 2008). Post-game, getting students to reflect<br />
505
Jonathan Moizer and Jonathan Lean<br />
on the knowledge gained holistically from playing a SG, and how the various strands of information fit<br />
together, post-play, can help promote deeper <strong>learning</strong>. Knotts and Keys (1997) recognise that<br />
students learn at various points along the gaming period. They specifically argue that <strong>learning</strong> occurs<br />
when students are forced to reflect on their experiences, as the experience itself will not provide<br />
<strong>learning</strong> alone. Students need to learn how they can to learn from experiences tested against<br />
espoused theories. This process of reflection can be facilitated through debriefing, where <strong>learning</strong><br />
closure can take place. End of game debriefing can crystallise or integrate the <strong>learning</strong> events that<br />
occurred during play (Léger et al, 2011). Debriefing is a form of instructional support that acts as<br />
scaffolding to help provide for an effective <strong>learning</strong> environment (Garris et al, 2002). It provides the<br />
means by which the game itself can be linked to the achievement of <strong>learning</strong> outcomes. It allows the<br />
opportunity to review and analyse the events that occurred during game play, allowing argumentation<br />
to be developed and assessed (Sutcliffe, 2002).<br />
For this student cohort and module, it was concluded that oral debriefing offered particular <strong>learning</strong><br />
benefits. Oral debriefing conducted in-role can be extremely valuable, as it allows the realism of the<br />
experience to be extended. Arguably, in this situation, the gaming participants are still thinking and<br />
acting as decision-makers, and thus, they are able to reflect more easily on the past, present and<br />
future direction of their company. Oral debriefings are commonly used, post-game, to guide students<br />
through a reflective process of their <strong>learning</strong>, where students and instructors engage in a question<br />
and answer type session (Petranek et al, 1992).<br />
Post-game, oral debriefing was particularly suited to this cohort of students given their managerial<br />
backgrounds, work experience and confidence in engaging in verbal discourse. The debriefing was<br />
used to measure the extent to which student participants learnt about managing their company’s<br />
strategy and operations within the BSG. The debriefing took the form of a role play discussion, with<br />
each student team (Company Directors) over a period of 20 to 30 minutes. This discussion was<br />
summatively assessed and contributed to 30% of the overall module grade. Prior to the debriefing, the<br />
teams prepared by producing a short formative report, incorporating relevant information, models and<br />
graphics that would clarify their understanding of key events and performance as well as provide the<br />
instructors with the necessary evidence to shape a full discussion.<br />
Within the debriefing discussions, the teams remained in-role as company directors and two<br />
instructors took on the parts of corporate investors seeking potential investment opportunities with<br />
each company. A scripted role-play was employed to structure the verbal interactions to ensure that<br />
the discussion addressed the assessment criteria and intended <strong>learning</strong> outcomes of the simulation<br />
gaming exercise. A discussion of critical incidents experienced during play featured as a key<br />
component of the debriefing activity.<br />
Flanagan’s technique was adapted and simplified in order to provide reflective insight at the postgame<br />
debriefing stage into a series of events or incidents that the students faced at certain points<br />
along the gaming timeframe. The students were advised that a critical incident in the context of the<br />
game play was an event which made them think differently than before about aspects of their<br />
simulated business and its operating environment.<br />
As part of the debriefing process, students were prompted to look back and reflect on the game play<br />
and to explore critical incidents (both positive and negative) which may have impacted on their<br />
understanding of business strategy and the running of their simulated company. They shared with the<br />
instructors up to three critical incidents (occurring at the company, industry and macro-environmental<br />
levels). They were asked to frame incidents in terms of cause, course and result (as per Edvardsson<br />
and Roos, (2001) interview guide method for critical incident capture).<br />
The surfacing of critical incidents centred on three questions:<br />
What happened in the critical incident? - CAUSE<br />
How did your strategic thinking change as a result of this event? - COURSE<br />
What strategic actions followed on from this event (behaviour)? - RESULT<br />
Evaluation of the critical incident debriefing activity is currently on-going; however, initial feedback<br />
from instructors and students involved indicates that students were able to frame and reflect upon<br />
incidents. In other words, the student teams were able, post-game, to identify critical incidents and<br />
506
Jonathan Moizer and Jonathan Lean<br />
make meaning of them from strategic perspective enabling them to evaluate the decision made and<br />
actions taken during the course of the game.<br />
5. Summary and conclusion<br />
Anecdotally, students were highly engaged with the BSG, and an evaluation of grade performance<br />
has indicated that an emphasis on critical incidents within the assessment of the gaming experience<br />
has brought about an enhanced level of reflection, allowing more comprehensive and insightful oral<br />
debriefings to take place. However, the researchers have been keen to conduct a fuller evaluation of<br />
the reasons why student understanding of decision strategic management has improved even when<br />
using the same simulation game to teach with; in particular what role has been played by the focus on<br />
understanding critical incidents within the game playing period and how it teases out a better dialogue<br />
bout the connections between causes, courses and results. To this end, over the last four years, data<br />
has been collected from the oral debriefings to establish how the <strong>learning</strong> and assessment design has<br />
fed through into the business understanding of students, i.e. how and to what extent has the<br />
debriefing approach had a positive impact on reflective <strong>learning</strong>. This will provide a fuller<br />
understanding of the contribution that critical incidents make to students’ understanding of and<br />
engagement with the strategic issues within the simulated business, to allow meaning to be made of<br />
decisions and outcomes with complex causes and consequences.<br />
References<br />
Ben-Zvi, T. (2010) “The Efficacy of Business Simulation Games in Creating Decision Support Systems: An<br />
Experimental Investigation”, Decision Support Systems, Vol 49, pp 61-69.<br />
Chen, N.S., Kinshuk, C.W. and Liu, C.C. (2011) “Effects of Matching Teaching Strategy to Thinking Style on<br />
Learner’s Quality of Reflection in an Online Learning Environment”, Computers & Education, Vol 56, pp 53-<br />
64.<br />
Cope, J. (2003) “Entrepreneurship Learning and Critical Reflection: Discontinuous Events as Triggers for ‘Higherlevel’<br />
Learning”, Management Learning, Vol 34, No. 4, pp 429-450.<br />
Cope, J. and Watts, G. (2000) “Learning by Doing: An Exploration of Experience, Critical Incidents and Reflection<br />
in Entrepreneurial Learning”, International Journal of Entrepreneurial Behaviour & Research, Vol 6, No. 3,<br />
pp 104-124.<br />
Doyle, D. and Brown, F.W. (2000) “Using a Business Simulation to Teach Applied Skills: The Benefits and the<br />
Challenges of Using Student Teams from Multiple Countries”, Journal of European Industrial Training, Vol<br />
24, No.6, pp 330-336.<br />
Edelheim, J. & Ueda, D. (2007) “Effective Use of Simulations in Hospitality Management Education – a Case<br />
Study”, Journal of Hospitality, Leisure, Sport & Tourism Education, Vol 6, No. 1, pp 18-28.<br />
Edvardsson B. & Roos, I. (2001) “Critical Incident Techniques: Towards a Framework for Analysing the Criticality<br />
of Critical Incidents”. International Journal of Service Industry Management, Vol 12, No. 3, pp 251-268.<br />
Finch, A. (2010) “Critical Incidents and Language Learning: Sensitivity to Initial Conditions”, System, Vol 38, pp<br />
422-431.<br />
Flanagan, J.C. (1954) “The Critical Incident Technique”, Psychological Bulletin, Vol 51, No.4, pp 327-358.<br />
Garris, R., Ahlers, R., and Driskell, J. E. (2002) “Games, Motivation, and Learning: A Research and Practice<br />
Model”, Simulation & Gaming, Vol 33, No. 4, pp 441–467.<br />
Gilgeous V. and D’Cruz, M. (1996) “A Study of Business and Management Games”, Management Development<br />
Review, Vol. 9, No. 1, pp 32-39.<br />
Gredler, M. E. (1996) “Games and Simulations and their Relationships to Learning”, in D. H. Jonassen (Ed.),<br />
Handbook of Research for Educational Communications and Technology. pp. 571–603, Macmillan Library<br />
Reference, New York.<br />
Gulezian, A.K. (1981) “Teaching Business Policy and Strategy Using the Incident Process”, Developments in<br />
Business Simulation & Experiential Exercises, Vol 8, pp 202-204.<br />
Hainey, T. Connolly, T.M., Stansfield, M. and Boyle, E.A. (2011) “Evaluation of a Game to Teach Requirements<br />
Collection and Analysis in Software Engineering at Tertiary Education Level”, Computers & Education, Vol<br />
56, pp 21-35.<br />
Johnston, B. (1995) “The Determinants of Service Quality: Satisfiers and Dissatisfiers”, International Journal of<br />
Service Industry Management, Vol 6 (5), pp 53-71.<br />
Kemppainen, J.K. (2000) “The Critical Incident Technique and Nursing Care Quality Research”, Journal of<br />
Advanced Nursing, Vol 32, No. 5, pp 1264-71.<br />
Knotts, U.S., and Keys, J.B. (1997) “Teaching Strategic Management with a Business Game”, Simulation<br />
Gaming, Vol 28, No.4, pp 377-394.<br />
Lee, C.Y. and Chen, M.P., Kinshuk, C.W. and Liu, C. (2008) “A Computer Game as a Context for Non-routine<br />
Mathematical Problem Solving”, Computers & Education, Vol 52, pp 530-542.<br />
Léger P.M., Charland, P., Feldstein, H.D. and Robert, J. (2011) “Business Simulation Training in Information<br />
Technology Education: Guidelines for New Approaches in IT Training”, Journal of Information Technology<br />
Education, Vol 10, pp 39-53.<br />
507
Jonathan Moizer and Jonathan Lean<br />
MacFarlane, B. (2001) “Developing Reflective Students: Evaluating the Benefits of Learning Logs within a<br />
Business Ethics Programme”, Teaching Business Ethics, Vol 5, No. 4, pp 375-387.<br />
MacFarlane, B. (2003) “Tales from the Front-line: Examining the Potential of Critical Incident Vignettes”,<br />
Teaching Business Ethics, Vol 7, No. 1, pp 55-67.<br />
Mallak, L.A., Lyth, D.M., Olson, S.D., Ulshafer, S.M. and Sardone, F.J. (2003) “Diagnosing Culture in Health-care<br />
Organizations using Critical Incidents”, International Journal of Health Care Quality Assurance, Vol 4, pp<br />
180–190.<br />
Moizer, J.D., Lean, J., Smith, G., and Towler, M.J. (2004) “Experiences of Using a Business Strategy Simulation:<br />
Lessons for Promoting Effective Learning”, International Journal of Management Education, Vol 4, No. 1, pp<br />
19-28.<br />
Moizer, J., Lean, J. Towler, M. and Smith, G. (2006) “Modes of Learning in the Use of a Computer-based<br />
Business Simulation Game”, International Journal of Learning Technology, Vol 2, No. 1, pp 49-60.<br />
Moenaert, R.K., Robben H., Antioco, M., De Schamphelaere, V. and Roks, E. (2010) “Strategic Innovation<br />
Decisions: What You Foresee is Not What You Get”, Journal of Product Innovation Management, Vol 27, pp<br />
840-855.<br />
Pasin, F. and Giroux, H. (2011) “The Impact of a Simulation Game on Operations Management Education”.<br />
Computer & Education, Vol 57, pp 1240-1254.<br />
Petranek, C.F., Corey, S. and Black, R. (1992) “Three Levels of Learning in Simulations: Participating, Debriefing<br />
and Journal Writing”, Simulation & Gaming, Vol 23, No. 2, pp 174-185.<br />
Roos, I. (2002) “Methods of Investigating Critical Incidents: A Comparative Review. Journal of Service Research”,<br />
Vol 4, No. 3, pp 193-204.<br />
Sitzmann, T. (2011) “A Meta-Analytic Examination of the Instructional Effectiveness of Computer-Based<br />
Simulation Games”, Personnel Psychology, Vol 64, pp 489-528.<br />
Smith, J.R. and Golden, P.A. (2003) Corporation: A Global Business Simulation (4 th ed.), Prentice Hall, NJ.<br />
Smith, J.R. and Golden, P.A. (2005) Human Resources Management Simulation (2 nd ed.), Prentice Hall, NJ.<br />
Smith, J.R. and Golden, P.A. (2008) Airline: A Strategic Management Simulation (4 th ed.), Prentice Hall, NJ.<br />
Sutcliffe, M. (2002) Games, Simulations and Role-playing. The Handbook for Economics Lecturers, LTSN,<br />
Bristol.<br />
Taylor, D.W. & Thorpe, R. (2004) “Entrepreneurial Learning: A Process of Co-participation”, Journal of Small<br />
Business and Enterprise Development, Vol 11, No. 2, pp 203-211.<br />
Thompson, A.A., Stappenbeck, G.J. and Reidenbach, M.A. (2010) The Business Strategy Game: Competing in a<br />
Global Marketplace, (2010 ed.), McGraw-Hill/Irwin, IL.<br />
Voss, R. (2009) “Studying Critical Classroom Encounters: The Experiences of Students in German College<br />
Education”, Quality Assurance in Education, Vol 17, No. 2, pp 156-173.<br />
Wolfe, J. (1975) “Effective Performance Behaviors in a Simulated Policy and Decision Making Environment”,<br />
Management Science, Vol 21, No. 8, pp 872-882.<br />
508