learning - Academic Conferences Limited
learning - Academic Conferences Limited learning - Academic Conferences Limited
Some Reflections on the Evaluation of Virtual Learning Environments Nabil Ben Abdallah and Françoise Poyet ELICO (Équipe de recherche de Lyon en sciences de l’Information et COmmunication), Lyon, France nabil.ben-abdallah@iut-dijon.u-bourgogne.fr francoise.poyet@free.fr Abstract: This paper aims at pondering over the evaluation question of the VLE. We have carried out an overview of concepts and methods commonly used in Human-Computer Interaction (HCI) in the evaluation of information systems and then we have discussed the limits of their application on the VLE. We are proposing to evaluate the latters according to an activity-oriented approach. It will allow us to expand our field of study to learning situations in which a VLE can be integrated. The learning activity is thus observed not only through the interactions of motivated learners and teachers with VLE but also with the objects and objectives of learning implemented via VLE. This observation is based on the theoretical framework proposed by activity theory. The Engeström model as a form of conceptualization of the structure of activity is essentially explored to understand the elements interacting in a learning situation integrating VLE. We conclude our contribution by the application of ideas developed by observing the use of a VLE in a learning situation of information seeking and involving small groups of students. Keywords: VLE evaluation; activity theory; usability; utility; practical acceptability; social acceptability 1. Introduction The increasingly generalized use of Virtual Learning Environments (VLE) in different learning situations places the issue of the evaluation of their impacts on learning as well as on the organizations at the core of designers and managers’ concerns regarding the systems of learning. VLE is not only a technical solution integrating heterogeneous technologies, it is also a space of information around which a social space that organizes and structures the various interactions necessary for any learning activity can be formed and developed. Several studies and research have been carried out (Britain 2004; Colace 2006; Keller 2005; Dobson 2001) to help the actors of learning to choose the appropriate VLE so as to cope with specific situations of learning, to assess the consequences of the use of these VLEs on learners and on learning itself, and to identify interaction problems with these VLEs. In this work, the learning activity is observed not only through the interactions of motivated learners and teachers with VLE but also with the objects and objectives of learning implemented via VLE. This observation is based on the theoretical framework proposed by activity theory. The Engeström model (Engeström 1987) as a form of conceptualization of the structure of activity is essentially explored to understand the elements interacting in a learning situation integrating VLE. The data resulting from the analysis of the activity system of learning based on the model of Engeström enable us to get, on the one hand, data on the degree of practical and social acceptability of VLE, and on the other hand, data that can be used to improve the methods of usability and utility evaluation of VLE. We conclude our contribution by the application of ideas developed above by observing the use of a VLE in a learning situation of information seeking and involving small groups of students. 2. What is a VLE? According to Weller (Weller 2007, p.5) a VLE is: "a software system that combines a number of different tools that are used to systematically deliver content online and to facilitate the learning experience around that content". Other authors such as Colace and al. (Colace 2006, p.2) admit that a VLE or a eLearning platform is organized in three macro-components: "In our opinion the most part of contemporary eLearning platform can be viewed as organized into three fundamental macro components: a Learning Management System (LMS), a Learning Content Management System (LCMS) and a Set of Tools for distributing training contents and for providing interaction". In the light of these definitions and some others, we are considering that a VLE is a particular computer system allowing to create learning situations with specific learning objectives (e.g. learning such a type of knowledge). A VLE is usually conceived with the aim to train the learning actors into achieving these 48
Nabil Ben Abdallah and Françoise Poyet objectives through a favorable activity. VLE offers a set of integrated functionalities to complete the requirements of various learning situations. Four groups of functionalities are usually mentioned: course management (design, maintenance, and diffusion), tracking learning, technical administration (set-up, security, integration, maintenance) and learner interface (offering communication and collective work tools, and a customizable space). Depending on the learning objectives, a VLE must also propose expanded parameterization facilities to create more or less elaborate learning situations. eLearning platforms such as Blackboard, Moodle, Claroline, and others are examples of VLE (either proprietary or free software). From a technical point of some view, a VLE integrates various types of software tools to support the various functionalities mentioned in the preceding paragraph. For examples we have: a database management server, a web server, a mail server, a file server. These servers are developed with technologies of various origins, mostly meeting with the requirements of various standards. Among these technologies, those of the Internet have had considerable repercussions on the design of VLEs. In addition to the technical possibility of having applications interconnected and making use of resources coming from different sites, collaborative or co-operative activities and as well as human interaction seem to be favored thanks to tools enabling communication, sharing and the production of documents. The use of a VLE encourages written communication and, except with the synchronous communication tools, makes the learner’s participation less improvised because he has a certain amount of time for consideration and easy access to resources enabling him to enrich his ideas and to pursue his thinking. In short, a learning situation built around a VLE bears little resemblance to a conventional class where student is constantly under a "social pressure" emerging from the activity of the other students and the requirements of the teacher. The metaphors (class, group, office, etc.) used in the design of VLEs (Falconer 2008) should not allow us to forget that this is above all a computer system having features which are not those of a learning device built around a conventional class. In order to understand these characteristics and thus expand our specific acception of VLEs beyond the principle integrating functional and technological aspects, we quote here Dillenbourg and his colleagues (Dillenbourg 2002) who think that VLEs can be identified through six characteristics : 1) A virtual learning environment is a designed information space 2) A virtual learning environment is a social space 3) The virtual space is explicitly represented 4) Students are not only active, but also actors 5) Virtual learning environments integrate heterogeneous technologies and multiple pedagogical approaches 6) Most virtual environments overlap with physical environments. It is obvious that a given VLE may not verify all these characteristics, the objective here is to have a comprehensive understanding of roles that a VLE can play in learning activities. The information space being composed, among other things, of the courses and of all the information resources that enrich and complement them must be sufficiently structured and organized so that it can be suitably exploited in learning activities. Beyond this information space, a social environment which organizes and structures the various interactions (learner to learner, learner to teacher, etc.) which are essential to any form of learning activity, is built up. The learning environment can be based on more or less sophisticated interfaces going from simple text interfaces to graphical user interfaces in 3D. Whatever the interface used, it must be sufficiently explicit so that learners can achieve the proposed learning. Although VLEs are often associated with distance learning because they are essentially Web-based applications, they are also widely used as a support for classroom teaching. Integration within a VLE of several software tools makes the adaptation of various approaches possible. For example, teachers can request individual or collective work from the students and decide to intervene or not in the various phases of the completion of this work. The package of software tools (exerciser, wiki, blog, chat ...) offered by a VLE makes it possible to adopt one or the other approach. All these characteristics are important to set up a learning situation with precise objectives. Beyond the impact which these VLE can have on the learning outcome, it is the learner's statute which changes: from a passive learner who merely memorizes some patterns or knowledges which are passed on to him, we will end up with a learner who is not only active but who is also an actor of his own learning. He will use and produce information, and he will continuous interaction with learning environment. According to Dillenbourg (Dillenbourg 2002, p.7) "… the notion of a learning activity in virtual learning environments refers to something richer than in individual courseware, closer to the 49
- Page 23 and 24: Research interests include the inve
- Page 25: Novita Yulianti is a PhD student at
- Page 28 and 29: Samuel Adu Gyamfi et al. the develo
- Page 30 and 31: Samuel Adu Gyamfi et al. completion
- Page 32 and 33: Samuel Adu Gyamfi et al. interactio
- Page 34 and 35: Survey of Teachers’ use of Comput
- Page 36 and 37: Babatunde Alabi Alege and Stephen O
- Page 38 and 39: Babatunde Alabi Alege and Stephen O
- Page 40 and 41: Babatunde Alabi Alege and Stephen O
- Page 42 and 43: Issues and Challenges in Implementi
- Page 44 and 45: Hussein Al-Yaseen et al. 2000/2001
- Page 46 and 47: Hussein Al-Yaseen et al. phase invo
- Page 48 and 49: Hussein Al-Yaseen et al. Berthold,
- Page 50 and 51: Antonios Andreatos Figure 1: Estima
- Page 52 and 53: Antonios Andreatos exchange applied
- Page 54 and 55: Antonios Andreatos knowledge space,
- Page 56 and 57: Figure 5: Video metadata from YouTu
- Page 58 and 59: Antonios Andreatos The organisatio
- Page 60 and 61: Constructing a Survey Instrument fo
- Page 62 and 63: Jonathan Barkand The teacher demon
- Page 64 and 65: Jonathan Barkand Indicator 2.3: Has
- Page 66 and 67: References Jonathan Barkand Allen,
- Page 68 and 69: 2. Pedagogical agents Orlando Belo
- Page 70 and 71: Orlando Belo Type (Tp), the refere
- Page 72 and 73: 3.3 The agent’s architecture Orla
- Page 76 and 77: Nabil Ben Abdallah and Françoise P
- Page 78 and 79: Nabil Ben Abdallah and Françoise P
- Page 80 and 81: Nabil Ben Abdallah and Françoise P
- Page 82 and 83: Designing A New Curriculum: Finding
- Page 84 and 85: Andrea Benn For this new course, it
- Page 86 and 87: Andrea Benn Technology is already i
- Page 88 and 89: Andrea Benn To bring about the co-o
- Page 90 and 91: Latefa Bin Fryan and Lampros Stergi
- Page 92 and 93: Latefa Bin Fryan and Lampros Stergi
- Page 94 and 95: Faculty development Online course
- Page 96 and 97: Latefa Bin Fryan and Lampros Stergi
- Page 98 and 99: Latefa Bin Fryan and Lampros Stergi
- Page 100 and 101: Alice Bird being reviewed under the
- Page 102 and 103: Alice Bird Developing the process m
- Page 104 and 105: Alice Bird Reflecting on the feasib
- Page 106 and 107: 3.3 Early stage implementation Alic
- Page 108 and 109: Enhancement of e-Testing Possibilit
- Page 110 and 111: Martin Cápay et al. of Likert scal
- Page 112 and 113: Martin Cápay et al. Figure 3 Proce
- Page 114 and 115: Martin Cápay et al. Figure 4: An e
- Page 116 and 117: Martin Cápay et al. On the other h
- Page 118 and 119: Tim Cappelli demand from students t
- Page 120 and 121: Tim Cappelli at a time and increasi
- Page 122 and 123: Tim Cappelli forms were processed a
Nabil Ben Abdallah and Françoise Poyet<br />
objectives through a favorable activity. VLE offers a set of integrated functionalities to complete the<br />
requirements of various <strong>learning</strong> situations. Four groups of functionalities are usually mentioned:<br />
course management (design, maintenance, and diffusion), tracking <strong>learning</strong>, technical administration<br />
(set-up, security, integration, maintenance) and learner interface (offering communication and<br />
collective work tools, and a customizable space). Depending on the <strong>learning</strong> objectives, a VLE must<br />
also propose expanded parameterization facilities to create more or less elaborate <strong>learning</strong> situations.<br />
eLearning platforms such as Blackboard, Moodle, Claroline, and others are examples of VLE (either<br />
proprietary or free software).<br />
From a technical point of some view, a VLE integrates various types of software tools to support the<br />
various functionalities mentioned in the preceding paragraph. For examples we have: a database<br />
management server, a web server, a mail server, a file server. These servers are developed with<br />
technologies of various origins, mostly meeting with the requirements of various standards. Among<br />
these technologies, those of the Internet have had considerable repercussions on the design of VLEs.<br />
In addition to the technical possibility of having applications interconnected and making use of<br />
resources coming from different sites, collaborative or co-operative activities and as well as human<br />
interaction seem to be favored thanks to tools enabling communication, sharing and the production of<br />
documents.<br />
The use of a VLE encourages written communication and, except with the synchronous<br />
communication tools, makes the learner’s participation less improvised because he has a certain<br />
amount of time for consideration and easy access to resources enabling him to enrich his ideas and<br />
to pursue his thinking. In short, a <strong>learning</strong> situation built around a VLE bears little resemblance to a<br />
conventional class where student is constantly under a "social pressure" emerging from the activity of<br />
the other students and the requirements of the teacher. The metaphors (class, group, office, etc.)<br />
used in the design of VLEs (Falconer 2008) should not allow us to forget that this is above all a<br />
computer system having features which are not those of a <strong>learning</strong> device built around a<br />
conventional class.<br />
In order to understand these characteristics and thus expand our specific acception of VLEs beyond<br />
the principle integrating functional and technological aspects, we quote here Dillenbourg and his<br />
colleagues (Dillenbourg 2002) who think that VLEs can be identified through six characteristics : 1) A<br />
virtual <strong>learning</strong> environment is a designed information space 2) A virtual <strong>learning</strong> environment is a<br />
social space 3) The virtual space is explicitly represented 4) Students are not only active, but also<br />
actors 5) Virtual <strong>learning</strong> environments integrate heterogeneous technologies and multiple<br />
pedagogical approaches 6) Most virtual environments overlap with physical environments.<br />
It is obvious that a given VLE may not verify all these characteristics, the objective here is to have a<br />
comprehensive understanding of roles that a VLE can play in <strong>learning</strong> activities. The information<br />
space being composed, among other things, of the courses and of all the information resources that<br />
enrich and complement them must be sufficiently structured and organized so that it can be suitably<br />
exploited in <strong>learning</strong> activities. Beyond this information space, a social environment which organizes<br />
and structures the various interactions (learner to learner, learner to teacher, etc.) which are essential<br />
to any form of <strong>learning</strong> activity, is built up. The <strong>learning</strong> environment can be based on more or less<br />
sophisticated interfaces going from simple text interfaces to graphical user interfaces in 3D. Whatever<br />
the interface used, it must be sufficiently explicit so that learners can achieve the proposed <strong>learning</strong>.<br />
Although VLEs are often associated with distance <strong>learning</strong> because they are essentially Web-based<br />
applications, they are also widely used as a support for classroom teaching. Integration within a VLE<br />
of several software tools makes the adaptation of various approaches possible. For example,<br />
teachers can request individual or collective work from the students and decide to intervene or not in<br />
the various phases of the completion of this work. The package of software tools (exerciser, wiki, blog,<br />
chat ...) offered by a VLE makes it possible to adopt one or the other approach.<br />
All these characteristics are important to set up a <strong>learning</strong> situation with precise objectives. Beyond<br />
the impact which these VLE can have on the <strong>learning</strong> outcome, it is the learner's statute which<br />
changes: from a passive learner who merely memorizes some patterns or knowledges which are<br />
passed on to him, we will end up with a learner who is not only active but who is also an actor of his<br />
own <strong>learning</strong>. He will use and produce information, and he will continuous interaction with <strong>learning</strong><br />
environment. According to Dillenbourg (Dillenbourg 2002, p.7) "… the notion of a <strong>learning</strong> activity in<br />
virtual <strong>learning</strong> environments refers to something richer than in individual courseware, closer to the<br />
49