learning - Academic Conferences Limited

Eugenijus Kurilovas et al.
Evaluation of quality of LS alternatives is a typical case where the criteria are conflicting, i.e., LS could
be very qualitative against several criteria, and not qualitative against the other ones, and vice versa.
Therefore, the authors propose to use multiple criteria decision analysis (MCDA) approach for
creation of LS quality evaluation model. In order to construct a proper comprehensive scientific quality
criteria system (model), the authors use the well known principles of identification of quality criteria
that have been proposed by Belton and Stewart (2002) in multiple criteria decision analysis (MCDA)
theory related research work. Practical application of these principles will be described below while
analysing LS quality criteria system (tree).
LS multiple criteria evaluation method used by the authors is referred here as the experts’ additive
utility function represented by formula (1) below including LS evaluation criteria, their ratings (values)
and weights. According to (Kurilovas and Serikoviene 2010b), this method is well-known in the theory
of optimisation methods and is named “scalarisation method”. A possible decision here could be to
transform multi-criteria task into one-criterion task obtained by adding all criteria together with their
weights. Therefore, here we have the experts’ additive utility function:
m
∑
i=
1
f ( X ) = a f ( X ) , a = 1,
a > 0 .
i
i
m
∑
i=
1
i
i
where f i (X j) is the rating (i.e., non-fuzzy value) of the criterion i for the each of the examined LS
alternatives X j. The weights here should be ‘normalised’ according to the ‘normalisation’ requirement
m
∑
i=
1
a = 1,
a > 0 .
i
i
According to (Zavadskas and Turskis 2010), the normalisation aims at obtaining comparable scales of
criteria values. The major is the meaning of the utility function (1) the better LS meets the quality
requirements in comparison with the ideal (i.e., 100%) quality (Kurilovas and Dagiene 2009ab).
The complexity of the analysed problem influences the application of more complex methods for
evaluating the quality of LS from the point of view of different learner groups. In this paper, a novel
method of consecutive triple application of Analytic Hierarchy Process (AHP) is used to establish
proper weights of LS quality criteria in the case when there are several experts evaluators. Triangular
and trapezoidal fuzzy numbers methods are used to establish proper values of LS quality criteria.
After that, formula (1) is used to calculate the values of additive utility functions for each of the
explored LS alternatives.
3. Literature analysis and research results
This section is aimed to apply the aforementioned scientific approaches in order:
To propose a suitable scientific model for evaluation of quality of LS paying especial attention to
their suitability to particular learners’ profiles (i.e., activist learners in our case);
To propose suitable scientific methods for evaluation of quality of LOs paying especial attention to
the use of novel method of consecutive triple application of AHP for establishing quality criteria
weights, and different fuzzy numbers methods to establish the values (ratings) of LS quality
criteria; and
To present the experimental evaluation results using proposed evaluation model and methods.
3.1 Learning scenarios quality model
Belton and Stewart (2002) have identified the following principles of identification of quality criteria
that are relevant to all MCDA approaches: (1) Value relevance; (2) Understandability; (3)
Measurability; (4) Non-redundancy; (5) Judgmental independence; (6) Balancing completeness and
conciseness; (7) Operationality; and (8) Simplicity versus complexity.
LS quality model based on these MCDA criteria identification principles is presented in the Fig. 2. The
model consists of three groups of quality criteria (i.e., components of LS), namely learning objects
383
(1)
(2)