framework for the implementation of a virtual design studio model in ...

framework for the implementation of a virtual design studio model in ... framework for the implementation of a virtual design studio model in ...

dspace1.isd.glam.ac.uk
from dspace1.isd.glam.ac.uk More from this publisher
17.01.2013 Views

Chapter 7: Evaluation 7.1 Introduction In this section VDS 3 is evaluated and the findings of the case study regarding the implementation of the third VDS (VDS 3) are represented. The main hypotheses are examined and addressed alongside the primary hypothesis. An evaluative approach is used. Many 309 have classified evaluation as a fine tool consisting of assessing educational programmes or other kinds of educational products while they are being developed to help shape them into their final forms. 7.2 Definitions of Evaluation The most frequently given definition of evaluation is found to be: “Evaluation is the systematic assessment of the worth or merit of some object.” 310 But as there are many types of evaluation that do not result in finding the worth or merit of an object many consider this definition as lacking. Another definition found to be more adequate is: “Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object.” 311 From this definition we conclude that the evaluation of an object is to get useful feedback about an object of study to help improve it and understanding its usefulness. There are many types of evaluation, but for the purpose of this study three types of evaluation were used: formative evaluation, summative evaluation and experience evaluation. Each of these evaluation types was used to achieve a certain outcome. 309 Ruggiero, J. (2006). ‘Performance evaluation in education’. In Handbook on Data Envelopment Model. (William W. Cooper, Lawrence M. Seiford and Joe Zhu71 eds.). 71. Springer: US. p323-346. 310 William, M. K. (2006). Research methods Knowledge Base. 2 nd ed. Atomic Dog Publishing. 311 Guba, E. G., Lincoln, Y. S., (1981). Effective evaluation: improving the usefulness of evaluation results through responsive and naturalistic approaches. San Francisco: Jossey-Base. 147

Formative evaluations improve and develop the object being tested. By examining the delivery of the program or technology it helps shaping and reshaping it accordingly, the quality of its implementation, and the assessment of the organisational context, personnel, procedures, inputs 312 . Summative evaluations, according to McKavanagh et al, 313 . “examine the effects or outcomes of an object; they summarise it by describing what happens subsequent to delivery of the program or technology, assessing whether the object can be said to have caused the outcome, determining the overall impact of the causal factor beyond only the immediate target outcomes, and estimating the relative costs associated with the object” For the purpose of this study both types were used. The formative evaluation was used through the whole study for the purpose of analysing and improving its performance, and at the later stage a summative analysis was used to examine the outcome and the effect of the VDS. Experience evaluation consists of the tools that the researcher used to achieve the results and feedback. For the final product the following scheme was used (see Figure 7.1): Figure 7.1: Evaluation Scheme for VDS 3 312 William, M. K. (2006). Research Methods Knowledge Base.2 nd ed. Atomic Dog Publishing. 313 McKavanagh et al. (2001) ‘Evaluation of web-based flexible learning: findings and implications’. In: 9th International Conference on Post-compulsory Education and Training. Queensland, Australia. 148

Formative evaluations improve and develop <strong>the</strong> object be<strong>in</strong>g tested. By<br />

exam<strong>in</strong><strong>in</strong>g <strong>the</strong> delivery <strong>of</strong> <strong>the</strong> program or technology it helps shap<strong>in</strong>g<br />

and reshap<strong>in</strong>g it accord<strong>in</strong>gly, <strong>the</strong> quality <strong>of</strong> its <strong>implementation</strong>, and <strong>the</strong><br />

assessment <strong>of</strong> <strong>the</strong> organisational context, personnel, procedures,<br />

<strong>in</strong>puts 312 . Summative evaluations, accord<strong>in</strong>g to McKavanagh et al, 313 .<br />

“exam<strong>in</strong>e <strong>the</strong> effects or outcomes <strong>of</strong> an object; <strong>the</strong>y summarise it by<br />

describ<strong>in</strong>g what happens subsequent to delivery <strong>of</strong> <strong>the</strong> program or<br />

technology, assess<strong>in</strong>g whe<strong>the</strong>r <strong>the</strong> object can be said to have caused<br />

<strong>the</strong> outcome, determ<strong>in</strong><strong>in</strong>g <strong>the</strong> overall impact <strong>of</strong> <strong>the</strong> causal factor beyond<br />

only <strong>the</strong> immediate target outcomes, and estimat<strong>in</strong>g <strong>the</strong> relative costs<br />

associated with <strong>the</strong> object”<br />

For <strong>the</strong> purpose <strong>of</strong> this study both types were used. The <strong>for</strong>mative<br />

evaluation was used through <strong>the</strong> whole study <strong>for</strong> <strong>the</strong> purpose <strong>of</strong><br />

analys<strong>in</strong>g and improv<strong>in</strong>g its per<strong>for</strong>mance, and at <strong>the</strong> later stage a<br />

summative analysis was used to exam<strong>in</strong>e <strong>the</strong> outcome and <strong>the</strong> effect <strong>of</strong><br />

<strong>the</strong> VDS. Experience evaluation consists <strong>of</strong> <strong>the</strong> tools that <strong>the</strong><br />

researcher used to achieve <strong>the</strong> results and feedback.<br />

For <strong>the</strong> f<strong>in</strong>al product <strong>the</strong> follow<strong>in</strong>g scheme was used (see Figure 7.1):<br />

Figure 7.1: Evaluation Scheme <strong>for</strong> VDS 3<br />

312 William, M. K. (2006). Research Methods Knowledge Base.2 nd ed. Atomic Dog Publish<strong>in</strong>g.<br />

313 McKavanagh et al. (2001) ‘Evaluation <strong>of</strong> web-based flexible learn<strong>in</strong>g: f<strong>in</strong>d<strong>in</strong>gs and implications’. In:<br />

9th International Conference on Post-compulsory Education and Tra<strong>in</strong><strong>in</strong>g. Queensland, Australia.<br />

148

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!