Intelligent Tutoring Systems for Ill-Defined Domains - Philippe ...

75 Gauthier, Naismith, Lajoie, & Wiseman
conversing with other participants or using the Internet to help them to solve the case
and a student researcher was available in each room to answer technical questions.
The session lasted approximately two hours.
In the control room, nine students completed the three test cases. After they
completed each case, they viewed the text-based expert summary and filled out the
case questionnaire. In the experimental room, eight students were given the
opportunity to view the expert decision maps in addition to the text-based expert
summaries. One student who arrived late was assigned to the experimental room, but
given the control conditions due to time limitations. As in the control room, the
students filled out the case questionnaire at the end of each case. All students were
asked to fill out the post-questionnaire before concluding the session.
4 Results
4.1 Performance
All students were able to use BioWorld successfully to complete the four cases.
Table 2 displays the percentage of students whose final diagnosis matched the expert
solution. Additional diagnoses submitted for Test Case 1 included arrhythmia (5),
panic attacks (5), diabetes type II (1) and myocardial infarction (1). For Test Case 2,
one student selected diabetes type II. For Test Case 3, the additional diagnoses
included panic attacks (3) and barbiturate intoxication (1).
Table 2. Number of students arriving at expert final diagnosis
Case Expert Diagnosis % of Students
Tutorial Celiac Disease 100% (18)
Case 1 Pheochromocytoma 33% (6)
Case 2 Diabetes, Type I 94% (17)
Case 3 Hyperthyroidism 78% (14)
No differences in performance were expected between the two groups for the
tutorial case, as all of the students completed this case under the same conditions, or
Case 1, as the students were required to submit their final diagnoses before they could
view the expert decision map. As shown in Table 3, however, the experimental group
outperformed the control group for Cases 2 and 3. A two-tailed t-test was performed
on the average time taken for each case, but none of the comparisons reached
statistical significance. Note that the average time spent per case does not include the
time spent interacting with the expert decision map.