Intelligent Tutoring Systems for Ill-Defined Domains - Philippe ...

Table 3. Number of students arriving at expert final diagnosis, by group
Case Correct Diagnosis Average Time
Control Experimental Control Experimental
Tutorial 100% (10) 100% (8) 31:32 35:38
1 20% (2) 50% (4) 21:29 20:21
2 90% (9) 100% (8) 14:01 13:37
3 70% (7) 88% (7) 10:52 11:34
The students were also required to indicate their level of confidence with their final
diagnosis. Table 4 displays the percentage confidence levels attained for both correct
and incorrect diagnoses. As before, no differences were expected between the two
groups for either the Tutorial case or Case 1. For Cases 2, the control group
demonstrated higher levels of confidence for both correct and incorrect diagnoses.
For Case 3, the experimental group demonstrated a higher average level of confidence
when their diagnoses were correct, while the control group demonstrated a higher
average level of confidence when their diagnoses were incorrect.
Table 4. Average confidence levels of final diagnosis, by group
Case Average Confidence –
Average Confidence –
Correct Diagnosis
Incorrect Diagnosis
Control Experimental Control Experimental
Tutorial .87 .90 - -
1 .58 .62 .74 .68
2 .91 .87 .75 -
3 .85 .91 .72 .40
4.2 Perceived Difficulty
In the case questionnaire, the students were asked to rate the level of difficulty of the
case using the following scale: 1 = too easy, 2 = good revision, 3 = challenging, 4 =
difficult. Table 5 displays the average difficulty rating for each case. For each group,
Case 1 (pheochromocytoma) was perceived to be the most difficult, followed by the
Tutorial Case (celiac disease), then Case 3 (hyperthyroidism) then Case 2 (diabetes
type I). A two-tailed t-test was performed for each case rating, with none of the
comparisons reaching statistical significance.
Table 5. Average difficulty rating, by group
Case Average Difficulty Rating
Control Experimental
Tutorial 2.10 2.14
1 2.60 2.88
2 1.90 1.71
3 2.10 1.75
Using Expert Decision Maps 76