Intelligent Tutoring Systems for Ill-Defined Domains - Philippe ...
Intelligent Tutoring Systems for Ill-Defined Domains - Philippe ...
Intelligent Tutoring Systems for Ill-Defined Domains - Philippe ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
9 Walker, Ogan, Aleven & Jones<br />
plate we provided and wrote their own feedback. While the support was underutilized,<br />
it did appear to have a positive effect on the moderator posts. The 4 moderator<br />
posts in this condition communicated elements of the model, like multiple perspectives,<br />
prior personal experience, and good argumentation. For example, one moderator<br />
chose to respond to the following post:<br />
“Do you think that, at the end, he made a little joke about the word Arab? For<br />
me, it's like he is commenting on the big problem of stereotypes of race and<br />
religion in France; that all the Arabs have stores and work all the time”<br />
This post was rated as being on-topic and having multiple perspectives, due to the<br />
recognition of the stereotypes existing in France. The moderator replied:<br />
“It is not a joke, it seems to me, but a stereotype that comes from the manner<br />
in which the French think. Here, in the US, it's the people who call storekeepers<br />
Koreans. Do you think <strong>for</strong> the same reason? Also, <strong>for</strong> the Catholics, Sunday<br />
is a sacred day, <strong>for</strong> God, but the Arabs are a different religion. Is Sunday<br />
also exceptional? If yes, why is it that they work?”<br />
In the first line, the moderator acknowledged a conclusion the poster made and<br />
clarified the point. The moderator then compared a US perspective to the French perspective,<br />
using prior knowledge about the US perspective. The moderator also<br />
brought up an important point <strong>for</strong> understanding the movie, which is that Sunday is a<br />
sacred day <strong>for</strong> Catholics but possibly not <strong>for</strong> Muslims. Finally, the moderator asked<br />
thought-provoking questions to keep the discussion going. Although the moderator’s<br />
understanding of the situation was not perfect, the end result was an improvement in<br />
the overall quality of the discussion.<br />
Next, we attempted to look at the quality of the regular student posts be<strong>for</strong>e and after<br />
they received feedback given by the moderators, a difficult task due to the low<br />
number of students who posted. We divided posts into two categories based on student<br />
exposure to moderator feedback. We placed posts that were made be<strong>for</strong>e any<br />
moderator feedback appeared in the thread in one category, and posts that were made<br />
after feedback appeared in the thread in a second category. Posts on the moderator<br />
board on average across students rated fairly low at the outset, but had a trend towards<br />
improvement after feedback from the moderator. While this result is initially<br />
promising, more investigation is necessary due in part to the low number of participants.<br />
6 Discussion<br />
In this exploratory work, we compared the relative advantages and disadvantages of<br />
two approaches <strong>for</strong> supporting discussion in an ill-defined domain. The first question<br />
in an assessment of whether these interventions could be successful is whether the<br />
feedback provider (human or computer) understands what is occurring in the discussion.<br />
In the individual support condition, the ratings generated by the system matched<br />
human raters in the majority of cases, even though the technology used to process the<br />
post was relatively simple. However, the system only made ratings on a subset of the<br />
dimensions. In the supported moderator condition, only one of the two moderators<br />
used the ratings appropriately, but those ratings matched our codes <strong>for</strong> the posts. The