elektronická verzia publikácie - FIIT STU - Slovenská technická ...

300 Selected Studies on Software and Information Systems
While many information retrieval methods can be evaluated statistically, e.g., by
computing precision and recall statistics, in many cases such exact evaluations cannot be
performed for approaches dealing with user interaction and user interfaces for adaptive
systems. The evaluation of such approaches can be done via experiments thoroughly described
in case studies, such as (Yee et al., 2003) or (Wang et al., 2007).
Since evaluation is crucial, it must be well defined beforehand and performed on several
variants of the test system against a baseline system which might either by an existing
system suitable for benchmarking or a system made using the best or most common features
of comparable systems (Yee et al., 2003).
Furthermore, layered evaluation principles should be employed to effectively separate
evaluation of individual stages of the information processing process – data collection,
data interpretation, user modeling, adaptation selection and adaptation application (Paramythis
& Weibelzahl, 2005).
In order to counteract possible errors due to user preferences, users can be divided into
multiple groups which perform the same test scenarios in different orders so that the
results between the groups can later be compared to identify potential bias. Furthermore,
different usage scenarios should be employed exploiting different user work styles, such
as search for a specific set of items (goal oriented, structured), or search for something you
like (more creative, unstructured). Individual experiments may or may not be time limited
and accompanied by questionnaires aimed at gathering subjective user feedback.
References
[1] Adkisson, H. P.: Use of Faceted Classification. Retrieved October 23, 2007, from Web design
practices:
http://www.webdesignpractices.com/navigation/facets.html, 2005.
[2] Andrejko, A., Barla, M., Bieliková, M., Tvarožek, M.: Softvérové nástroje pre získavanie
charakteristík používatea. In P. Vojtáš, & T. Skopal (Ed.), Proceedings of DATAKON ‘06, pp.
139-148. Brno, Czech Republic, 2006.
[3] Aurnhammer, M., Hanappe, P., Steels, L.: Augmenting Navigation for Collaborative
Tagging with Emergent Semantics. In I. Cruz, S. Decker, D. Allemang, C. Preist, D. Schwabe,
P. Mika, et al. (Ed.), ISWC 2006: Proceedings of the 5th International Semantic Web Conference.
LNCS 4273, pp. 58-71. Athens, GA, USA: Springer-Verlag, Berlin Heidelberg, 2006.
[4] Brusilovsky, P.: Methods and techniques of adaptive hypermedia. In User Modeling and
User-Adapted Interaction , 6 (2-3), 87-129, 1996.
[5] Geman, D.: Interactive image retrieval by mental matching. In Proceedings of the 8th ACM
international workshop on Multimedia information (pp. 1-2). Santa Barbara, California, USA:
ACM Press, New York, NY, USA, 2006.
[6] Hildebrand, M., van Ossenbruggen, J., Hardman, L.: /facet: A Browser for Heterogeneous
Semantic Web Repositories. In I. Cruz, S. Decker, D. Allemang, C. Preist, D. Schwabe, P. Mika,
et al. (Ed.), ISWC 2006: Proceedings of the 5th International Semantic Web Conference.
LNCS 4273, pp. 272-285, Athens, GA, USA: Springer-Verlag, Berlin Heidelberg, 2006.
[7] Levene, M., Wheeldon, R.: Navigating the World Wide Web. In M. Levene, & A. Poulovassilis
(Eds.), Web Dynamics – Adapting to Change in Content, Size, Topology and Use, pp.
117-151, Springer-Verlag, Berlin Heidelberg, 2004.