Unni Cathrine Eiken February 2005

language processing tasks tend to benefit from lazy learning methods, particularly because the
individual examples in the training material are not abstracted away from in the process of
creating rules. When a new data instance is classified, it is compared to all previously seen
examples, including low-frequent ones. This suggests that in the case of relatively small data
sets, such as the one in the present work, MBL tools are particularly suitable.
By consulting previously seen data and estimating the similarity between old and new instances
of data, MBL algorithms such as TiMBL are able to calculate the likelihood of new instances of
data. This is done by creating a classifier which essentially consists of an example set of
particular patterns together with their associated categories. The classifier can subsequently
classify unknown input patterns by applying algorithms to calculate the similarity, or distance,
to the known patterns stored in memory. The Nearest Neighbor approach is one commonly used
means to estimate this distance and is described in more detail in the following section.
4.1.1 The Nearest Neighbor approach
Daelemans et al. (2003, p. 19) state that all MBL approaches are founded on the classical k-
Nearest Neighbor (k-NN) method of classification (Cover and Hart 1967). This approach
classifies patterns of numeric data by using information gained from examining and classifying
pattern distributions observed in a data collection. In the k-NN algorithm, a new instance of data
is classified as nearest to a set of previously classified points. The intuition is that observations
which are close together will have categories which are close together. When classifying a new
instance of data, the k-NN approach weights the known information about the closest similar
data instances most heavily. In other words, a new instance of data is classified in the category
of its nearest neighbour. In large samples, this rule can be modified to classifying according to
the majority of the nearest neighbours, rather than just using the single nearest neighbour. The
k-NN approach has several implementations in TiMBL. As TiMBL is designed to classify
linguistic patterns, which in most cases consist of discrete data values and allow for a large
number of attributes with varying relevance, the k-NN algorithm is not used directly. Instead,
the classification of discrete data is made possible through a modified version of the k-NN
approach, as well as other algorithms.
60