Eckhard Bick - VISL

In a brute force approach, for an average word ambiguity of 2, two to the power of n
combinations would have to be computed. Such exponential complexity growth is, of
course, quite prohibitive. But since what is wanted is only the most likely reading (and
not the probabilities of all possible readings), the program can be set to only use the
highest probability chain encountered so far (i.e. from word 1 up to word i) when
moving on to the next word in the string (i.e. making the transition i -> i+1). This socalled
Viterbi-algorithm yields linear (and therefore manageable) complexity growth,
where the number of operations is proportional to 2n for n words that are on average
two-way ambiguous. Due to the problem of limited training data, zero-probability
transitions have to be replaced by small default values or by lower-n-gram values (i.e.
trigrams by bigrams). Other necessary ad-hoc solutions include heuristics for proper
nouns and lexicon failures (e.g. the use of suffix/PoS probabilities).
Interestingly, while the existence of PoS-lexica is a conditio-sine-qua-non for
most languages, the lack of a tagged training corpus for the estimation of transition
probabilities can be partly compensated for by estimating the parameters of the HMM
by means of an iterative re-estimation process on a previously untagged corpus (called
"forward-backward"-algorithm or Baum-Welch-algorithm). On the other hand, if a
sizeable tagged corpus is available for the language concerned, even the lack of a
lexicon is no real hurdle, since a lexicon file can be automatically compiled from the
tagged corpus, and will have a fair coverage at least for texts from the same domain.
Thus, the 1 million word Brown corpus contains some 70.000 word forms. The
importance of good lexicon coverage has been tested by Eeg-Olofsson (1991, IV p.43)
for a system combining lexicon entries with a heuristics based on 610 suffix strings:
using a 50.000 word corpus of spoken English, the system had an error rate of 2.4%
with full lexicon coverage, but 6% when using a lexicon compiled from one half of the
corpus and then tested on the other. Even with a large suffix module a sizeable lexicon
appears to be necessary, in order to cover those words that are exceptions to the suffix
patterns.
The big advantage of probabilistic taggers is that they are fast, and can be trained
in a short time, without the need of writing a real grammar of rules. Biasing a
probabilistic tagger by adding hand written rules or exceptions, may actually have an
adverse effect on its performance, since intervening on the behalf of a few irregular
words, for example, would interfere with the much more important statistical modelling
of the regular "majority" cases (Chanod and Tapanainen, 1994). Rumours have it that
such phenomena, as well as development speed and cross-language portability of
probabilistic tools, have made some commercial NLP enterprises believe that system
improvement can actually be improved by firing a linguist (and hiring a mathematician
instead). This view, of course, opportunistically ignores the fact that without linguists,
there would be no lexica and no tagged corpora to train a probabilistic parser on in the
first place.
- 136 -