Eckhard Bick - VISL

other languages from both the Germanic, Romance and Finno-Ugric language families
(Swedish, German, French, Finnish etc.) 105 . A mature CG-grammar for the
morphological level (word class or PoS disambiguation), typically consists of at least
1.000-2.000 rules. For the English ENGCG system, word class error rates of under 0.3%
have been reported at a disambiguation level of 94-97% (Voutilainen, 1992).
In a recent direct comparison 106 between an updated ENGCG and a statistical
tagger trained on a 357.000 107 word section of the Brown corpus, Samuelsson &
Voutilainen (1999) found that error rates for the Constraint Grammar system were at
least an order of magnitude lower than those of the probabilistic system at comparable
disambiguation levels. Thus, ENGCG error rates were 0.1% with a 1.07 tag/word ratio
and 0.43% with a 1.026 tag/word ration, while the statistical system achieved error rates
of 2.8% and 3.72%, respectively.
Constraint Grammar type rules have also been used in hybrid systems, for
instance where an automated learning algorithm is trained on a morphologically tagged
corpus with the objective of constructing or selecting local context discard rules. Thus,
Lindberg (1998), using Progol inductive logic programming 108 and a ±2 word context
window, reports 98% recall in Swedish test texts, with a residual ambiguity of 1.13
readings pr. word, and a rule body of 7000 rules. Another hybrid system is decribed in
Padró i Cirera (1997) , where a relaxation labelling tagger is applied to English and
Spanish. In this system, CG style rules for POS-tagging were integrated with HMM
tagging, creating a statistical model for for the distribution of tag targets and context
conditions. Constraint rules were partly learned from a training corpus using statistical
decision trees, and partly hand-written on the basis of output errors in probabilistic
HMM taggers 109 . In comparison with HMM and relaxation labelling base line taggers,
both types of constraint rules improved tagger performance individually, and resulted -
when combined - in an overall precision rate of 97.35% for fully disambiguated Wall
Street Journal text.
While hybrid systems thus seem to offer some advances in comparison with
ordinary HMM modelling and related techniques, they are still far from achieving
ENGCG level results, one likely explanation residing in the fact that the automatically
learned rules of such systems (so far) lack the global scope (i.e. sentence scope) and
105 For a short comparison of CG systems, cp. chapter 8.1.
106 Both systems used the same tag set: CG-tags were filtered into the kind of fused single tags typical of statistical taggers.
Both systems were tested on the same 50.000 word benchmark text, consisting of both journalistic, scientific and manual
excerpts.
107 At this training corpus size, the learning curve of the statistical tagger flattened out, suggesting that larger training
corpora would not lead to any significant improvement in tagging performance.
108 In addition, Lindberg used so-called “lexical” rules (not to be induced), removing rare readings of frequent word forms,
much like the heuristic rules in a regular CG - but with the important difference, that the CG rules would be
used after at least one round of regular disambiguation, whereas Lindberg’s lexical rules came into play before ordinary
(induced) rules.
109 With only 20 linguist-written rules, the balance was heavily in favour of the automatically generated constraints (8473).
- 149 -