Eckhard Bick - VISL

Even trigrams, however, are far from expressing real syntactic structure, and the
lexical collocation knowledge expressed in Hidden MMs is diluted considerably by the
fact that it is seen through a word class filter. While the lexicalisation problem to a
certain degree also haunts rule based grammars, the syntactic structure problem is
"unique" to probabilistic HMM grammars and resides in the "Markov assumption" that
p(tn|t1 ... tn-1) = p(tn|tn-1) (for bigrams), or = p(tn|tn-1tn-2) (for trigrams). In generative
grammar, syntactic structure is handled in an explicit way, and functions both as the
traditional objective and as the main tool of disambiguation. In CG, finally, syntactic
structure can be expressed, but results as a kind of by-product of sequential contextual
disambiguation rules. Of course, it does matter what the objective of disambiguation is:
in fact, as shown in chapter 3.7.3, two thirds of all morphological CG-rules make do
without "global" rules, i.e. they could be expressed as statistical n-gram transitions
(though even here, most rules use a larger-than-trigram window), while only 10-20% of
syntactic CG-rules can manage without unbounded contexts.
One proposed solution to the syntax problem in probabilistic systems has been to
expand context-free grammars (CFGs) into probabilistic context-free grammars
(PCFGs), where CFG-productions are assigned conditional probabilities on the nonterminal
being expanded, and the probability for a given syntactic (sub)tree can be
computed as the product of the probabilities of all productions involved. The two
readings of the sentence 'Einstein lectures last.', for instance, can be described by the
following mini-PCFG, consisting of CFG-rules weighted with - arbitrary - production
probabilities:
(A) (B)
(2) Einstein lectures last.
1. S -> NP VP (p = 0.5) S S
2. VP -> v (p = 0.3)
3. VP -> v adv (p = 0.2)
4. NP -> n (p =0.4) NP VP NP VP
5. NP -> n n (p =0.1)
n n v n v adv
The complex NP reading (Einstein @>N lectures @SUBJ> last @FMV) involves
productions 1, 2 and 5, yielding a complex probability of 0.5 x 0.3 x 0.1 = 0.015, while
the single noun reading (Einstein @SUBJ> lectures @FMV last @