NP - Cognitive Science

Introduction to Computational Linguistics Summer Term 2010
Intro CL | Session 8:
Feature Unification Grammars
Stefan Evert
Institute of Cognitive Science
University of Osnabrück

Context-free grammars
2

Context-free grammars
◆ Advantages of CFG
• generate “appropriate” structural analyses
• recursion & long-distance dependencies
• efficient parsers available (e.g. CYK with O(n 3 · m))
2

Context-free grammars
◆ Advantages of CFG
• generate “appropriate” structural analyses
• recursion & long-distance dependencies
• efficient parsers available (e.g. CYK with O(n 3 · m))
◆ Problems & shortcomings
• cannot account for all human languages
• modelling free word order (German, Latin)
• agreement & complementation
• broad-coverage grammars unintuitive & cumbersome
2

A simple phrase-structure CFG
◆ Phrase structure rules
• S → NP VP
VP → V NP NP | V NP | V | VP PP
NP → Det A N | Det N | A N | N | Pron | NP PP
PP → P NP
• no clausal complements, subcategorised PPs,
coordination, full NPs, adverbs, complex tenses, …
◆ Lexical rules for preterminals
• N → boy | boys | girl | girls | water | …
Det → the | a | no | one | two | …
V → give | gives | gave | sleep | sleeps | slept | …
3

A simple phrase-structure CFG
◆ Phrase structure rules
• S → NP VP
VP → V (NP (NP)) | VP PP
NP → (Det) (A) N | Pron | NP PP
PP → P NP
• no clausal complements, subcategorised PPs,
coordination, full NPs, adverbs, complex tenses, …
◆ Lexical rules for preterminals
• N → boy | boys | girl | girls | water | …
Det → the | a | no | one | two | …
V → give | gives | gave | sleep | sleeps | slept | …
4

A simple phrase-structure CFG
S
NP NP
VP
NP NP
Det N V Det N Det N P N
The boys gave the girl a glass of water
NP
PP
5

Problems of this CFG
6

Problems of this CFG
◆ Ungrammatical structures licensed by CFG
• the boys gives the girl a book ➞ agreement (number)
• the boy gave she a book ➞ agreement (case)
• the cats slept a book, the boy gave ➞ complementation
6

Problems of this CFG
◆ Ungrammatical structures licensed by CFG
• the boys gives the girl a book ➞ agreement (number)
• the boy gave she a book ➞ agreement (case)
• the cats slept a book, the boy gave ➞ complementation
◆ Word order variation, e.g. in German
Maria gibt dem Mann ein Buch.
Ein Buch gibt Maria dem Mann.
Dem Mann gibt Maria ein Buch.
Ein Buch gibt dem Mann Maria.
…
6

CFG solution: subcategorisation
7

CFG solution: subcategorisation
◆ Expand nonterminals into multiple,
fine-grained subcategories
• e.g. NP ➞ NPsg / NPpl
• agreement ensured by using only valid combinations
of subcategories in CFG productions
7

CFG solution: subcategorisation
◆ Expand nonterminals into multiple,
fine-grained subcategories
• e.g. NP ➞ NPsg / NPpl
• agreement ensured by using only valid combinations
of subcategories in CFG productions
◆ This procedure is called subcategorisation
• argument structure (complementation) of verbs is
one particular example of subcategorisation
(intransitive, transitive, ditransitive, …)
7

Verb complementation in CFG
8

Verb complementation in CFG
◆ The number and form of verbal complements
(argument structure) is lexically determined
8

Verb complementation in CFG
◆ The number and form of verbal complements
(argument structure) is lexically determined
◆ Many types of “subcategorisation frames”:
• intransitive: Peter left.
• transitive: Mary read [NP the book].
• ditransitive: Mary gave [NP Peter] [NP the book].
• ditrans-PP: Peter put [NP the book] [PP on a shelf].
• trans-CP: Paul says [CP that Mary kissed Peter].
• ditrans-VP: Paul saw [NP Mary] [VP kiss Peter].
…
8

Verb complementation in CFG
9

Verb complementation in CFG
◆ Add new “types” of verbs ➞ subcategories
• Vintrans → sleep | walk | run | leave | …
Vtrans → read | write | see | eat | leave | …
Vditrans → give | tell | write | leave | …
Vcomp → say | claim | think | believe | …
• inflected forms of verbs need to be listed on RHS
9

Verb complementation in CFG
◆ Add new “types” of verbs ➞ subcategories
• Vintrans → sleep | walk | run | leave | …
Vtrans → read | write | see | eat | leave | …
Vditrans → give | tell | write | leave | …
Vcomp → say | claim | think | believe | …
• inflected forms of verbs need to be listed on RHS
◆ Separate complementation rules for verb types
• VP → Vintrans
VP → Vtrans NP
VP → Vditrans NP NP | Vditrans NP PPto
VP → Vcomp CP
9

Agreement in CFG
10

Agreement in CFG
◆ English verbs agree with their subject NP
• in number (sg, pl) and person (1 st , 2 nd , 3 rd )
10

Agreement in CFG
◆ English verbs agree with their subject NP
• in number (sg, pl) and person (1 st , 2 nd , 3 rd )
◆ Introduce new NP and V categories
• NP ➞ NP1sg, NP2sg, NP3sg, NP1pl, NP2pl, NP3pl
• V ➞ V1sg, V2sg, V3sg, V1pl, V2pl, V3pl
10

Agreement in CFG
◆ New lexical entries & duplicated CFG rules
• S → NP1sg VP1sg | NP2sg VP2sg | NP3sg VP3sg | …
• VP1sg → V1sg (NP (NP)) | VP1sg PP
VP2sg → V2sg (NP (NP)) | VP2sg PP
…
• NP3sg → (Detsg) (A) Nsg | NP3sg PP
NP3pl → (Detpl) (A) Npl | NP3pl PP
• V1sg → go | went | say | said | …
V3sg → goes | went | says | said | …
• NP1sg → I, NP2sg → you, NP3sg → he | she | it, …
• Nsg → book|mouse|water|…, Npl → books|mice|…
11

Agreement in CFG: problems
12

Agreement in CFG: problems
◆ Multiplies out with verb complementation
• VP1sg → V1sg,intrans | V1sg,trans NP | V1sg,ditrans NP NP | …
VP2sg → V2sg,intrans | V2sg,trans NP | V2sg,ditrans NP NP | …
…
12

Agreement in CFG: problems
◆ Multiplies out with verb complementation
• VP1sg → V1sg,intrans | V1sg,trans NP | V1sg,ditrans NP NP | …
VP2sg → V2sg,intrans | V2sg,trans NP | V2sg,ditrans NP NP | …
…
◆ Underspecification ➞ redundant lexical entries
• leave can be V1sg, V2sg, V1pl, V2pl, or V3pl
• or rather: V1sg,intrans, V1sg,trans, V1sg,ditrans, V2sg,intrans,
V2sg,trans, V2sg,ditrans, V1pl,intrans, V1pl,trans, V1pl,ditrans, …
12

Agreement in CFG: problems
13

Agreement in CFG: problems
◆ Further dimensions of agreement
• German nouns agree with determiners and adjectives
in case (nom, gen, acc, dat), number (sg, pl) and
gender (masc, fem, neut)
• NP ➞ NPnom,sg,masc, NPgen,sg,masc, NPdat,sg,masc,
NPacc,sg,masc, NPnom,pl,masc, …, NPnom,sg,fem, …
• underspecified lexical entries: e.g. Blume (sg) and
Blumen (pl) underspecified for case ➞ 4 categories
13

Agreement in CFG: problems
◆ Further dimensions of agreement
• German nouns agree with determiners and adjectives
in case (nom, gen, acc, dat), number (sg, pl) and
gender (masc, fem, neut)
• NP ➞ NPnom,sg,masc, NPgen,sg,masc, NPdat,sg,masc,
NPacc,sg,masc, NPnom,pl,masc, …, NPnom,sg,fem, …
• underspecified lexical entries: e.g. Blume (sg) and
Blumen (pl) underspecified for case ➞ 4 categories
◆ Interaction with subject–verb agreement
• S → NPnom,sg,masc,1st VP1sg | NPnom,sg,masc,2nd VP2sg | …
| NPnom,sg,fem,1st VP1sg | NPnom,sg,fem,2nd VP2sg | …
13

Free word order in CFG
14

Free word order in CFG
◆ German ditransitives with arbitrary ordering
• S → NPnom VPditrans,acc,dat | NPacc VPditrans,nom,dat
| NPdat VPditrans,nom,acc
• VPditrans,acc,dat → Vditrans NPacc NPdat | Vditrans NPdat NPacc
VPditrans,nom,dat → Vditrans NPnom NPdat | Vditrans NPdat NPnom
VPditrans,nom,acc → Vditrans NPnom NPacc | Vditrans NPacc NPnom
• a second set of similar rules for verb-final clauses
14

Free word order in CFG
◆ German ditransitives with arbitrary ordering
• S → NPnom VPditrans,acc,dat | NPacc VPditrans,nom,dat
| NPdat VPditrans,nom,acc
• VPditrans,acc,dat → Vditrans NPacc NPdat | Vditrans NPdat NPacc
VPditrans,nom,dat → Vditrans NPnom NPdat | Vditrans NPdat NPnom
VPditrans,nom,acc → Vditrans NPnom NPacc | Vditrans NPacc NPnom
• a second set of similar rules for verb-final clauses
◆ Multiplies out with agreement & complementation
• NPnom is really NPnom,sg,masc,1st or NPnom,sg,masc,2nd or …
• Vditrans is really Vditrans,1sg or Vditrans,2sg or …
14

Problem summary
◆ New subcategories for different phenomena
multiplied out into huge number of CFG rules
• new rules and improvements of existing rules have to
be kept consistent across all subcategories
◆ Lexical entries become highly ambiguous
• redundancy in lexicon, becomes difficult to maintain
◆ Parsing with such CFG becomes inefficient
• large number of nonterminals and CFG rules
• high degree of (local) ambiguity
15

A solution: feature bundles
16

A solution: feature bundles
◆ Decompose syntactic subcategories into
bundles of features
• major category (S, NP, VP, PP, V, N, …)
• agreement (case, number, gender, person)
• verb complementation (intrans, trans, ditrans, …)
16

A solution: feature bundles
◆ Decompose syntactic subcategories into
bundles of features
• major category (S, NP, VP, PP, V, N, …)
• agreement (case, number, gender, person)
• verb complementation (intrans, trans, ditrans, …)
dem Mann: NPdat,sg,masc,3rd ➞
⎡
category
⎢
⎢agreement
⎣
np
⎡
case
⎢
⎢num
⎢
⎣gend
⎤
⎤⎥
dat ⎥
sg
⎥
⎥
⎥
⎥
⎥
⎥
masc⎦
⎥
⎦
pers 3
16

Lexical entries
for feature bundles
NPnom,sg,3rd → he | she | it
V3sg,trans → reads | eats | …
17

Lexical entries
for feature bundles
NPnom,sg,3rd → he | she | it
⎡
category
⎢
⎣agreement
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎦
⎥
⎦
pers 3
V3sg,trans → reads | eats | …
he | she | it
17

Lexical entries
for feature bundles
NPnom,sg,3rd → he | she | it
⎡
category
⎢
⎣agreement
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎦
⎥
⎦
pers 3
V3sg,trans → reads | eats | …
⎡
category
⎢
⎢agreement
⎣
v
num
pers
⎤
⎥
sg ⎥
3 ⎥
⎦
subcat trans
he | she | it
reads | eats | …
17

Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
VP3sg → V3sg,trans NPacc,pl,2nd | …
18

Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
18

Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
sg ⎥
3 ⎥
⎦
subcat trans
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
18

⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
sg ⎥
3 ⎥
⎦
subcat trans
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
19

⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
x
y
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
sg ⎥
3 ⎥
⎦
subcat trans
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num xsg
⎥
agr
⎦
pers 3
y
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
19

⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
x
y
x
y
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
xsg
⎥
y3
⎥
⎦
subcat trans
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num xsg
⎥
agr
⎦
pers 3
y
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
19

⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
Phrase-structure rules
for feature bundles
S → NPnom,sg,3rd VP3sg | …
cat s
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
VP3sg → V3sg,trans NPacc,pl,2nd | …
x
y
x
y
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
xsg
⎥
y3
⎥
⎦
subcat trans
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num xsg
⎥
agr
⎦
pers 3
y
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
?
?
19

a → b c
Phrase-structure rules
for feature bundles
= vp
= v
= np
= [= x]
= [= y]
= trans
= acc
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
x
y
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
xsg
⎥
y3
⎥
⎦
subcat trans
path
notation
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
?
?
20

VP → V NP
Phrase-structure rules
for feature bundles
=
=
= trans
= acc
⎡
⎤
cat vp
⎢ ⎥
⎢
⎣ num sg ⎥
agr
⎦
pers 3
x
y
⎡
cat
⎢
⎢agr
⎣
v
num
pers
⎤
⎥
xsg
⎥
y3
⎥
⎦
subcat trans
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
pl ⎥
⎦
pers 2
?
?
21

Underspecified feature bundles
⎡
⎤
cat np
⎣ ⎦
agr case acc
22

Underspecified feature bundles
◆ The object NP in our VP rule is underspecified
• only = acc
• encodes partial information
⎡
⎤
cat np
⎣ ⎦
agr case acc
22

Underspecified feature bundles
◆ The object NP in our VP rule is underspecified
• only = acc
• encodes partial information
◆ NP expansion rules require fully specified NP
• = acc
• = sg
• = 3
⎡
⎤
cat np
⎣ ⎦
agr case acc
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
◆ How can we match these feature bundles?
22

Unification
◆ Feature bundles are not matched by equality,
but by merging (partial) information
• possible whenever feature bundles are consistent
◆ This process is called unification
• unification result = “smallest” feature bundle that
contains all information from the two arguments
• unification does not add further information
23

Conflicting vs. consistent
feature bundles
24

Conflict:
Conflicting vs. consistent
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
feature bundles
↯
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
↯ ↯
24

Conflict:
Conflicting vs. consistent
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
Consistent:
feature bundles
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
↯
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
nom ⎥
⎥
⎥
sg ⎥
⎦
pers 3
↯ ↯
✓
⎡
⎤
cat np
⎣ ⎦
agr case acc
24

Subsumption
25

Subsumption
◆ Feature bundle A subsumes feature bundle B
iff B contains more information than A
• A and B must be consistent
• B may contain additional information
• A must not contain any information not present in B
25

Subsumption
◆ Feature bundle A subsumes feature bundle B
iff B contains more information than A
• A and B must be consistent
• B may contain additional information
• A must not contain any information not present in B
◆ Notation: A ⊑ B
⎡
⎤
cat np
⎣ ⎦
agr case acc
⊑
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
25

Unification
◆ Consistent feature bundles can be unified
• even if A does not subsume B, and vice versa
◆ Notation: A ⊔ B
⎡
⎤
cat np
⎢ ⎥
⎢
⎣ case acc ⎥
agr
⎦
pers 3
⊔
⎡
⎤
cat np
⎢ ⎥
⎢
⎣ case acc ⎥
agr
⎦
num sg
26

Unification
◆ Consistent feature bundles can be unified
• even if A does not subsume B, and vice versa
◆ Notation: A ⊔ B
⎡
⎤
cat np
⎢ ⎥
⎢
⎣ case acc ⎥
agr
⎦
pers 3
=
⊔
⎡
⎤
cat np
⎢ ⎥
⎢
⎣ case acc ⎥
agr
⎦
num sg
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣num
⎤
⎤⎥
acc ⎥
⎥
⎥
sg ⎥
⎦
pers 3
26

Underspecification
and unification
27

Underspecification
and unification
◆ Underspecification and unification are
important devices of feature grammars
• lexical entries are often underspecified
• unification combines partial information from
different feature bundles
27

Underspecification
and unification
◆ Underspecification and unification are
important devices of feature grammars
• lexical entries are often underspecified
• unification combines partial information from
different feature bundles
◆ Consider German NP agreement: die Blume
• NP → Det N
=
=
= 3
• merge (= unify) and expand AGR feature bundles
27

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
28

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
This is a problem we are
not going to deal with
→ multiple lexical entries
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
28

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
29

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
29

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
↯
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
29

Underspecification
⎡
cat
⎢
⎢agr
⎣
det
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
⎡
cat
⎢
⎣agr
det
⎡
⎣ case
⎤
⎤
⎥
nom, acc ⎥
⎦
num pl
and unification
die Blume
✓
⎡
⎤
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
29

Underspecification
and unification
die Blume
⎡
⎤
cat det
⎢ ⎡ ⎤⎥
⎢ case nom, acc
⎥
⎢ ⎢
⎥
⎥
⎢agr
⎢
⎥
⎥
⎣ ⎣gend
fem
⎥
⎦
⊔
⎡
⎤
num sg
cat n
⎢ ⎥
⎢
⎣ gend fem ⎥
agr
⎦
num sg
29

Underspecification
and unification
die Blume
⎡
cat
⎢
⎢agr
⎣
np
⎡
case
⎢
⎣gend
⎤
⎤⎥
nom, acc
⎥
⎥
⎥
⎥
⎥
fem
⎥
⎦
num sg
30

Towards feature structures
31

Towards feature structures
◆ We still need many different rules for verb
complementation (depending on )
• VP → Vintrans
VP → Vtrans NPacc
VP → Vtrans,dat NPdat (in German)
VP → Vcomp CP
• specified in terms of feature constraints, of course
31

Towards feature structures
◆ We still need many different rules for verb
complementation (depending on )
• VP → Vintrans
VP → Vtrans NPacc
VP → Vtrans,dat NPdat (in German)
VP → Vcomp CP
• specified in terms of feature constraints, of course
◆ We would like to be able to write this:
• VP → V x
= x
• lexical entry of V specifies category, case, … of argument
31

Towards feature structures
32

Towards feature structures
◆ Value of the feature is a full
feature bundle, not just an atomic symbol
• looked like a feature bundle, but can be seen
as a convenient notation for grouping related features
• for verb complementation, we want to allow arbitrarily
complex structures as values of
32

Towards feature structures
◆ Value of the feature is a full
feature bundle, not just an atomic symbol
• looked like a feature bundle, but can be seen
as a convenient notation for grouping related features
• for verb complementation, we want to allow arbitrarily
complex structures as values of
◆ Feature bundles that
allow such complex
values are called
feature structures
⎡
⎤
cat v
⎢ ⎥
⎢
num sg ⎥
⎢agr
⎥
⎢ pers 3 ⎥
⎢ ⎡
⎤⎥
⎢ cat np ⎥
⎣subcat
⎣
⎥
⎦
agr case acc
32

Feature structures
33

Feature structures
◆ Recursive unification of feature structures
• atomic values must match exactly to be unified
• complex values are unified recursively
• atomic value cannot be unified with feature structure
(except for empty, or “null” feature structure [])
33

Feature structures
◆ Recursive unification of feature structures
• atomic values must match exactly to be unified
• complex values are unified recursively
• atomic value cannot be unified with feature structure
(except for empty, or “null” feature structure [])
◆ Typical case: unification of sub-structures
• e.g. agreement information from Det, A and N
33

Feature structures
◆ Recursive unification of feature structures
• atomic values must match exactly to be unified
• complex values are unified recursively
• atomic value cannot be unified with feature structure
(except for empty, or “null” feature structure [])
◆ Typical case: unification of sub-structures
• e.g. agreement information from Det, A and N
◆ Implementation based on graph representation
• useful to define and understand unification procedure
33

Representation of
feature structures
⎡
⎤
cat v
⎢ ⎥
⎢
num sg ⎥
⎢agr
⎥
⎢ pers 3 ⎥
⎢ ⎡
⎤⎥
⎢ cat np ⎥
⎣subcat
⎣
⎥
⎦
agr case acc
attribute-value matrix (AVM)
34

= v
= sg
= 3
= np
= acc
path notation
Representation of
feature structures
⎡
⎤
cat v
⎢ ⎥
⎢
num sg ⎥
⎢agr
⎥
⎢ pers 3 ⎥
⎢ ⎡
⎤⎥
⎢ cat np ⎥
⎣subcat
⎣
⎥
⎦
agr case acc
attribute-value matrix (AVM)
34

= v
= sg
= 3
= np
= acc
path notation
CAT
AGR
SUBCAT
graph
notation
Representation of
feature structures
v
NUM
PERS
CAT
AGR
sg
3
np
⎡
⎤
cat v
⎢ ⎥
⎢
num sg ⎥
⎢agr
⎥
⎢ pers 3 ⎥
⎢ ⎡
⎤⎥
⎢ cat np ⎥
⎣subcat
⎣
⎥
⎦
agr case acc
attribute-value matrix (AVM)
CASE
acc
34

Unification example
=
⎡
cat
⎢
⎢agr
⎢
⎣subcat
v
num
pers
⎡
cat
⎣
agr
sg
3
np
case
⎤
⎥
⎤ ⎥
⎥
⎦
acc
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
35

Unification example
=
⎡
cat
⎢
⎢agr
⎢
⎣subcat
v
num
pers
⎡
cat
⎣
agr
sg
3
np
case
⎤
⎥
⎤ ⎥
⎥
⎦
acc
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
35

Unification example
=
⎡
cat
⎢
⎢agr
⎢
⎣subcat
v
num
pers
⎡
cat
⎣
agr
sg
3
np
case
⎤
⎥
⎤ ⎥
⎥
⎦
acc
CAT
AGR
SUBCAT
v
NUM
PERS
CAT
AGR
sg
3
np
CASE
acc
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
CAT
ANIMATE
AGR
np
+
NUM
PERS
pl
1
35

Unification example
=
⎡
cat
⎢
⎢agr
⎢
⎣subcat
v
num
pers
⎡
cat
⎣
agr
sg
3
np
case
⎤
⎥
⎤ ⎥
⎥
⎦
acc
CAT
AGR
SUBCAT
v
NUM
PERS
CAT
AGR
sg
3
np
CASE
acc
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
CAT
ANIMATE
AGR
np
+
NUM
PERS
pl
1
35

CAT
AGR
SUBCAT
CAT
AGR
SUBCAT
v
Unification example
v
NUM
PERS
CAT
AGR
NUM
PERS
sg
3
np
CAT
ANIMATE
AGR
CASE
sg
3
np
+
acc
CASE
NUM
PERS
acc
pl
1
CAT
ANIMATE
AGR
np
+
NUM
PERS
pl
1
36

Shared structure & reentrancy
37

Shared structure & reentrancy
◆ Merged information vs. structure sharing
• unification can either build new FS with merged
information, or identify unified substructures
(➞ multiple features point to the same FS)
• standard algorithm modifies and links unification
arguments in the process (Jurafsky & Martin, Ch. 15)
37

Shared structure & reentrancy
◆ Merged information vs. structure sharing
• unification can either build new FS with merged
information, or identify unified substructures
(➞ multiple features point to the same FS)
• standard algorithm modifies and links unification
arguments in the process (Jurafsky & Martin, Ch. 15)
◆ Feature structures with shared substructures
are called reentrant
• = nodes with multiple incoming edges in graph
• important distinction: equal values (value identity)
vs. shared structure (reentrancy)
37

Shared structure and reentrancy
we speak
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
⎡
⎤
cat vp
⎢
⎥
⎢
⎣ num pl ⎥
agr
⎦
tense pres
38

Shared structure and reentrancy
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
CAT
ANIMATE
AGR
we speak
np
+
NUM
PERS
=
pl
1
⎡
⎤
cat vp
⎢
⎥
⎢
⎣ num pl ⎥
agr
⎦
tense pres
CAT
AGR
vp
NUM
TENSE
pl
pres
38

Shared structure and reentrancy
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
CAT
ANIMATE
AGR
we speak
np
+
NUM
PERS
=
pl
1
⎡
⎤
cat vp
⎢
⎥
⎢
⎣ num pl ⎥
agr
⎦
tense pres
CAT
AGR
vp
NUM
TENSE
pl
pres
38

Shared structure and reentrancy
we speak
⎡
cat
⎢
⎢agr
⎣
np
num
pers
⎤
⎥
pl ⎥
1 ⎥
⎦
animate +
CAT
ANIMATE
AGR
AGR
CAT
np
+
vp
NUM
PERS
TENSE
⎡
⎤
cat vp
⎢
⎥
⎢
⎣ num pl ⎥
agr
⎦
tense pres
pl
1
pres
39

Shared structure and reentrancy
we speak
⎡
cat
⎢
⎣agr
⎤
np
⎥
1 ⎦
animate +
CAT
ANIMATE
AGR
AGR
CAT
np
+
vp
⎡
cat
⎢
⎣agr
vp
⎡
num
⎢
1⎣pers
pl
1
⎤
⎤ ⎥
⎥
⎥
⎦
tense pres
NUM
PERS
TENSE
pl
1
pres
40

Shared structure and reentrancy
we speak
⎡
cat
⎢
⎣agr
⎤
np
⎥
1 ⎦
animate +
reentrancy
CAT
ANIMATE
AGR
AGR
CAT
np
+
vp
⎡
cat
⎢
⎣agr
vp
⎡
num
⎢
1⎣pers
pl
1
⎤
⎤ ⎥
⎥
⎥
⎦
tense pres
NUM
PERS
TENSE
pl
1
pres
40

Full parse example
⎡
⎤
cat v
⎢ ⎡ ⎤ ⎥
⎢ case nom ⎥
⎢ ⎢ ⎥ ⎥
⎢agr
⎣pers
3 ⎦ ⎥
⎢ num sg ⎥
⎡
⎤ ⎡
⎤⎢
⎡
⎤⎥
⎢
cat det cat n ⎢ cat np ⎥
⎣ ⎦
⎣ ⎦⎣subcat
⎣
⎥
⎦
agr pers 3 agr pers 3
agr case acc
The sheep likes me
41

Full parse example
VP → V NP
=
=
⎡
⎤
cat v
⎢ ⎡ ⎤ ⎥
⎢ case nom ⎥
⎢ ⎢ ⎥ ⎥
⎢agr
⎣pers
3 ⎦ ⎥
⎢ num sg ⎥
⎡
⎤ ⎡
⎤⎢
⎡
⎤⎥
⎢
cat det cat n ⎢ cat np ⎥
⎣ ⎦
⎣ ⎦⎣subcat
⎣
⎥
⎦
agr pers 3 agr pers 3
agr case acc
The sheep likes me
41

⎡
⎤ ⎡
⎤
cat det cat n
⎣ ⎦
⎣ ⎦
agr pers 3 agr pers 3
Full parse example
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
The sheep likes me
42

⎡
⎤ ⎡
⎤
cat det cat n
⎣ ⎦
⎣ ⎦
agr pers 3 agr pers 3
Full parse example
NP → Det N
=
=
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
The sheep likes me
42

⎡
⎤
cat np
⎣ ⎦
agr 3 pers 3
cat det cat n
agr 3
Full parse example
agr 3
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
The sheep likes me
43

⎡
⎤
cat np
⎣ ⎦
agr 3 pers 3
cat det cat n
agr 3
Full parse example
agr 3
S → NP VP
=
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
The sheep likes me
43

cat np
agr 1
cat det cat n
agr 1
Full parse example
agr 1
cat s
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
The sheep likes me
44

cat np
agr 1
cat det cat n
agr 1
Full parse example
agr 1
cat s
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
2
⎡
cat
⎢
⎣agr
np
⎡
case
⎢
⎣pers
⎤
⎤⎥
acc ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
The sheep likes me
45

cat np
agr 1
cat det cat n
agr 1
Full parse example
agr 1
cat s
CAT AGR CAT AGR AGR AGR CAT CAT AGR SUBCAT CAT
⎡
cat
⎢
⎣agr
vp
⎡
case
⎢
1⎣pers
⎤
⎤⎥
nom ⎥
AGR
⎥
⎥
3 ⎦
⎥
⎦
num sg
n np
det vp v
sg
NUMPERS CASE
3
nom
⎡
cat
⎤
v
⎢
⎣agr
⎥ 1 ⎦
subcat 2
1
2
⎡
cat
⎢
⎣agr
np
⎡
acc case
⎢
⎣pers
⎤
⎤⎥
acc ⎥
⎥
⎥
3 ⎦
⎥
⎦
num sg
The sheep likes me
sg
CAT
np
NUM PERS CASE
45

Feature CFG in Python/NLTK
% start S
toy_english.fcfg
## lexical rules (NB: full AVM must be on single line!)
Det[AGR=[PERS=3]] -> 'the'
N[AGR=[PERS=3]] -> 'sheep'
V[AGR=[CASE=nom, PERS=3, NUM=sg],…
…SUBCAT=[CAT=np, AGR=[CASE=acc]]] -> 'likes'
Pron[AGR=[CASE=acc, PERS=1, NUM=sg]] -> 'me'
## phrase-structure rules
S -> NP[AGR=?x] VP[AGR=?x]
NP[AGR=?x] -> Det[AGR=?x] N[AGR=?x]
NP[AGR=?x] -> Pron[AGR=?x]
VP[AGR=?x] -> V[AGR=?x, SUBCAT=[CAT=np, AGR=?y]] NP[AGR=?y]
46

Feature CFG in Python/NLTK
from nltk.parse.featurechart import load_earley
parser = load_earley("file:toy_english.fcfg")
# add trace=2 to see how parser works
input = "the sheep likes me".split()
trees = parser.nbest_parse(input)
for tree in trees:
print tree # no true unification of AVMs!
# graphical presentation of parse trees
from nltk.draw.tree import draw_trees
draw_trees(*trees) # better layout with typewriter font
47

Reentrancy in Python/NLTK
from nltk.featstruct import FeatStruct
AVM = """
[ SUBJ=[CAT=np, AGR=(1)[NUM=sg, PERS=3, CASE=nom]],
OBJ =(2)[CAT=np, AGR=[NUM=sg, PERS=1, CASE=acc]],
PRED=[CAT=v, AGR->(1), SUBCAT->(2)]
]
"""
# - coindexed FS must be defined before it's referenced
# - cannot represent reentrancy between multiple AVMs
fs = FeatStruct(AVM)
print fs
48

Advanced topics
◆ Verb argument lists in SUBCAT feature
◆ Handling movement with “slash” features
◆ Lexicon-driven grammars ➞ free word order
◆ Grammar formalisms based on feature
structures and unification
• PATR (Shieber 1983), FUG (Kay 1979), GPSG (Gazdar
et al. 1985), LFG (Bresnan 1982–present), HPSG
(Pollard & Sag 1987, 1994–present)
◆ Parsing with feature structures
49

Reading & homework
◆ Reading assignment
NLTK Book, Chapter 9
• optional: Jurafsky & Martin, Ch. 15
◆ Homework assignment #5
• will be available on homepage later today, as usual
50