Large-Scale Semi-Supervised Learning for Natural Language ...

More documents

Recommendations

Info

Lang. Feat. Wt. ExampleFr (Bitext) ées - ed +8.0 vérifiées:verifiedJp (Dict.) ru - l +5.9 penaruti:penaltyDe (Bitext) k - c +5.5 kreativ:creativeDe Btxt eren$ - e$ +5.2 ignorieren:ignoreFr Btxt lement$ - ly$ +5.2 admirablement admirablyEs (Bitext) ar - ating +5.0 acelerar:acceleratingRs (Dict.) irov - +4.9 motivirovat:motivateGr (Dict.) f - ph +4.1 symfonia:symphonyGr (Dict.) kos - c +3.3 anarchikos:anarchicGr (Dict.) os$ - y$ -2.5 anarchikos:anarchyJp (Dict.) ou - ou -2.6 handoutai:handoutEs (Dict.) - un -3.1 balance:unbalanceFr (Bitext) s$ - ly$ -4.2 fervents:ferventlyFr (Dict.) er$ - er$ -5.0 former:formerEs (Bitext) mos - s -5.1 toleramos:toleratesTable 7.4: Example features and weights for various Alignment-Based Discriminative classifiers(Foreign-English, negative pairs in italics).age derivational and inflectional morphology. For example, Greek-English pairs with theadjective-ending correspondence kos-c, e.g. anarchikos:anarchic, are favoured, but pairswith the adjective ending in Greek and noun ending in English, os$-y$, are penalized; indeed,by our definition, anarchikos:anarchy is not cognate. In a bitext, the feature ées-edcaptures that feminine-plural inflection of past tense verbs in French corresponds to regularpast tense in English. On the other hand, words ending in the Spanish first person pluralverb suffix -amos are rarely translated to English words ending with the suffix -s, causingmos-s to be penalized. The ability to leverage negative features, learned from appropriatecounter examples, is a key innovation of our discriminative framework.Table 7.5 gives the top pairs scored by our system on the three bitext and three of thedictionary test sets. Notice that unlike traditional similarity measures that always scoreidentical words higher than all other pairs, by virtue of our feature weighting, our discriminativeclassifier prefers some pairs with very characteristic spelling changes.We performed error analysis by looking at all the pairs our system scored quite confidently(highly positive or highly negative similarity), but which were labeled oppositely.Highly-scored false positives arose equally from 1) actual cognates not linked as translationsin the data, 2) related words with diverged meanings, e.g. the only error in Table 7.5:makaroni in Greek actually means spaghetti in English (makaronada is macaroni), and 3)the same word stem, a different part of speech (e.g. the Greek/English adjective/noun synonymos:synonym).Meanwhile, inspection of the highly-confident false negatives revealedsome (often erroneously-aligned in the bitext) positive pairs with incidental letter match(e.g. the French/English recettes:proceeds) that we would not actually deem to be cognate.Thus the errors that our system makes are often either linguistically interesting or point outmistakes in our automatically-labeled bitext and (to a lesser extent) dictionary data.105
Fr-En Bitext Es-En Bitext De-En Bitextfilm:film agenda:agenda akt:actambassadeur:ambassador natural:natural asthma:asthmabio:bio márgenes:margins lobby:lobbyradios:radios hormonal:hormonal homosexuell:homosexualabusif:abusive radón:radon brutale:brutalirréfutable:irrefutable higiénico:hygienic inzidenz:incidenceGr-En Dict. Jp-En Dict. Rs-En Dict.alkali:alkali baiohoronikusu:bioholonics aerozol:aerosolmakaroni:macaroni mafia:mafia gondola:gondolaadrenalini:adrenaline manierisumu:manierisme rubidiy:rubidiumflamingko:flamingo ebonaito:ebonite panteon:pantheonspasmodikos:spasmodic oratorio:oratorio antonim:antonymamvrosia:ambrosia mineraru:mineral gladiator:gladiatorTable 7.5: Highest scored pairs by Alignment-Based Discriminative classifier (negative pairin italics).7.7 Conclusion and Future WorkThis is the first research to apply discriminative string similarity to the task of cognateidentification. We have introduced and successfully applied an alignment-based frameworkfor discriminative similarity that consistently demonstrates improved performance in bothbitext and dictionary-based cognate identification on six language pairs. Our improvedapproach can be applied in any of the diverse applications where traditional similarity measureslike edit distance and LCSR are prevalent. We have also made available our cognateidentification data sets, which will be of interest to general string similarity researchers.Furthermore, we have provided a natural framework for future cognate identificationresearch. Phonetic, semantic, or syntactic features could be included within our discriminativeinfrastructure to aid in the identification of cognates in text. In particular, we couldinvestigate approaches that do not require the bilingual dictionaries or bitexts to generatetraining data. For example, researchers have automatically developed translation lexiconsby seeing if words from each language have similar frequencies, contexts [Koehn andKnight, 2002], burstiness, inverse document frequencies, and date distributions [Schaferand Yarowsky, 2002]. Semantic and string similarity might be learned jointly with a cotrainingor bootstrapping approach [Klementiev and Roth, 2006]. We may also comparealignment-based discriminative string similarity with a more complex discriminative modelthat learns the alignments as latent structure [McCallum et al., 2005].Since the original publication of this work, we have also applied the alignment-basedstring similarity model to the task of transliteration identification [Jiampojamarn et al.,2010] with good results. In that work, we also proposed a new model of string similaritythat uses a string kernel to implicitly represent substring pairs of arbitrary length, liftingone of the computational limitations of the model in this chapter.In addition, we also looked specifically at the cognate identification problem from amultilingual perspective [Bergsma and Kondrak, 2007b]. While the current chapter looksto detect cognates in pairs of languages, we provided a methodology that directly forms106
Page 1 and 2:
University of AlbertaLarge-Scale Se
Page 5 and 6:
Table of Contents1 Introduction 11.
Page 7 and 8:
7 Alignment-Based Discriminative St
Page 9 and 10:
List of Figures2.1 The linear class
Page 11 and 12:
drawn in by establishing a partial
Page 13 and 14:
(2) “He saw the trophy won yester
Page 15 and 16:
actual sentence said, “My son’s
Page 17 and 18:
Uses Web-Scale N-grams Auto-Creates
Page 19 and 20:
spelling correction, and the identi
Page 21 and 22:
Chapter 2Supervised and Semi-Superv
Page 23 and 24:
emphasis on “deliverables and eva
Page 25 and 26:
Figure 2.1: The linear classifier h
Page 27 and 28:
The above experimental set-up is so
Page 29 and 30:
and discriminative models therefore
Page 31 and 32:
their slack value). In practice, I
Page 33 and 34:
One way to find a better solution i
Page 35 and 36:
Figure 2.2: Learning from labeled a
Page 37 and 38:
algorithm). Yarowsky used it for wo
Page 39 and 40:
Learning with Natural Automatic Exa
Page 41 and 42:
positive examples from any collecti
Page 43 and 44:
generated word clusters. Several re
Page 45 and 46:
One common disambiguation task is t
Page 47 and 48:
3.2.2 Web-Scale Statistics in NLPEx
Page 49 and 50:
For each target wordv 0 , there are
Page 51 and 52:
ut without counts for the class pri
Page 53 and 54:
Accuracy (%)10090807060SUPERLMSUMLM
Page 55 and 56:
We also follow Carlson et al. [2001
Page 57 and 58:
Set BASE [Golding and Roth, 1999] T
Page 59 and 60:
pronoun (#3) guarantees that at the
Page 61 and 62:
807876F-Score747270Stemmed patterns
Page 63 and 64: anaphoricity by [Denis and Baldridg
Page 65 and 66: ter, we present a simple technique
Page 67 and 68: We seek weights such that the class
Page 69 and 70: each optimum performance is at most
Page 71 and 72: We now show that ¯w T (diag(¯p)
Page 73 and 74: Training ExamplesSystem 10 100 1K 1
Page 75 and 76: Since we wanted the system to learn
Page 77 and 78: Chapter 5Creating Robust Supervised
Page 79 and 80: § In-Domain (IN) Out-of-Domain #1
Page 81 and 82: Adjective ordering is also needed i
Page 83 and 84: Accuracy (%)10095908580757065601001
Page 85 and 86: System IN O1 O2Baseline 66.9 44.6 6
Page 87 and 88: 90% of the time in Gutenberg. The L
Page 89 and 90: VBN/VBD distinction by providing re
Page 91 and 92: other tasks we only had a handful o
Page 93 and 94: without the need for manual annotat
Page 95 and 96: DSP uses these labels to identify o
Page 97 and 98: Semantic classesMotivated by previo
Page 99 and 100: empirical Pr(n|v) in Equation (6.2)
Page 101 and 102: Verb Plaus./Implaus. Resnik Dagan e
Page 103 and 104: SystemAccMost-Recent Noun 17.9%Maxi
Page 105 and 106: Chapter 7Alignment-Based Discrimina
Page 107 and 108: ious measures to learn the recurren
Page 109 and 110: how labeled word pairs can be colle
Page 111 and 112: Figure 7.1: LCSR histogram and poly
Page 113: 0.711-pt Average Precision0.60.50.4
Page 117 and 118: Chapter 8Conclusions and Future Wor
Page 119 and 120: 8.3 Future WorkThis section outline
Page 121 and 122: My focus is thus on enabling robust
Page 123 and 124: [Bergsma and Cherry, 2010] Shane Be
Page 125 and 126: [Church and Mercer, 1993] Kenneth W
Page 127 and 128: [Grefenstette, 1999] Gregory Grefen
Page 129 and 130: [Koehn, 2005] Philipp Koehn. Europa
Page 131 and 132: [Mihalcea and Moldovan, 1999] Rada
Page 133 and 134: [Ristad and Yianilos, 1998] Eric Sv
Page 135 and 136: [Wang et al., 2008] Qin Iris Wang,
Page 137: NNP noun, proper, singular Motown V
show all

Large-Scale Semi-Supervised Learning for Natural Language ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?