Unni Cathrine Eiken February 2005
Unni Cathrine Eiken February 2005 Unni Cathrine Eiken February 2005
The aim of the tests performed in this section was to see if the accuracy of the classifier could be enhanced by training on a complete context pattern with the appropriate concept group as category label. Test 1 Training set: EPAS_arg1 with no pronouns and concept classes as category label, argument 1 ignored. Test method: leave-one-out Result: 56,54% (108/191) In this test, the classifier was trained on two features of the EPAS, ignoring argument 1. This test is analogous to test 2 in section 4.1.2.1, which had an accuracy of 41,20% correct classifications. In addition to the 108 correctly classified instances, additional five instances were assigned categories which are semantically similar to the correct category. This was true for Kripos-spesialist (Kripos specialist), politimester (chief of police), medarbeider (co-worker) and polititjenestefolk (police workers), which were all assigned the category POLICE. These words are not part of the concept group POLICE, but are obviously semantically related to the members of this concept group. Had these words occurred more frequently in the data material, they could have been expected to show a distribution allowing for their inclusion in POLICE. The results of this test suggest that labeling EPAS with concept group labels heightens the accuracy of the classifier. This is not surprising, given the fact that a higher number of context patterns/EPAS are labeled with the same category in such an approach, making the generalisable material larger. Test 2 Training set: EPAS_arg1 with no pronouns and concept classes as category label. Test method: leave-one-out Result: 86,91% (166/191) This test was performed to see if training the classifier on the entire structure of an EPAS increases the accuracy of assigning concept labels to the structures. The classifier was trained on all three features of the EPAS. In this case, the classifier performed with a fairly high accuracy, assigning the correct category in 166 of 191 cases. It is obviously an advantage that all parts of 76
the EPAS can be used in the classification phase when the category to be assigned is not literally a part of the structures to be learnt from. Test 3 Training set: EPAS_arg1 with no pronouns and concept classes as category label, argument 1 ignored. Test set: EPAS_arg1 with pronouns and concept classes as category label Result: 76,92% (20/26) As was the case in the corresponding test in section 4.1.2.1, two of the wrongly assigned categories were in fact within the same semantic group as the correct category. Regarding these as correct assignments would heighten the result to 84,61%. As was the case in the previous two tests, the assigned categories in these cases are too infrequent in the EPAS list to surface in the associated concept groups. Test 4 Training set: EPAS_arg2 with no pronouns and concept classes as category label, argument 2 ignored. Test set: EPAS_arg2 with pronouns and concept classes as category label Result: 83,33% (5/6) When training the classifier on the EPAS_arg2 list using concept class labels as categories and testing on the set of EPAS with pronouns in argument 2 position, the classifier resolved five of the six test instances correctly. In the corresponding test in section 4.1.2, the classifier did not assign the correct category in any of the six test cases. We did, however, see that five of the test instances were assigned categories which were semantically similar to the correct antecedent. In view of the results in the initial test, it came as no surprise that the classifier performed so much better when used in connection with the concept class labels. 77
- Page 31 and 32: section. The theory dates back to t
- Page 33 and 34: 2.2.2 Different types of context So
- Page 35 and 36: neighbours. For example, a target w
- Page 37 and 38: with it. Selectional constraints al
- Page 39 and 40: 3 From text to EPAS - the extractio
- Page 41 and 42: 3.2 Predicate-argument structures "
- Page 43 and 44: speaker flexibility with regards to
- Page 45 and 46: and woman occur together both in su
- Page 47 and 48: occur with. Arguments which are unl
- Page 49 and 50: 3.3.1 NorGram in outline Norsk komp
- Page 51 and 52: Figure 3 The most useful structure
- Page 53 and 54: 3.4 Altering the source As already
- Page 55 and 56: (3- 12) (3- 13) Politiet leter ette
- Page 57 and 58: ARG1 and ARG2 arrays display a valu
- Page 59 and 60: (3- 20) Anne Slåtten bodde i et st
- Page 61 and 62: value and highly desirable. As such
- Page 63 and 64: this project, this can be interpret
- Page 65 and 66: The process of classifying the cons
- Page 67 and 68: There are several different distanc
- Page 69 and 70: . ankomme,etterforsker,?,? ankomme,
- Page 71 and 72: Test 2 Training set: EPAS_arg1 with
- Page 73 and 74: The training and test material was
- Page 75 and 76: • level 0: words which co-occur w
- Page 77 and 78: (4- 9) avklare,obduksjon,? bede-om,
- Page 79 and 80: (4-10) below shows the output for t
- Page 81: In the introduction to this chapter
- Page 85 and 86: exemption of jobbe-utfra, none of t
- Page 87 and 88: antecedent for (4-15a). In the case
- Page 89 and 90: Figure 7 Interestingly enough, howe
- Page 91 and 92: When testing on knowledge-dependent
- Page 93 and 94: Firth, J. R. (1957): A synopsis of
- Page 95 and 96: Appendix A: Ekstraktor.pl - algorit
- Page 97 and 98: finnARG2(); This function has exact
- Page 99 and 100: #legger lest linje inn i @prt derso
- Page 101 and 102: sub fjernEP{ #fjerner elementer fra
- Page 103 and 104: } splice(@ARGx); $imax = @ARG3ep; @
- Page 105 and 106: } else{ } } } push(@liste, $ARG0ep[
- Page 107 and 108: 101 Appendix C: the EPAS list 23-å
- Page 109 and 110: 103 obdusere,,kvinne observere,,23-
- Page 111 and 112: Appendix D: Text aligned with EPAS
- Page 113 and 114: eventualiteter. Vi varslet Kripos.
- Page 115 and 116: Etterforskerne har flere observasjo
- Page 117 and 118: # Subrutine som tar inn argumentnum
- Page 119 and 120: Appendix F: POS-based structures SE
- Page 121: Vi har ingen spesiell teori som vi
The aim of the tests performed in this section was to see if the accuracy of the classifier could be<br />
enhanced by training on a complete context pattern with the appropriate concept group as<br />
category label.<br />
Test 1<br />
Training set: EPAS_arg1 with no pronouns and concept classes as category label, argument 1<br />
ignored.<br />
Test method: leave-one-out<br />
Result: 56,54% (108/191)<br />
In this test, the classifier was trained on two features of the EPAS, ignoring argument 1. This<br />
test is analogous to test 2 in section 4.1.2.1, which had an accuracy of 41,20% correct<br />
classifications. In addition to the 108 correctly classified instances, additional five instances<br />
were assigned categories which are semantically similar to the correct category. This was true<br />
for Kripos-spesialist (Kripos specialist), politimester (chief of police), medarbeider (co-worker)<br />
and polititjenestefolk (police workers), which were all assigned the category POLICE. These<br />
words are not part of the concept group POLICE, but are obviously semantically related to the<br />
members of this concept group. Had these words occurred more frequently in the data material,<br />
they could have been expected to show a distribution allowing for their inclusion in POLICE.<br />
The results of this test suggest that labeling EPAS with concept group labels heightens the<br />
accuracy of the classifier. This is not surprising, given the fact that a higher number of context<br />
patterns/EPAS are labeled with the same category in such an approach, making the generalisable<br />
material larger.<br />
Test 2<br />
Training set: EPAS_arg1 with no pronouns and concept classes as category label.<br />
Test method: leave-one-out<br />
Result: 86,91% (166/191)<br />
This test was performed to see if training the classifier on the entire structure of an EPAS<br />
increases the accuracy of assigning concept labels to the structures. The classifier was trained on<br />
all three features of the EPAS. In this case, the classifier performed with a fairly high accuracy,<br />
assigning the correct category in 166 of 191 cases. It is obviously an advantage that all parts of<br />
76
Change language