sentiment-annotated lexicon construction for an urdu ... - Paas.com.pk

Pakistan Journal of Science (Vol. 63 No. 4 Dec, 2011)
parsing based chunking. It uses lexicon for comparing all
the words/phrases present in the text. As a result, all the
subjective terms in the given text become annotated. On
the basis of the polarities of individual words, the
sentence and then its total review polarity is calculated.
The overall system performance is evaluated by using a
corpus of movie reviews in Urdu language. The
classification algorithm is applied on the review corpus.
Each subjective word in the review is compared with
lexicon entries for the computation of the polarity scores.
MATERIAL AND METHODS
In this section, the construction, structure and
integration of the sentiment-annotated lexicon of the
Urdu words developed for a sentiment classification
model is described. The model is designed to distinguish
between the objective and subjective terms in a given
review. Objective terms are with neutral sentiments,
which have no effect on the final decision of the
classification and subjective terms are considered as the
carriers of the sentiments and their presence can alter the
final classification. Keeping this distinction in view, the
lexicon entries are also categorized as objective and
subjective terms. Before going into details, some terms
are defined below:
• Orientation. Orientation describes either the
positivity or the negativity of a lexicon entry. For
most of the entries, orientation is predefined during
lexicon construction phase. But, in a given text it can
be altered with the use of a polarity shifter in the
sentence, e.g. the word ‏”اچھھھا“‏ (acha, good) have
positive orientation but, with the polarity shifter “
expression, (naheen, not), it becomes a negative ‏”نہیں
i.e., نہیں“‏ ‏”اچھا (acha naheen, not good). Moreover,
the orientation of some words (though their number
is few) is highly domain specific or depends upon the
context within which they are used. But, these two
issues are beyond the scope of this research.
• Intensity. This is the intensity of orientation of a
lexicon entry. This describes the force of positivity
or negativity of a term. Usually, the modifiers, e.g., “
(bohat, more) describe the intensity of an ‏”بہھھت
expression. Like other languages, in Urdu there are
three degrees of intensity; absolute (only positive or
negative orientation), comparatives (two distinct
entities are compared with each other) and
superlative (one of all entities is with highest
orientation)
• Polarity. The polarity mark is annotated with each
lexicon entry to show its orientation and intensity.
This is done at the implementation level.
Lexicon Construction: A sentiment-annotated lexicon
becomes more intricate as compared to other Natural
Language Processing (NLP) lexicons. There are two
reasons for this intricacy:
• Each lexicon entry demonstrates its polarity
information in addition to its orthographic,
phonological, syntactic and, morphological features.
This polarity information is usually represented as
either positive, or negative or neutral. For example,
SentiWordNet (Andreevskaia and Bergler, 2006),
use triplets [positive, negative, objectives], with
minimum value 0.0 and maximum 1.0.
• Most of the words exhibit multiple orientations
depending upon their use and domain. For example,
“This damage is everlasting”. In this sentence, the
everlasting is a positive word, but the comment’s
overall orientation is negative. Also, unpredictable is
a positive word when used about a movie’s plot, and
becomes negative for the performance of a
microwave oven.
Construction Steps: The lexicon construction task is
divided into following steps:
Figure 1. Structure of the sentiment-annotated lexicon
with respect to O and I
• Categorize the words either subjective or objective.
When the classification algorithm is applied on these
words, then the classifier simply ignores objective
terms, in this way its performance totally depends
upon subjective words.
• Categorize these words according to morphological
rules, which work at the word level. These rules can
change the structure, meaning, and part of speech of
the words. For example, rules for marking of an
adjective with the noun it qualifies, etc.
• Identify their grammatical rules, which describe the
possible structures of a sentence and position of the
parts of speech with respect to each other. As Urdu is
a free order language so theses rules are more
difficult to define and implement. For example, use
of modifiers with adjectives or use of auxiliaries with
verbs, etc.
• Discover relationships between different lexicon
entries. These relationships can define synonyms,
antonyms, and cross references, etc.
219