Eckhard Bick - VISL

(d) as part of words. For instance, $” will become a tag (left quote
border) if attached left of an alphanumeric string, and (right quote border) if
attached right. Also, abbreviations often include punctuation (. , - /), which is
especially problematic, since ambiguity with regard to sentence boundary
punctuation arises. To solve the ambiguity, the preprocessor consults an abbreviation
lexicon file and checks for typical sentence-initial/final context or typical context for
individual abbreviations.
Second, the preprocessor separates what it thinks are words by line feeds.
Here, the basic assumption of word-hood defines words as alphanumeric strings
separated by blank spaces, hyphens, non-abbreviation-punctuation, line feeds or tabs.
The reason for including hyphenation in the list is the need to morphologically
analyse enclitic and mesoclitic pronouns (e.g. ‘dar-lhe-ei’), and to decrease the
number of - lexiconwise - unknown words: The elements of hyphenated strings can
thus be recognised and analysed individually by the PALMORF analyser, even if the
compound as such does not figure in the lexicon. Thus, a word class and inflexional
analysis can usually be provided and passed on to the syntactic and higher modules
of the parser, even if only the last part of a hyphenated string is “analysable”.
Third, for pragmatic reasons, a number of polylexicals has been entered in the
PALMORF lexicon, consisting of several space- or hyphen-separated units that
would otherwise qualify as individual words (e.g. ‘guarda-chuva’, ‘em vez de’).
These polylexicals have been defined ad hoc by parsing needs (e.g. complex
prepositions), semantic considerations (machine translation) or dictionary tradition.
Polylexicals are treated like ordinary words by the parser, i.e. assigned form and
function tags etc., and can be addressed as individual contexts by Constraint
Grammar rules. In the newest version of the parser, one type of polylexical is
assembled independently of existing lexicon entries: Proper noun chains are fused
into polylexical “words” if specified patterns of capital letters, non-Portuguese letter
combinations and name chain particles (like ‘de’, ‘von’, ‘van’ etc.) are matched.
Criteria for the heuristic identification of non-Portuguese strings are, among
others, letters like ‘y’ and ‘w’, gemination of letters other than ‘r’ and ‘s’, and wordfinal
letters other than vowels, ‘r’, ‘s’ and ‘m’. Apart from name recognition,
identification of non-Portuguese strings is useful in connection with hyphenated
word chains - which will not be split if they contain at least one non-Portuguese
element, in order to avoid “accidental” (i.e. affix or inflexion-heuristics based)
assignment of non-noun word class 9 .
2.2.2.3 Data bases and searching techniques
On start-up the program arranges its data-bases in a particular way in RAM:
a) the grammatical lexicon is organised alphabetically with grammatical information
attached to the head word string. Each grammatical field has its own pointer. The
9 N (noun) and PROP (proper noun) are the overwhelminly most common word classes for foreign language material in
Portuguese.
- 19 -