Eckhard Bick - VISL

The NURC speech corpus (“Norma urbana culta”) 241 , described in (Castilho, 1989)
ca. 100.000 words, for testing purposes only (Bick, 1998-2)
Brazilian transcribed interviews, monologue and conversation
Folha de São Paulo (1994-1996 running editions)
ca. 90.000.000 words, for a research project 242 at the University of São Paulo
Brazilian newspaper texts (all topics)
The Tycho Brahe corpus (17th century sample), cp. www.ime.usp.br/~tycho
ca. 50.000 words, for external use
historical Portuguese (Antonio das Chagas) 243
To make automatic comparison possible, the system’s morphological tag set was
filtered into specific synthetic tags also recognized by the probabilistic tagger used
in the Tycho Brahe project.
The NILC corpus (Núcleo Interinstitucinal de Lingüística Computacional,
http://www.nilc.icmc.sc.usp.br/) 244
ca. 39.000.000 words, used for testing purposes
ca. 100.000 words for external evaluation
journalistic, didactic and student essay texts
Originally, I tagged this corpus for internal purposes only, as a means of testing the
robustness of the morphological part of the CG parser. However, part of the corpus
(100.000 words of mixed science, literature and economy) also exists in a handtagged
version established by NILC in order to train a probabilistic or hybrid
tagging system. Like in the Tycho Brahe case, the CG morphological tag set proved
rich enough to allow filtering into the specific synthetic tags preferred by the NILC
team, making direct comparison possible. A special challenge in this case was the
distinction between 6 different verbal “valency word classes”, VAUX, VLIG, VINT,
VTD, VTI, VBI, roughly matching the (instantiated) CG valency tags , ,
, , and /, respectively.
As can be seen from the list, the parser can handle a fairly broad spektrum of
Portuguese language data. The largest task, the tagging of 3 years of running
newspaper text (Folha de São Paulo) for a research group at the Catholic University
of São Paulo, took 50 hours of CPU processing time on a linux system, averaging a
speed of 500 words per second, and demonstrated the robustness of the system not
only in grammatical, but also in technical terms.
So far, no large scale semantic annotation has been attempted, and automatic
post-CG tree structure annotation of running text has only be performed on test texts
and a 20.000 word corpus of teaching sentences.
241
I would like to thank professor Ataliba de Castilho for making the NURC corpus accessible to me in electronic form.
242
In this connection, I would like to mention Tony Berber Sardinha who is having a great deal of to-be-rewarded
confidence in my parser.
243
This text and the Tycho Brahe tag set was kindly made available by Helena Britto.
244
I would like to thank the NILC team for letting me have a go at their corpus, and Sandra Maria Aluisio for having
patience in discussing tagging differences with me.
- 430 -