Military Communications and Information Technology: A Trusted ...

Chapter 3: Information Technology for Interoperability and Decision...
275
ISAF-MT
In our research project Machine Translation for ISAF Forces (ISAF-MT) [4]
we have built a Dari-German translation system. As the project takes place in a German
– U.S. cooperation, research in SMT for Dari and English is also being
conducted. The objective of our project is the application of statistical machine
translation technology for the construction of a Dari-German translation system
in a military context.
The software framework we used for the ISAF-MT project is Moses [19].
Moses is the most widely used toolbox for constructing SMT systems. It is an opensource
project and has a very large user and developer community. This ensures
that the latest concepts and techniques are available for developers of SMT systems.
In the course of our project we built a parallel German-Dari corpus because no
corpus for this language pair was available. We have built a text corpus that consists
mainly of news texts focusing on OSINT terrorism topics. To find a compromise
between size of training data and quality of translation, we applied a multilevel
approach for corpus creation. On the one hand, we efficiently extracted parallel
texts from the internet to meet the requirement of corpus size. On the other
hand, high quality translations were generated by native speakers and professional
translators. Our current corpus contains about 27 000 lines. 13 000 lines are web
extracted news text and about 4500 lines are high quality translations. About 9500
corpus lines consist of diverse text material, such as open source subtitles, public
translation examples, the Dari constitution and texts of common information.
We further improved the system by integrating a Dari-German dictionary of about
80 000 entries that also contains some military terms.
In the course of our project we ran different experiments for the improvement
of system performance. In the following, we will describe some of the objects
of the experiments that significantly influenced SMT performance.
• Quality of the training corpus: We ran experiments with different subsets
of the corpus, focusing on different translation quality or type of text. We also
looked into various ways of integrating dictionaries into our training data
in the most beneficial way.
• Corpus preparation: System performance improved by the normalization
of the training corpus, e.g., in terms of different Unicode normalizations
and normalization that were done specifically for the language Dari.
• Software experiments: To build our SMT system we tried different available
software, e.g., for training of translation model or language model.
• Training configuration: The Moses framework provides different possible
approaches and configurations for SMT. We looked into various settings
(e.g. maximum length of trained phrases) as well as into the integration
of linguistic knowledge (e.g., lemmas) into the translation model.