IPP Annual Report 2007 - Max-Planck-Institut für Plasmaphysik ...

Introduction
The Rechenzentrum Garching
(RZG) traditionally provides supercomputing
power and archival
services for the IPP and other
Max Planck Institutes throughout
Germany. Besides operation
of the systems, application support
is given to Max Planck Institutes
with high-end computing
needs in fusion research, materials science, astrophysics,
and other fields. Large amounts of experimental data from
the fusion devices of the IPP, satellite data of the MPI for
Extraterrestrial Physics (MPE) at the Garching site, and data
from supercomputer simulations are administered and stored
with high lifetimes. In addition, the RZG provides network
and standard services for the IPP and part of the other MPIs
at the Garching site. The experimental data acquisition software
development group XDV for both the W7-X fusion
experiment and the current ASDEX Upgrade fusion experiment
operates as part of the RZG.
Furthermore, the RZG is engaged in several large projects in
collaboration with other, partly international scientific institutions.
One of these projects is a bioinformatics project
dealing with genome research, another one the ATLAS project,
which is part of the LHC experiment at CERN. And finally,
the RZG is a member of DEISA, a consortium of the leading
European supercomputing centers supporting the advancements
of computational sciences in Europe. In this project the
RZG holds the task leaderships for global file systems, for the
operation of the distributed infrastructure, for applications
enabling, and for joint research activities in plasma physics and
in materials science. All these projects are based on new software
technologies, among others so-called Grid-Middleware
tools. Since the importance of Grid technology for international
collaborations has significantly increased in recent years,
broad competence has to be established also in this field.
Major Hardware Changes
The supercomputer landscape, consisting of the IBM pSeries
690 based supercomputer and the IBM p575 based cluster of
8-way nodes, has been augmented in September 2007 with an
IBM BlueGene/P system with 8,192 PowerPC@850MHz-based
cores which is especially suited for applications scaling up to
1,024 cores and beyond. Furthermore, a series of Linux clusters
with Intel Xeon and AMD Opteron processors is operated,
which has been further extended in the area of Intel Xeon quadcore
and AMD dual-core Opteron based Linux clusters. Besides
the generally available systems, dedicated compute servers
are operated and maintained for: IPP, Fritz-Haber-Institute, MPI
for Astrophysics, MPI for Polymer Research, MPI for Quantum
Computer Center Garching
Head: Dipl.-Inf. Stefan Heinzel
A major task has been the optimization of complex
applications from plasma physics, materials
science and other disciplines. The data acquisition
system of W7-X has been implemented
on a smaller existing device (WEGA) and reaches
its test phase. For the FP6 EU project DEISA,
codes from Plasma Physics (GENE and ORB5)
have been enabled for hyperscaling to make
efficient use of up to 32,000 processors.
95
Developments for High-End Computing
Optics, MPI for Extraterrestrial
Physics, MPI for Biochemistry,
MPI for Chemical Physics of
Solids, MPI for Physics and MPI
for Astronomy. In the mass storage
area, the capacity of the new automated
tape library Sun SL8500
has been extended to 6 PB of
compressed data. Both LTO3 and
LTO2 tape drives are supported.
The application group of the RZG gives support in the field of
high-performance computing. This includes supervising the
start-up of new parallel codes, giving advice in case of software
and performance problems as well as providing development
software for the different platforms. One of the major
tasks, however, is the optimization of complex codes from
plasma physics, materials sciences and other disciplines on the
respective, in general parallel high-performance target architecture.
This requires a deep understanding and algorithmic
knowledge and is usually done in close collaboration with
the authors from the respective disciplines. In what follows
selected optimization projects are presented in more detail.
GEM Code
The GEM (Gyrofluid-ElectroMagnetic) code from the IPP
plasma theory solves nonlinear gyrofluid equations for electrons
and one or more ion species in tokamak geometry. It is
restricted to a local approach in geometry, a so-called fluxtube
approach. According to the parallelization concept of
one-dimensional domain decomposition along the magnetic
field the maximum number of processors to be used was 16.
The new code version GEMR treats the full geometry in the
radial x-direction. Hence, more realistic simulations of turbulence
in experiments like JET and ITER are now in progress.
However, the necessary grid resolution of at least 1024×512×16
is already far too large to be run on just 16 processors.
Correspondingly, the scaling properties of the GEMR code
had to be improved towards many hundreds of processors.
After the single-processor performance had been increased
by 50 %, the parallelization concept was expanded to a twodimensional
domain decomposition by additionally parallelizing
along the x-coordinate. For this purpose the index structure
of the most important arrays had to be adapted to avoid
unnecessary copying in connection with communication. The
parallelization of the matrix solver in x-direction was a nontrivial
task which could finally be solved with an elaborated
parallel transpose of the data and corresponding matrices.
As a result, the scalability could be increased by the envisaged
factor of 32; a parallel efficiency of 89 % from 64 to 512
processors could be observed. Hence, both the distributed