Best Practices for SAP BI using DB2 9 for z/OS - IBM Redbooks

the chassis for blade-to-blade connections. Data traffic between System p and
the BI Accelerator cluster of blades was mainly through Remote Function Calls
(RFCs). A standard Maximum Transmission Unit (MTU) size of 1500 was used
because the data package traffic was small.
Connection between blade cluster and DS8300
A GPFS file system was mounted and accessible to all 140 blades. Generally, a
Storage Area Network (SAN) is effective with 64 blades or fewer. As the number
of blades scales up, GPFS topology with its cluster of network shared disk (NSD)
server nodes performs better. In our large environment with 10 BladeCenter®
chassis and 140 blades in total, 10 GPFS NSD servers were employed. They
were System x3650 machines (two dual-core CPUs at 3.0 GHz with 4 GB RAM)
running Linux SLES9 SP3.
Since InfiniBand (IB) has better bandwidth and latency than GbE, we chose IB
for the network between the blades and the GPFS NSD servers. Each configured
HS21 blade chassis had two embedded Cisco SDR IB switches. Blades 1
through 7 were configured to use the first IB switch, and blades 8 through 14
were on the second one, to have a non-blocking configuration. Each SDR IB
daughter card on each blade is rated at 10 Gbps, or 4x, where x = 2.5 Gbps.
Each SDR IB switch had two external 12x ports and two external 4x ports, with
an aggregate bandwidth of 80 Gbps. Each GPFS NSD server had 4x or 10 Gbps
IB connection.
On the other end, the 10 GPFS NSD servers were connected to GPFS storage
(DS8300) via twenty 4 Gbps Fibre Channel connections, two connections per
GPFS NSD server.
Since each blade had GbE and IB interfaces, blade-to-blade communication
could use GbE or IB. In our case, IB was mainly used for reading and writing to
the GPFS file system and GbE was used for blade-to-blade communication.
11.3 SAP BI database and InfoCube load
One of the major challenges for project Jupiter was to build a large, complex
(25 TB) BI database within a very short period of time. According to the plan, a
minimum InfoCube data load rate of 1 TB per day was needed in order to meet
the aggressive schedules. In the following sections, we describe the database
structures and the InfoCube load techniques that we implemented to meet the
project requirements.
234 Best Practices for SAP BI using DB2 9 for z/OS