Fabric Manager Users Guide, Version 6.1, Revision A - QLogic

2–Advanced Fabric Manager Capabilities
Mesh/Torus Topology Support
It is important to understand that the QLogic Fabric Manager’s shortestpath
algorithm can be used to temporarily route a Mesh/Torus for low bandwidth basic
operations such as fabric topology verification and the configuration of externally
managed switches. The shortestpath algorithm will be able to route the fabric in a
usable manner for such operations, however it will have credit loops. Such credit
loops will not cause issues for low bandwidth operations nor for operations where
only two nodes in the fabric are using high bandwidth (such as loading firmware
on a externally managed switch). However attempts to run real MPI jobs with
multiple nodes or collectives or even FastFabric MPI Performance verification
tools while using shortestpath are likely to induce credit loops and cause
extremely poor performance.
Topology Configuration in the Fabric Manager
When using the dor-updown algorithm, the Mesh/Torus topology information must
be configured in the Fabric Manager configuration file. Only the ISLs that
constitute the different dimensions of the Mesh or Torus must be defined. All ISLs
in the fabric which have not been configured in this section will be considered
invalid and ignored when the dor-updown algorithm is selected as the routing
algorithm.
The QLogic Fabric Manager only needs the “ISL pattern” to be defined. It merely
needs to know which pairs of ports will be connected to form each dimension. It
does not need to know any specific details about the switches. Replacement of
switches will not require any configuration changes.
This is necessary so that the Fabric Manager can be aware of the intended
topology and therefore properly configure SLs and VLs. This approach allows the
Fabric Manager to minimize impacts when large fabric changes occur, such as the
loss/reappearance of toroidal links or loss/reappearance of an entire dimension.
The SM will ignore the invalid links and proceed ahead with the sweep and will
make the best case effort to finish the sweep, so that other tools (like FastFabric)
can be used to verify the condition of the fabric. If there are warnings related to
invalid links, it is highly recommended to verify the cable connections in the fabric.
At the end of the sweep, the SM will show all the invalid ISL port pairs it found
during the sweep.
In Mesh fabrics, the QLogic Fabric Manager will allow Channel Adapters to be
connected to unused switch ports at the edges of each mesh dimension. For
example, in the picture below, when using 36 port switches. There could be
additional servers on edge and corner switches, such as:
• Switch E: 32 servers
• Switch B, D, F, H: 33 servers
• Switch A, C, G, I: 34 servers
IB0054608-01 B 2-15