Fabric Manager Users Guide, Version 6.1, Revision A - QLogic

2–Advanced Fabric Manager Capabilities
Mesh/Torus Topology Support
• Pairwise – cover every lid on both sides at least once: SLID1/DLID1,
SLID2/DLID2, SLID1/DLID3, SLID2/DLID4
• OrderAll – cover every combination, but start with pairwise set:
SLID1/DLID1, SLID2/DLID2, SLID1/DLID3, SLID2/DLID4 SLID1/DLID2,
SLID1/DLID4, SLID2/DLID1, SLID2/DLID3
• SrcDstAll – cover every combination with simple all src, all dst:
SLID1/DLID1, SLID1/DLID2, SLID1/DLID3, SLID1/DLID4 SLID2/DLID1,
SLID2/DLID2, SLID2/DLID3, SLID2/DLID4
Mesh/Torus Topology Support
Mesh/Torus fabrics are a popular option when building large fabrics. This option
can trade-off interconnect cost with performance. By their nature, Mesh/Torus
fabrics are highly oversubscribed and can have significant differences in latency
depending on which node pairs are communicating. Applications must be
carefully placed and tuned to obtain optimum performance.
In contrast Fat-Tree networks offer much higher bisectional bandwidth and much
more consistent node-to-node latency. When budget permits, a Fat-Tree is often
recommended.
Mesh/Torus fabrics consist of a grid of edge-switches interconnected in a uniform
pattern. Mesh/Torus fabrics must be built from individual edge-switches (such as
12200 or 12300 models or comparable blade switches). Mesh/Torus fabrics may
not be built using larger multi-slot chassis such as the 12800 models.
For example a two-dimensional 4x4 mesh can appear as shown in Figure 2-4.
Figure 2-4. 2D 4x4 Mesh Fabric Example
More dimensions can decrease node-to-node latency but often reduce
node-to-node bandwidth and require more complex cable routing. For example a
three-dimensional 3x3x3 mesh can appear as follows:
2-10 IB0054608-01 B