Fabric Manager Users Guide, Version 6.1, Revision A - QLogic

4–Virtual Fabrics
Configuration
The Default Partition
The InfiniBand Standard requires every fabric to have a default partition. At a
minimum this vFabric is used by all end nodes to interact with the SM/SA.
To meet this requirement there must be an enabled vFabric with the following
items:
• A PKey of 0x7ffff (or 0xffff)
• The SA application (for example, an Application with
SA)
• The only enabled vFabric that includes the SA application
• Have AllSMs (for example, a DeviceGroup with
AllSMs) as a Member
• Have All (for example, a DeviceGroup with All) as
a Member or LimitedMember
• Additional Application as needed
• Additional DeviceGroups as Members or LimitedMembers as needed
• All other vFabric policies (QoS, Security, MTU...) may be set as needed
NOTE:
It is a requirement that all chassis-managed spines (for example, SWE0) be
a member of the default partition so that the chassis can access and
manage its own leaf switch chips.
IPoIB and vFabrics
vFabrics are configured within the hardware in the order in which they appear in
the configuration file. When IPoIB runs, it uses the first PKey on the given port for
the default (ib0...) network device. Therefore, it is best to place the
Networking/IPoIB vFabric first.
IPoIB starts with the PKey for the IPoIB interface and uses that to define the MGID
of the VLAN’s broadcast multicast group. Many aspects of the IPoIB VLAN are
defined by the multicast group itself. Among them are the MTU for the VLAN.
A given port or node can participate in more than one IPoIB subnet. Each such
subnet must have its own unique PKey. For vFabrics other than the first, the PKey
should be manually specified in the VirtualFabric section and the PKey must
be supplied to IPoIB. On some Linux systems with the QLogic InfiniBand or OFED
stack, additional IPoIB virtual interfaces can be created by a command such as:
echo 0x1234 > /sys/class/net/ib0/create_child
IB0054608-01 B 4-19