Fabric Manager Users Guide, Version 6.1, Revision A - QLogic

4–Virtual Fabrics
Configuration
MPI and vFabrics
The PKey given is ORed with 0x8000 to define the PKey for the multicast group.
This creates an ib0.9234 interface that can be assigned the appropriate IP
address and IP parameters.
The operation of Linux with multiple IPoIB subnets is very similar to the use of IP
over Ethernet when VLANs are being used. It is up to the adminstrator which
network interfaces are actually used and assigned IP addresses. This is done
using the standard ifcfg files.
Many MPI implementations do not follow the InfiniBand Standard. MPIs such as
openmpi, mvapich, and mvapich2 when ran using InfiniBand verbs, use
nonstandard mechanisms to establish connections, do not make PathRecord
requests, and do not use ServiceIDs to determine connection parameters.
To use vFabrics in conjunction with those MPIs, the PKey and BaseSL must be
manually specified in the VirtualFabric section. The selected PKey and SL
also need to be specified to MPI at job startup. Some examples of this are shown
in the /opt/iba/src/mpi_apps/ofed*.params files that are provided with
QLogic FastFabric Toolset.
When using MPI with the QLogic PSM API, path record queries can be enabled in
conjunction with the QLogic distributed SA. In which case there is no need to
manually specify the PKey and SL at MPI job startup. Refer to the QLogic OFED+
Host Software User Guide for more information about enabling Path Record
queries in PSM.
Pre-Created Multicast Groups
The Multicast.Multicast Group section of the qlogic_fm.xml
configuration file can specify multicast groups that should be pre-created by the
SM. If neither a VirtualFabric nor PKey is specified for a given pre-created
MulticastGroup, it is created for all vFabrics that contain the given MGID as an
application.
When no MGIDs are explicitly specified, the necessary IPoIB multicast groups for
IPv4 and IPv6 are pre-created against the selected VirtualFabric/PKeys (all
applicable vFabrics if no specific VirtualFabric/PKey selected). When such
automatic pre-creation occurs, the PKey assigned to the vFabric is inserted into
the MGIDs per the IPoIB standard.
An example of this capability is provided in the sample configuration file.
4-20 IB0054608-01 B