Fabric Manager Users Guide, Version 6.1, Revision A - QLogic

2–Advanced Fabric Manager Capabilities
Packet and Switch Timers
Packet LifeTime
• HeadOfQueueLife (HoqLife)
• SwitchLifeTime
• VLStallCount
These can be used to relieve fabric congestion and avoid fabric deadlocks by
discarding packets. Discards help prevent back pressure from propagating deep
into the core of the fabric, however such discards will cause end nodes to time-out
and retransmit.
If a packet stays at the Head of a Switch Egress Port for more than HoqLife, it is
discarded. Similarly a packet queued in a switch for more than SwitchLifetime is
discarded. SwitchLifetime and HoqLife can also be set to infinite in which case no
discards will occur.
VLStallCount controls a second tier more aggressive discard. If VLStallCount
packets in a row are discarded due to HoqLife by a given VL on an egress port,
that egress port's VL enters the VL Stalled State and discards all that VL's egress
packets for 8*HoqLife.
Packets discarded for any of these reasons will be included in the TxDiscards
counter for the Port, which can be queried using FastFabric. Such discards are
also included in the Congestion information monitored by the PM and available
using FastFabric tools such as iba_top, iba_rfm and iba_paquery. A
congestion which is severe enough to cause packet discards is given a heavy
weight so that it will not go unnoticed.
Within an Host Channel Adapter/Target Channel Adapter, every Reliable Queue
Pair (QP) has a time-out configured. If there is no acknowledgment (ACK) for a
transmitted QP packet within the time-out, the QP will retry the send. There is a
limit on retries (up to 7) after which the QP will fail with a Retry Timeout Exceeded
error.
The InfiniBand standard defines that the timeout for a QP should be computed
based on the Packet LifeTime reported by the SA in a PathRecord. The LifeTime
represents the one way transit time through the fabric. Therefore, the actual QP
timeout will be at least 2x the Packet LifeTime (plus some overhead to allow for
processing delays in the Target Channel Adapter/Host Channel Adapter at each
end of the fabric).
Careful selection of Packet LifeTime (and QP time-outs) is important. If time-outs
are set too large, then the impact of a lost packet could be significant. Conversely
if the time-outs are set to low, then minor fabric delays could cause unnecessary
retries and possibly even Retry Timeout Exceeded errors and the resulting
disruption of applications.
The SM allows for two approaches to configure Packet LifeTime
2-20 IB0054608-01 B