DesignCon 2002

DesignCon 2008
BladeServer 10 Gb/s Ethernet
Backplane Design With
Equalization Using Statistical
Channel Analysis
Pravin Patel, IBM
Email: pravinp@us.ibm.com
Phone: +1-919-543-3837
Jeff Cutcher, Ansoft Corporation
Email: jcutcher@ansoft.com
Tony Donisi, Ansoft Corporation
Email: tony_dw@comcast.net
Michael Tsuk, Ansoft Corporation
Email: mtsuk@ansoft.com
Steve Pytel, Ansoft Corporation
Email: spytel@ansoft.com

Abstract
A fundamentally important aspect of high speed channel design at 10 Gb/s is that
significant optimization of the physical channel is required. Path degradation in channels,
from ISI effects, causes poor signal quality and must be minimized or eliminated. This
paper explores how equalization, statistical analysis, and accurate models can be used to
optimize high speed channels. A case study, based on a 10 Gb/s BladeServer Ethernet
backplane, will serve to illustrate how an integrated approach using equalization, accurate
physical models, and statistical simulation can be used to optimize the design of an
industrial application.
Author(s) Biography
Pravin Patel
Pravin Patel pravinp@us.ibm.com Pravin Patel is a Senior Engineer and Technical
Leader working for IBM xSeries Server Development. He is involved in defining product
architecture, creating product strategies, and time and Frequency Domain SI
analysis/simulation of Intel base server products. His current activities include the design
and analysis of Serdes interfaces for BladeCenter product. He performs modeling,
simulation and measurements of the high speed serial link Channels. The other areas of
his responsibilities include design and development of models for line cards, backplanes,
traces, via and performing simulations for system level voltage and timing budgets and
jitter characterization. He is an active member of T11.2 and IEEE802.3 channel model
standard committee from IBM. He received a B.S. degree in electrical engineering from
the New Jersey Institute of Technology in 1989. Pravin has received 12 U.S. Patents, 3
IBM Invention Plateau Achievement Awards, 7th IBM Authors Recognition Awards, one
IBM Outstanding Technical Achievement Awards, and One IBM Corporate Award.
IBM. He has authored or co-authored 25 technical papers in IEEE EPEP, IEEE ECTC
and DesignCon conferences.
Jeff Cutcher
Jeff Cutcher joined Ansoft Corporation as an Applications Engineer in 2005 with a focus
on supporting their Designer, Nexxim, and SIwave products. Prior to joining
Ansoft, Jeff held the position of Senior Staff Electrical Engineer at Motorola, where he
was involved in various wireless communications development projects that included
digital, analog, and RF circuit design, to pcb layout, and system modeling and simulation.
Jeff is currently a Senior Member of the IEEE, a registered Professional Engineer in the
State of Florida, and holds his BSEE and MSEE from New Jersey Institute of
Technology.
Tony Donisi
Tony Donisi has been a Senior Applications Engineer for Ansoft since 2000. During his
time with Ansoft, he has developed a broad range of applications and designs in many
different areas, such as RF & microwave circuits, wireless communication systems, and
SI related products. Prior to Ansoft, he worked as a RF design engineer for over 17

years. He worked at companies including Raytheon, M/A-Com, and Alpha industries.
He was primarily involved with military and commercial receiver and transmitter module
design, including MIC’s and MMIC’s. Design and analysis work included broadband
and low noise MMIC & MIC amplifiers, components and subsystems; image reject and
sub-harmonic mixers; frequency conversion modules; phase locked loops; monopulse
radar systems and microwave and millimeter wave T/R modules. He earned his BSEE
from Northeastern University, Boston, MA and his MSEE from UMASS Amherst.
Michael Tsuk
Michael Tsuk received the BSc, MSc, and Ph.D. degrees in Electrical Engineering from
MIT in 1984, 1986, and 1990, respectively. In 1990, he joined Digital Equipment
Corporation, where his primary responsibilities were designing and maintaining Digital’s
internal electromagnetic CAD software, and consulting on its use for signal integrity,
power integrity, and EMI modeling. He was promoted to Consulting Engineer at Digital,
became a Member of the Technical Staff after Digital’s merger with Compaq in 1998,
and an Expert after Compaq’s merger with Hewlett-Packard in 2002, after each merger
expanding the user base of his software throughout the larger corporations.
In 2003, he joined Ansoft Corporation, becoming part of the Nexxim team,
where his primary responsibilities concern models for transient simulation of lossy
coupled transmission lines and general frequency-dependent structures described by
scattering parameter matrices.
He is the holder of four patents and is the author of numerous technical papers.
Steve Pytel
Steve Pytel joined Ansoft Corporation as an Applications Engineer in 2007 focusing in
high speed interconnects; he supports tools such as Ansoft Designer, Nexxim,
SIwave, HFSS, and Q3D. Prior to joining Ansoft, Steve held the position of a
Senior Hardware Design and Signal Integrity Engineer at Intel Corporation where he was
responsible for the signal integrity of Telco, Enterprise, and ATCA Blade form factor
servers. He also focused on high speed electrical interconnect modeling research where
he and a team developed frequency dependent dielectric transmission line models that
incorporated various forms of copper surface roughness loss. Steve received a Bachelors
of Science in Electrical Engineering from Northern Illinois University, a Masters of
Engineering, and a Doctor of Philosophy in Electrical Engineering from the University of
South Carolina.

Introduction
High-speed channel designs, such as 10 Gb/s Ethernet, PCIe 3.0, 8.5 Gb/s Fiber channel,
demand accurate channel simulation and modeling to ensure first-pass system success in
a BladeServer environment. The backplane product in a BladeServer is defined here as
the flexible interface to attach shared devices such as processor blades, 10 Gb/s Ethernet
switches, management modules, etc… Modeling was used, primarily, to guide decisions
on the design trade-offs which promote system-level compatibility between shared
devices. The simulation strategy was developed to balance implied business goals (cost,
time-to-market) against the requirements of product performance. This paper discusses
the implementation of simulation to cope with AC loss (the predominant concern), with
inter-symbol interference (ISI) effects due to impedance geometry changes such as via
stubs, and signal jitter. At high data rates, layout discontinuities coupled with long delays
bring on severe ISI that may be mitigated by equalization or topology manipulation.
Statistical methods have been used to rapidly generate two well known channel
performance metrics: eye diagrams and bit error rate (BER) curves. Simulation time
becomes a critical factor because the real-time simulation of standard pseudo-random bit
sequence (PRBS) patterns such as PRBS23 or PRBS31 is impractical for a designer who
is trying to detect BER rates lower than 10 -12 using transient simulators. These statistical
methods provide a useful approximation of channel performance with the benefit of very
fast simulation times.
The BladeServer 10 Gb/s channel design case study incorporates statistical and transient
analyses within a schematic entry simulator. To achieve the greatest channel accuracy,
three-dimensional models of vias and connectors were incorporated along with onedimensional
distributed frequency dependent dielectric transmission line models. The
statistical channel analyses include a combination of Touchstone S-parameter files,
buffers with de-emphasis, w-elements, passive components, feed forward and decision
feedback equalization.
Typical high speed channels include differential traces meandering along some given
length, usually many wavelengths relative to the data rate. The aforementioned statistical
methods offer a quick means of analyzing these high speed channels for impairments
caused by discontinuities. This paper introduces two such methods, one that specifically
analyzes the channel, independent of bit pattern, while the other utilizes a known bit
pattern such as a standard PRBS pattern in order to produce an eye diagram and bit error
rate (BER) plots. The latter analysis produces a BER curve that is as accurate as the
number of bits simulated; however, the statistical nature of the modeling allows for a
quick result unattainable by traditional transient simulation engines. Both analyses will be
explained in detail including the inherent assumptions and modeling details that provide a
fast and accurate simulation result.
Finally, equalization will play a major role in the design of next generation high speed
channels beyond 5 Gb/s. In this paper different equalization schemes will be discussed
including feed-forward equalizers (FFE), continuous time linear equalizers (CTLE), and
decision feedback equalizers (DFE). A multi-tapped FFE combined with a multi-tap DFE

eceiver has been implemented to show corresponding statistical eye information at the
pad of the receiver. Putting it all together; equalization, accurate physical models,
statistical models and methods provide an integrated suite of tools that were utilized in
BladeServer 10Gb/s high-speed channel designs.
The BladeServer Channel
The IBM eServer BladeCenter backplane provides the electrical interconnections among
all of the chassis components, including the processor blades, switch modules,
management modules, media devices, power modules, and blower modules. It also serves
as the redundant power distribution medium from the power supplies to all of these
components. The backplane provides up to 14 blade slots, four switch slots, one media
bay, two management modules, and four power modules, all of which support hot-swap
capability. Interconnections are redundant in order to provide high availability and
maximize uptime for the customer. To provide a multiblade system that supports multiple
protocols, such as Gigabit Ethernet, Fibre Channel, and InfiniBand, over a common
copper-trace medium, a serialized/de-serialized (SerDes) interface was used for this highspeed
internal input/output (I/O) fabric. High-speed designs often require complex, highspeed
printed circuit boards that can easily drive costs higher, usually through the use of
overly stringent design methodologies or expensive printed circuit board materials that
are not required for the system operating parameters. To achieve a cost-competitive
backplane for the BladeCenter system, an extensive pre-design modeling and simulation
analysis was performed to determine the appropriate material and practices to be used in
the design of the backplane. This paper discusses the implementation of solutions to
overcome some of the challenges of 10 Gb/s data transmission over multiple printed
circuit boards and connectors in order to reduce board costs. A benefit of a multi-server
chassis is that it provides the capability to share devices and hence lower the per-server
cost. In the BladeCenter system, all 14 blades share a keyboard and mouse via the
management module, and a Universal Serial Bus (USB) CD-ROM/DVD (compact disc,
read-only memory/digital video disc) drive and USB floppy disk drive (FDD) located in
the front of the chassis. This paper describes the initial pre-design analysis that was
performed to establish the backplane design requirements for both material and design
practices for the high-speed SerDes wiring.
This section describes the electrical design challenges that were associated with the highspeed
SerDes interfaces and were encountered during the definition, design, and
verification of the BladeCenter system. SerDes is a full-duplex, high-speed serial bus
comprising of a single transmit and a single receive differential pair of signals. The
BladeCenter design uses SerDes channels for interconnecting processor blades and highspeed
switches that support Gigabit Ethernet, Fibre Channel, InfiniBand, and Myrinet
I/O fabrics. A comprehensive electrical design methodology, including accurate and
detailed modeling and simulation of the complete design space, was required in order to
achieve the speeds required by these standard interfaces while at the same time using
low-cost printed circuit board material. Some of the obstacles and solutions to supporting
10 Gb/s data transmission over a copper backplane, multiple boards, and multiple
connector technologies are highlighted. The pre-design analysis was performed to predict
interconnect performance for relatively long trace lengths (across three boards and two

connectors) in FR4 material in order to satisfy the high-speed design requirements and
ensure the overall performance objective for this system. The pre-design analysis was
used to create board design guidelines that would prevent signal quality issues and
enhance signal eye opening. The signal and power distribution integrity effects—
including frequency-dependent losses, inter-symbol interference (ISI), crosstalk,
impedance discontinuities, and skew were integrally analyzed across the complete design
space. The signal loss was determined to be the prominent issue. To develop and design
guidelines in a timely fashion before the first hardware prototype was built; the pre-route
simulation capability in Ansoft HFSS, Ansoft Q3D, Ansoft Designer with
Nexxim was used to explore various topology configurations and to determine the
limiting factors for total link loss. Given the system mechanical and electrical boundary
conditions for the high-speed serial link, the topology shown in Figure 1 depicts the
BladeCenter end-to-end SerDes trace design, in which the differential pair traverses three
boards and two connectors with data rates varying from 1 to 10 Gb/s. Simulations were
performed to cover all permutations of design cases to determine the worst-case scenario.
Behavioral device models supplied by external component suppliers and DFE/FFE
equalization techniques were used in fine-tuning the solution space. A variety of tools
and techniques were used to generate a channel model as part of the BladeCenter
backplane pre-design simulation and analysis. Channel models included the backplane
Molex VHDM connectors [1], board traces and vias for the I/O expansion adapter, blade,
backplane, and switch module [2].
The BladeCenter electrical topology, as shown in Figure 1, provides electrical
interconnections between all of the components including the processor blades, switch
modules, management modules and media device. The backplane accepts up to 14 blade
slots, 4 switch slots, 1 media bay, 2 management modules and 4 power modules, all of
which support hot-swap capability. The IBM BladeServer Backplane hardware contains
electrically optimized reference channels constructed using an FR4 loss dielectric with a
½ ounce differential stripline construction using 7 mil trace and 100 ohm impedance
construction. The BladeServer Backplane was chosen to support physical layer testing of
this configuration. The Reference Backplane consists of a backplane test, demonstration
vehicle and a set of line cards to support the operation of up to four adjacent active
channels. Each passive channel contains two VHDM TM connectors, two line cards with
layout optimized connector interface conditions and a passive backplane channel
indicated in Figure 1.

Figure 1: Channel model topology
Each daughter card includes 2.5 to 6 inches of differential signal traces. The remainder of
the channel length is contained within the backplane. The BladeServer Backplane is 24
layers, 225 mils thick, and supports channel lengths from 8 to 32 inches. To minimize
crosstalk, a channel-to-channel separation of 40 mils was determined to provide adequate
isolation throughout the backplane routing. The channel margin hardware contains
selectively degraded-performance channels. These degraded performance channels can
be exchanged with the standard optimized daughter cards in any combination to achieve
discrete parametric control or to achieve the composition of two or more sources of
parametric degradation. Return loss performance can be degraded by a variety of channel
and packaging features such as PCB via stubs formed to accommodate press-fit connector
pins, AC coupling features or PCB layer transition [3].
BladeCenter Packaging
A BladeCenter is a rack-optimized server architecture designed to provide the
consolidation benefits of 1U and 2U rack systems while eliminating the complications
associated with these systems. This is explained in this report by summarizing
BladeCenter package with respect to a 1U rack system package. BladeCenter systems
take less space and put the enterprise server infrastructure within easy reach of the
administrator. BladeCenter provides modular scalability, versatility in server types,
reliability, availability, and serviceability, system management enhancements, thermal
advantage and list of potential cost savings.

The current enterprise infrastructure in the industry is built by 1U and 2U system racks.
Each 1U or 2U server requires its own power cables, Ethernet or Fiber Channel switches,
systems management, power distribution units (PDUs), and keyboard/video/mouse
(KVM) switches. A rack of 42 1U servers can have hundreds of cables strung throughout
the rack, making it difficult to determine where cables are attached and increasing the
complexity of adding or removing servers from the rack. The typical 1U and 2U rack
need to be connected externally to ensure the entire rack is functional.
The BladeCenter unit is a high-density, high-performance rack-mounted server system
developed for medium-to-large businesses. BladeCenter provides up to 14 functionally
separate servers and their shared resources in a single center. The term BladeCenter
package is a chassis that can hold 14 hot swappable devices called blades, which is an
independent server containing one or more processors, memory, disk storage, and
network controllers, 4 hot swappable switches, 2 management modules, 4 power supplies
and 2 blowers. As shown in Figure 2, BladeCenter systems take less space and provide
higher density.
Figure 2: BladeCenter system packaging.
Simulation Methodologies
The following sections describe the methods and results associated with simulating the
high speed SerDes channels. Numerous challenges must be overcome to design a system
that can reliably transmit and receive 2.5 Gb/s serial data over a backplane without errors.
The crucial first step in developing the set of design ground rules to be used in the
development of the BladeCenter backplane was to understand high-speed design criteria
and the effects design materials and methodologies would have. An initial investigation

of high-speed design characteristics yielded four primary areas that had to be taken into
consideration: printed circuit board losses, impedance discontinuities, crosstalk, and skew
due to board routing and connectors. In addition, to validate some of the simulation
results, a number of tests were performed using a standard VHDM backplane evaluation
board to guide decisions for designing and building a backplane capable of supporting a
bit rate of 10 Gb/s. Simulations that used a behavioral channel model were employed to
process all design permutations. The goal was to determine the worst-case scenario and
then optimize the solution space. The mechanical and electrical constraints for the highspeed
serial link, shown in the topology of Figure 1, represent the typical blade design, in
which the signal traverses three boards and two connectors with data rates that vary from
1 to 10 Gb/s.
Accurate modeling of high speed serial channels from the driver to the receiver requires
numerous building blocks that accurately interoperate. Tackling these design issues
would be an impossible task without the use of proper tools such as electromagnetic
models within a circuit simulation environment that can demonstrate silicon performance
including equalization. No longer are purely analytic methods an acceptable solution to
today’s high speed serial design challenges. In addition to pre-layout system simulation
analysis, another key to ensuring proper system operation is post-layout extraction that
quickly analyzes the actual architecture that will be sent off to the fabrication house.
This work will concentrate on a single IEEE 802.3ap KR BladeServer channel that will
include analyses from the driver to the receiver. This BladeServer channel spans over 26
inches of FR-4 and includes three multi-layered circuit boards consisting of a switch card,
backplane, blade card, two high speed connectors, and standard through hole vias (Figure
1). The pre-layout consisted of full wave three-dimensional electromagnetic models
created from Ansoft’s HFSS for the differential vias used within the BladeServer
channel, SMP connectors used for channel measurements, and the VHDM high speed
connectors. The differential traces for the switch, mid-plane, and blade were modeled
using the two-dimensional electrostatic solver within Ansoft’s Q3D. The striplines
were modeled as differential cross-sections that included frequency dependent dielectric
responses of the FR-4 within the PWBs. Ansoft’s Q2D produces a Spice syntax netlist
containing a tabular RLGC w-element file that is used within Ansoft’s Designer with
Nexxim.
This methodology was used to perform pre-layout simulations by defining the channel
within the Designer simulation environment and then performing time-domain and
statistical domain simulations using the Nexxim simulation engine. Initially, a
behaviorally based circuit model with de-emphasis was used to represent the driver side
of the channel. This buffer model consisted of five taps of continuous time linear
equalization.

Figure 3: Blade server simulated channel.
Post-layout modeling was achieved by extracting the Allegro CAD layout for each of the
three PWBs with Ansoft Links and then importing them into Ansoft’s SIwave.
SIwave was then setup to provide a channel model from DC to 20GHz utilizing the 24
layer stack-up for each board. The full-wave, package field solver models each PWB
using a two-dimensional Finite Element Method for all extracted plane geometries and
cavities. This is combined with a Method of Moments solution for each signal trace that
results in a very accurate touchstone or Full Wave Spice file that is used to determine the
system response of the BladeServer. To ensure passive and causal transmission line
responses, the full-wave, package field solver uses a Djordjevic [4] frequency-dependent
model for the dielectrics contained within the PWB.
Finally, time-domain simulations were run for all of the cases so that a prediction of the
system performance at 10 Gb/s could be obtained with and without equalization.
Statistical analysis methods were employed to assist in determining the sensitivity to
transmitter random jitter (TXRJ) and duty-cycle distortion (DCD). Feed forward
equalization and decision feedback equalization algorithms inherent to the circuit
simulator tool were utilized to determine tap weights and eye openings at the pad of the
silicon receiver.
Statistical Analysis – VerifEye
VerifEye is a new, statistical tool for generating contour eye diagrams and bathtub curves
for a channel. By combining the statistics of the bit stream with the variation in

transitions due to jitter, it is possible to generate the information needed in much less
computation time.
First, consider a particular point on a given waveform passed through a channel. That
point is the sum of all the voltage excursions caused by the various transitions from high
to low or low to high at bit interfaces that have occurred prior to the time point in
question.
Rather than waiting for the voltage excursions for all possible combinations of bits and
then looking to see which combination caused an error, the process is shortened through
the calculation of a probability distribution for the sum of all the excursions.
Perhaps it is easiest to consider this concept in terms of what are known as “cursors”; the
deviations from the ideal waveform.
Figure 4: The two largest cursors from the step response of the RC example.
If the channel were perfect, then the response would be identical to the input; the voltage
at each point in time would be either +1V or -1V, depending on the bit in question.
Referring to Figure 4, imagine what the waveform would look like if there were a very
large number of 1s in a row, versus a large number of 0s, followed by a transition to a 1.
In the first case, the waveform would have time to settle to its correct final value of +1V.
In the second case, the waveform would be in transition from -1V to +1V because of

imperfections in the channel. The difference between those two waveforms is a “cursor”,
as shown in Figure 4.
Now, let us explore how we might calculate the inter-symbol interference (ISI)
statistically. Let us assume that the current bit is a 1. This can happen in two different
ways: either the previous bit was a 1, in which case there was no transition between them,
or the previous bit was a 0, in which case there was a low-to-high transition between
them. The probability of a transition is 0.5, if the bits are independent. In the first case,
the cursor is zero; in the second case, it has some significant negative value,
corresponding to the slow rise time of the step response at the end of the channel.
Statistically, therefore, the waveform has a 50% chance of having no deviation, and a
50% chance of a negative deviation, equal to the cursor value.
It is clear that we can chain this process forwards and backwards, considering each
“postcursor” and precursor in turn. The primary difficulty is that, even if the bits are
independent, the transitions are not; a high-to-low transition cannot be followed by
another high-to-low transition; there must be a low-to-high transition in between.
However, this constraint can be handled by the appropriate application of conditional
probabilities.
What comes out of all of the above considerations are probability distribution functions
(PDFs) for the voltage value of the waveforms of high and low bits. These functions can
be combined into cumulative distribution functions (CDFs), contours of which give the
traditional eye diagram.
One of the other measures of the signal integrity of a channel is the so-called “bathtub
curve”. This is a measure of the width of an eye opening at a certain level, usually
halfway between the high and low levels. The bathtub curve turns out to be particularly
easy to generate using statistical methods: it is simply a slice through the CDF
determined above.
Figure 5: Bathtub curve for RC example, with 10ps random transmit jitter. X axis is time in
seconds; Y axis is common logarithm of BER.

One of the great advantages of the statistical technique is that we can easily include the
effects of random transmit jitter (standard deviation and not p-p), without resorting to
Monte Carlo methods. This is done by applying the Gaussian probability distribution
function to the time-location of each cursor, so that the cursors are not just impulses in
voltage, but are spread out due to the Gaussian distribution in time, as shown in Figure 6:
Figure 6: How random transmit jitter smears out the PDFs for the voltage cursors.
One of the advantages of VerifEye over the public-domain tool called StatEye [5] is that
VerifEye uses edges as its basis for calculations, rather than the pulses of StatEye. In
part, this approach is made possible due to the high-quality transient responses that
Nexxim provides for S-parameter-based circuits.
The advantages of edge-based, rather than pulse-based, statistical eye calculations are
primarily in two areas. First, by separating the rising and falling edges of the pulse, we
can ensure that random jitter is indeed truly random: the timing of the edges can be
statistically independent. And also, we can include the effects of duty cycle distortion
(DCD), which can cause the width of pulse to vary.
Pattern-based Analysis – QuickEye
QuickEye uses some simplifying assumptions to create an eye diagram from transient
simulation of a known sequence of single transitions.

A fundamental assumption of both QuickEye and VerifEye is that the drivers and channel
are linear time-invariant systems. QuickEye and VerifEye are based on the principle of
superposition, which makes use of this assumption [6].
The step response is the waveform at the far end of the channel to a single transition at
the input, from low to high or high to low. To obtain the response of the overall channel,
individual step responses for rising and falling edges are superposed. The result is the
channel response shown in Figure 8.
Figure 7: Intermediate QuickEye waveforms for the RC example. Rising edges are shown as solid
curves; falling edges as dashed. When all these transitions are added together, we get the waveform
in Figure 8.
While it is relatively easy to calculate the effects of ISI with the QuickEye algorithm,
random transmit jitter is much more difficult. The most accurate method is to do a
Monte-Carlo method: each transition is displaced a random amount from the nominal
time, depending on the probability distribution of the random jitter, but this greatly
increases simulation time, as there needs to be at least several different jittered transitions
for each bit in the original stream.

Figure 8: The response of a given channel.
Equalization Requirements
Several equalization techniques exist to help combat inter-symbol interference (ISI)
caused by the channel. More often, long channels at high data rates in the Gb/s are
beginning require feed-forward equalization (FFE) followed by a non-linear decision
feedback equalizer (DFE). Typical equalization architectures can be seen in Figure 9.
These examples were directly implemented in the simulation tools. The FFE assists in
sharpening up the edges (pre-cursor) while the DFE utilizes decisions to assist in
removing the interference from previous bits (post-cursor). The computation of the tap
weights is indeed an important step, however, an equally important step is the
determination of the minimum number of taps required to provide an open eye that yields
acceptable BERs. Equalization requirements will be shown for both pre-layout and postlayout
as well as for the measured channel.

Discrete
Time
1.5
1
0.5
0
-0.5
-1
Input
Output (Equalized)
-1.5
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Output
Waveform
Feedback
Taps
K
T T
ff1 ff2 ff3 ffn
fbm fb2
fb1
T
SUM
T
T
T
Decision
Forward
Taps
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
20 40 60 80 100 120 140 160
Output
Bits
Figure 9: FFE/DFE Architecture implemented within QuickEye and VerifEye.
The new statistical simulation tools allow the user to determine the number of taps
required for equalizing the channel based on the architecture shown above. The tap
weights may be entered manually or calculated by the software. The input of the
equalizer quantizes the waveform in UI-spaced time slots and passes the voltage
amplitudes through the FFE delays and taps. Currently a zero forcing function has been
implemented to determine the optimum sampling point for the bits along with the tap
weights. At each unit interval (UI) a decision is made between a binary 1 or 0 or (1 and
-1) and that decision is fed back through the feedback delays and taps. The current output
of the equalizer is the sum of both the forward and reverse weighted amplitudes stored in
the UI-spaced taps.
Figure 10: Graphical user interface used for setting up equalization within QuickEye and VerifEye.
Pre-Layout Analysis
Prior to the task of designing and laying out high-speed channels, simulation analysis is
required and performed to determine an acceptable system level performance which is
typically defined by many standards to be a BER = 1x10 -12 at a minimum. Simulation

analysis enables rapid design changes and evaluation without the expense of actual
hardware fabrication resulting in increased profit margin.
The differential strip-line traces were model using a two-dimensional electrostatic
simulation tool with the BladeServer stack-ups being predefined for the switch,
backplane, and blade PWBs. The differential characteristic impedance was defined to be
100 ohms. The electrostatic solver was setup to sweep each stripline cross-section from
DC to 20GHz and a distributed RLGC tabular model was generated for each board. The
2D electrostatic field solver incorporates a Djordjevic model to capture passive, causal
transmission line responses The Djordjevic dielectric model uses a seed parameter
specified at a specific frequency point for loss tangent and permittivity. It then
extrapolates the real and imaginary parts of permittivity for the specified frequency range
by using a multi-pole Debye relaxation fit that guarantees causality by enforcement of the
Hilbert transform on the real and imaginary parts of permittivity. The electrostatic field
solver outputs a SPICE syntax tabular w-element circuit model that allows the user to
specify the length of the transmission line. A typical cross-section of the stripline
transmission lines is shown in Figure 11.
Very High Density Metric (VHDM) connectors were selected for the system design,
however an exact model of the VHDM connector was unavailable so a similar high
speed connector was modeled and simulated using a full wave 3D electromagnetic field
solver to generate the connector model from DC to 20GHz. New technologies within the
solver, including an improved port solution algorithm for driven terminal modes
combined with a refined adaptive, finite element method mesh solver, enable solutions to
be extracted to DC very accurately. Finally, the vias and SMP connectors were included
in the full-wave, 3D simulations.
Figure 11: Typical stripline cross-section model from the 2D electrostatic field simulator.

Figure 12: Differential via model.
With all of the building block models completed the Ansoft Designer environment was
used to create a system level schematic of the entire BladeServer system including the
PWBs, connectors, vias, drivers, and receivers.. The final pre-layout schematic is shown
in Figure 13.
0 1 2 3 4 5 6 7 0
ref
0
Port1
switch
smp_p HFSS sw_p
1
sw_p HFSS conn_p HFSS
+ +
conn_p HFSS mid_p
1
mid_p HFSS conn_p HFSS
+ +
conn_p HFSS blade_p
1
blade_p HFSS smp_p
blade
Port2
Port3
switch
smp_n
Switch
ref
sw_n
2
sw_n
Switch
ref
conn_n
- -
VHDM
ref
conn_n
mid_n
Mid-Plane
ref
2
mid_n
conn_n
Mid-Plane
ref
- -
VHDM
ref
conn_n
Blade
ref
blade_n
2
blade_n
Blade
ref
smp_n
blade
Port4
ref
0
W1239
0 0 0 0 0
W1238
0 0 0
0
0
W1237
0 0 0
ref
0
0
0 = SMP Connectors (HFSS)
1 = Driver-Side Via Stub (Full Stackup HFSS Model)
2 = Switch PCB Model (W-Elements from Q3D/2D)
3 = Connector Model (HFSS)
4 = Mid-plane PCB Model (W-Elements from Q3D/2D)
5 = Connector Model (HFSS)
6 = Blade PCB Model (W-Elements from Q3D/2D)
7 = Receiver-Side Via Stub (Full Stackup HFSS Model)
Figure 13: Final pre-layout channel schematic.
Eye diagrams and BER contours were obtained using w-element models to represent the
switch, backplane, and blade transmission lines while the connectors and vias used threedimensional
models imported from the full wave 3D EM (electromagnetic) solver. The
initial system level model reflects the pictorial diagram shown in Figure 3. As expected,
the entire system model produced a closed eye without any equalization being used;
initially the SMP connectors were not modeled and idealized connectors were used
between the backplane, switch, and processor blade to obtain a quick solution based upon
the design timeline. The initial pre-layout simulation results are shown in Figure 14. The
initial simulation required 5 taps of FFE at the receiver to obtain an open eye with
ref
0

acceptable BER. The native equalization algorithms embedded in the statistical tools
were used to equalize and open the eye , achieving a BER of 1x10 -14 (Figure 14).
A
C
B
D
Figure 14: A- Initial pre-layout simulation without any equalization using QuickEye analysis. B-
QuickEye analysis using 5 taps of FFE at the receiver. C- Contour BER eye plot from VerifEye
analysis using 5 taps of FFE at the receiver. D- Bathtub BER plot from VerifEye analysis with 5 taps
of FFE at the receiver.
The contour eye BER plot shown in Figure 14 C yields BER as a function of eye
magnitude and eye width. The color gradient shows different BER values with the blue
eye corresponding to a BER of 1x10 -14 and the red corresponding to a BER of 5x10 -2 . A
VerifEye analysis of transmit Gaussian random jitter (RJ) was performed to determine
what operating margin was left in the system. Transmit RJ of 3 ps (Gaussian average
value) was incorporated into VerifEye. Using a six sigma standard to include 99.73% of
the jitter the peak RJ is ~ 9 ps with a delta of ~18 ps.

Figure 15: VerifEye contour BER diagram with 3 ps of RJ introduced into the system with 5 taps of
FFE at the receiver.
VerifEye BER results for the final pre-layout channel (Figure 13) that included the SMP
connector and an additional four 50 mil via stubs are shown in Figure 16. This shows that
additional equalization was required to achieve an acceptable system BER; this required
the inclusion of DFE into the receiver along with FFE. The final design showed 8 taps of
FFE and 3 taps of DFE at the receiver were required to equalize the system channel.
A B
Figure 16: A- VerifEye contour BER plot for Figure 13 yielding BERs up to 1x10
bathtub
-12 . B- VerifEye
BER curve shown up to a BER of 1x10 -12 . Both A and B plots are using 8 taps of FFE and 3
taps of DFE at the receiver.
Post-Layout Analysis
The Allegro CAD design was translated into an Ansoft Neutral File format (.anf) using
Ansoft links. The translated PWB layout was then imported into Ansoft’s SIwave fullwave
package and PWB extractor tool. The package and PWB extractor tool was used to
create a Touchstone S-parameter file that included the exact layout of each lane analyzed.
A differential pair for the 10 Gb/s (5 GHz) lane is shown in Figure 17 for the Blade and
switch cards.

A B
Figure 17: A: SIwave isometric view of the extracted differential pair on the Blade card. B:
SIwave top-down view of the Switch card.
The post-layout model is shown in Figure 18 and consists of the three SIwave models,
SMP connector models, and a high speed connector model. A Touchstone file was
created for each PWB in the system (Blade, Switch, and Backplane); these were used to
model the channel in lieu of w-elements and discreet vias. The full-wave package and
PWB extraction tool performs a 2D FEM solution on all planar objects within the PWB
and combines this with a Method of Moments solution for the signal traces. The final
solution incorporates the fabricated board including the stack-up, plane geometries, trace
geometries, components, vias, via padstacks, and frequency dependent dielectrics for
each layer. Upon completion of the simulation, a Touchstone file was exported to the
statistical tools for rapid,statistical analysis.
0 1 2 3 4 5 0
ref
0
Port1 Port2
SWITCH
HFSS
Mid-Plane
HFSS
Blade
switch
switch
Port3 Port4
ref
ref
ref
0
smp+
smp- conn-
ref
conn+
C
sw+
blade+
sw- blade-
ref
0 0 0 0 0
1 = Switch PCB Model (SIwave)
2 = Connector Model (HFSS)
3 = Mid-plane PCB Model (SIwave)
4 = Connector Model (HFSS)
5 = Blade PCB Model (SIwave)
Figure 18: Post-layout circuit model.
The post-layout simulation results show a very similar solution space to that of the final
pre-layout simulation (2 ps difference at a BER of 1x10 -12 ). VerifEye BER results are
shown in Figure 19 for the post-layout simulation.
C
conn+
smp+
conn- smp-
ref
blade
blade
ref
ref
0
0

Figure 19: Post-layout results for the extracted Switch-Backplane-Blade channel extracted from
SIwave
using VerifEye analysis with 8 taps of FFE and 3 taps of DFE at the receiver.
Measured Analysis
To determine if the simulated solution space was representative of the actual design an
end-to-end frequency domain channel measurement was made and the measured
Touchstone file was used to determine the channel BER using the statistical simulation
tool. The measurement methodology used Gore 3.5 mm (EN0CB0CA036) cables along
with a 3.5 mm to SMP adapter cable (Rosenberger 101076-4604) that allowed mating to
the female SMP connectors on the Switch and Blade. A SOLT calibration was performed
on an Agilent E8361A PNA with an N4421BH67 test switch up to 20 GHz to the end of
the Gore cables and therefore did not calibrate out the 3.5 mm to SMP adapter cable. This
adapter cable has not been taken into consideration in any of the simulations. The
measurement was taken from 10 MHz to 10 GHz with a frequency step of 10 MHz. The
authors realize this measurement methodology will not provide the most accurate data.
However, given the physical size and constraints of the entire system which included:
three PWBs along (Switch, Backplane, and Blade), a right angle connector, and a left
angle connector the measurement setup was considered sufficient. The awkwardness of
the physical setup precluded the use of RF probes with which extremely accurate
measurements could have been made. Due to limitations in the measurement setup the
authors limited the upper frequency of the measurement to 10 GHz and believe error
introduction started to become a factor beyond 6 GHz.
The measured .s4p file was imported into the circuit simulator tool and then simulated
using VerifEye with 8 taps of FFE and 3 taps of DFE at the receiver. Comparing the pre-
and post-layout simulations to the measured channel .s4p simulation good correlation was
obtained. For a BER of 1x10 -12 the measured s-parameters yielded an eye opening of 43
ps where the pre- and post-layout simulations yielded eye opening of 49 ps and 51ps
respectively. The authors believe even better correlation can be obtained by incorporating
the actual VHDM connector model and obtaining better measurement results up to 15
GHz for the simulation. However, despite these anomalies good correlation (8 ps for a
BER of 1x10 -12 ) was achieved. These results have been used in defining an operating
BladeServer KR channel (5 GHz or 10 Gb/s) which includes over 26 inches of FR4 with
very large via stubs (up to 125 mils) within the channel and an additional four 50 mil via
stubs to support the measurement methodology.

Figure 20: BladeServer frequency domain test setup.
Figure 21: VerifEye simulation using the VNA .s4p measurement file.
Conclusion
This paper has demonstrated the ability to accurately predict system level performance
for an extremely harsh 10 Gb/s Ethernet architecture that included 3 PWBs, 2 connectors,
more than 26 inches of standard FR4 (tan δ ~.021), 125 mil via stubs using pre- and postlayout
simulation methodologies in both the time and statistical domains. The results
show very good correlation between measured and simulated data. It has been shown that
pre- and post-layout simulations correlate to within 8 ps of the measured channel at a
BER of 1x10-12. In order to increase the correlation, models of the transitions and
connectors should be included. Inclusion of these models is critical to increase accuracy
of the simulation above the achieved 8%. This will be addressed in future simulations.

The paper describes a BladeCenter along with the advantages it possesses over traditional
rack mount server arrangements. The paper provides an in depth analysis of the elements
required to design and develop 10 Gb/s channels used in such a system.
Finally, the paper introduces two new statistical tools, VerifEye and QuickEye, that
address the industry-wide challenge of impossibly long transient measurement times
implied by current BER requirements and traditional methods. These statistical tools
complement an advanced suite of circuit simulation tools which are used to model the
system performance of a detailed BladeServer channel. An overview of the modeling
algorithms for QuickEye and VerifEye is discussed along with the implementation of
FFE and DFE within each analysis technique.
Acknowledgements
The authors would like to extend thanks to Wahid Ahmed, Michael Tsuk, Matt
Commens, Rich Hall, François Pradeau, Rob Holoboff, Larry Williams, and Bill Kofoed
for their invaluable help in providing support and technical guidance.
References
[1] VHDM Backplane Connector System; see
http://www.molex.com/cmc_upload/0/000/0–8/388/tab01vhdm.pdf
[2] P. Patel, B. Herrman, J. Hughes, J. Wong, and M. Cobo, ‘‘Gigabit Ethernet Allegro
PCB SI Simulation Matching Lab Measurements’’,
http://www.cadence.com/community/allegro/Resources/resources_pcbsi/si/TPIBM_gigab
itsimulation.pdf
[3] Plateau HS Mezz Connector System; see
http://www.molex.com/cgibin/bv/molex/index_login.jsp.
[4] A.R. Djordjevic, R.M. Biljic, V.D. Likar-Smiljanic, T.K. Sarkar, “Wideband
Frequency-Domain Characterization of FR-4 and Time-Domain Causality,” IEEE Trans.
on Electromagnetic Compatibility, V. 43, n4, November 2001, pp 662 – 667.
[5] Anthony Sanders, Mike Resso, John D.Ambrosia, Channel Compliance Testing
Utilizing Novel Statistical Eye Methodology, DesignCon 2004.
[6] R.I. Mellitz, M. Tsuk, T. Donisi, S.G. Pytel, “Strategies for Coping with Non-linear
and Time Variant Behavior for High Speed Serial Buffer Modeling”, DesignCon 2008,
February 4 – 8.