PCI Express

PCI Express PCI Express

from plxtech.com More from this publisher

13.02.2014 Views

PLX Overview PCI Express & Storage - Two Fast Growing Markets - Now over 50% of Company’s Revenue PCI Express Switches & Bridges Storage Controllers Connectivity USB & PCI Bridges, Controllers & UARTs PLX Revenue Split by Product Line Public NASDAQ Company (PLXT) Financially Solid with Zero Debt, Cash Flow Positive 2

PLX Overview

PCI Express & Storage

- Two Fast Growing Markets

- Now over 50% of Company’s Revenue

PCI Express

Switches & Bridges

Storage

Controllers

Connectivity

USB & PCI Bridges,

Controllers & UARTs

PLX Revenue Split

by Product Line

Public NASDAQ Company (PLXT)

Financially Solid with Zero Debt, Cash Flow Positive

Market Leader

#1 Supplier PCI Express Interconnect

PCI Express

Switches & Bridges

- Over 55% Marketshare*

- Designs with all Market Leaders

- 4 Million Units Shipped

- Broadest Offering of Switches & Bridges

Communications

Embedded

Server

Storage

PC/Consumer

* 55% for Bridges & Switches Combined. Over 65% for Switches.

PCIe Switch Is a Basic Building Block

SPARC CPU

with Native PCIe

x86 Processors

Network, Security,

Graphic

& Co-Processors

Processors

Chip Sets

ASICs, Logic,

& FPGAs

Communication

& Storage

PCI Express

Switch

PCI Express

Cleaner, Lower Cost, Lower Power

Switch is a basic building block

Product Summary

& Road Maps

PLX Products

No NDA Required

48

32

24

16

12

8

Lanes

96

80

64

48

32

24

16

12

8

6

4

1 st Gen 2 Family

PCIe Gen 2 Switch Road Map

Server, Storage & Dual Graphics

PEX 8648

48 Lanes, 12 Ports, NT

PEX 8632

32 Lanes, 12 Ports, NT

2KB, 3HPC, DC, RP, DB

PEX 8624

24 Lanes, 6 Ports, NT

2KB, 3HPC, DC, RP, DB

PEX 8616

16 Lanes, 4 Ports, NT

2KB, 3HPC, DC, RP, DB

PEX 8612

12 Lanes, 3 Ports, NT

2KB, 3HPC, DC, RP, DB

Multi-Host MR

& Multicast

Blade & Rack Servers,

Storage & Networking

PEX 8647

48 Lanes, 3 x16 Ports

2KB, DC, DB, RP

High Port Count, 2VC

, DMA

& SSC for Control Plane

Networking, Embedded & Storage

PEX 8617

16 Lanes, 4 Ports, NT

PEX 8618

16 Lanes, 16 Ports, NT

PEX 8613

12 Lanes, 3 Ports, NT

PEX 8614

12 Lanes, 12 Ports, NT

2VC, DC, RP, SSC, DB

PEX 8608

8 Lanes, 8 Ports, NT

2VC, DC, RP, SSC, DB

PEX 8606

6 Lanes, 6 Ports, NT

PEX 8604

4 Lanes, 4 Ports, NT

2VC, DC, RP, SSC, DB

PEX 8696

96 Lanes, 24 Ports, NT

Multi-Root/Host

PEX 8680

PEX 8619

16 Lanes, 16 Ports, NT

2VC, DC, RP, SSC, DB, DMA

PEX 8615

12 Lanes, 12 Ports, NT

2VC, DC, RP, SSC, DB, DMA

PEX 8609

8 Lanes, 8 Ports, NT

2VC, DC, RP, SSC, DB, DMA

80 Lanes, 20 Ports, NT

Multi-Root/Host

PEX 8664

64 Lanes, 16 Ports, NT

Multi-Root/Host

PEX 8649

48 Lanes, 12 Ports, NT

(MR)/Host & Multicast

Low Lane

Count

General Purpose

Shipping Now

In Development

Planned/Concept

2008 2009

Approximate Timeframes 2010

2010

PLX Exclusive Features

‣ Here are the exclusive features we will talk about today:

• visionPAK TM Suite

• Extraction of Receive Data “Eye Width”

• PCIe Packet Generator

• Performance Monitoring

• Error Injection

• SerDes Loopback Modes

• performancePAK TM Suite

• Read Pacing

• Multicast

• Dynamic Buffer Allocation

‣ All valuable in both Gen 2 and Gen 1 modes!!

visionPAK Suite

System Debug Features

Exclusive to PLX 8600 Switches

Best-in-Class Gen 2 SerDes

‣ Best-in-class SerDes (ARM)

• Can support up to 60 inches of trace length for backplanes

‣ Optional programmable features

• Transition amplitude and Non-transitional amplitude

• Pre-emphasis, De-emphasis and drive strength granularity to 50mV

• Receiver Detect and Electrical Idle bits

• SerDes BIST and AC JTAG

• Automatic impedance calibration

Non-Transitional Eye

Transitional Eye

Extraction of Receiver Eye Width

‣ What is it?

• PLX Exclusive pre-system design tool

• Used on PLX RDK to test how clean a link is

• Early indicator of potential link issues

• A very open eye means a very reliable link

• A tight eye indicates a weak link

• Supported by all PEX 86xx switches

Extraction of Receiver Eye Width

‣ How does it work?

1. User selects port, lane, Rx equalization, dwell time, etc.

2. Software finds the center of the eye

3. Software steps through the eye left & right until data errors occur

4. Determines eye width and displays on the screen

Center of the Eye

Eye Width

Steps Left

Steps Right

Extraction of Receiver Eye Width

‣ Minimum Eye Width Test

• Customer selects Port, Lanes, and Minimum Eye Width

• Software tests all selected lanes for Minimum Eye Width

• Returns “Pass” or “Fail” for each selected lane

• Extremely convenient and pain-free tool for Customers

Extraction of Receiver Eye Width

‣ Auto-Calibrate Feature

Rx = Receiver

• Customer selects Port and Lane and range of Rx Equalization

• Software steps through each eye and finds optimal setting

• Returns value of Rx Equalization that gives best eye width

• Extremely convenient and useful tool for Customers

Extraction of Receiver Eye Width

‣ Customer Benefits

• Allows customer to see how much margin their

link has at the PEX 86xx Receiver

• Convenient features

• Minimum Eye Width Test

• Auto-Calibrate Feature

• Identify potential link issues earlier

…GET TO MARKET FASTER!!!

PCIe Packet Generator

‣ What is it?

• PLX Exclusive feature that allows customers to create

their own traffic patterns using a PLX switch

• Enables high density traffic

• Up to x16 port saturation (not easy to achieve)

• Can create error messages

• See how software/system reacts to errors

• PLX RDKs can be used as PCIe packet generators

• Great alternative to expensive PCIe Exercisers

• All-in-one solution

PCIe Packet Generator

‣ User-programmable traffic

• Memory Reads/Writes

• Payload Size

• PCI Address

‣ View

command list

‣ Create

looped traffic

PCIe Packet Generator

‣ Customer Benefits

• Convenient, inexpensive way of stress testing their system

• Programmable traffic

• Ability to fully saturate their links

• Test system software

• No external equipment needed

…SAVES $$$!!!

Performance Monitor

‣ What is it?

• PLX Exclusive feature that allows customers to monitor

PEX 86xx switch performance real-time

• Displays performance for each individual port

• Displays Ingress and Egress performance

• Completely passive

• Does not impact system performance

‣ How does it work?

Performance Monitor

• Customer selects Port to monitor

• Software reads PEX 86xx registers and displays the data

• Link Utilization % indicates unused link potential

• Displays total rate, payload rate, reads vs. writes, etc.

Performance Monitor

‣ Graphically ‘see’ the traffic on each port during runtime

Total

Byte Rate

% Link

Utilization

Payload

Byte Rate

Average

Payload Size

(Bytes)

Performance Monitor

‣ Count Ingress & Egress TLPs for every port

• Extensive Granularity

• Posted header & dword

• Non-Posted dword

• Completion header & dword

• Filter for various types of Posted and Non-Posted packets

• For example, count just MWr64

‣ Count Ingress & Egress DLLPs for every port

• Extensive Granularity

• Filter for ACKs

• Filter for NAKs

• Filter for UpdateFCs (Posted, Non-Posted, or Completions)

Performance Monitor

‣ Customer Benefits

• Allows customers to track real-time link utilization

• Helps find any weak links & potential bottlenecks

• Gives additional visibility into traffic patterns

• Convenient, inexpensive system bring-up tool

…THE COMPLETE SOLUTION!!!

Error Injection

‣ What is it?

• Software development tool

• Inject ECC errors in PCIe packet

• Inject PCIe error messages

‣ Customer Benefits

• Allows a customer to see how their software would

react to these errors

…SPEEDS UP SOFTWARE DEVELOPMENT!!!

Loopback

‣ Customer Benefits

• Four convenient ways to test the SerDes and Logic of the

PEX 86xx and/or connected device

• Internal Tx

• External Tx

• Recovered CLK

• Recovered Data

…FOUR LOOPBACK MODES!!!

Internal Transmitter Loopback

‣ Used to test PLX SerDes and Logic

External Transmitter Loopback

‣ Used to test the link and logic of connected device

Recovered Clock Loopback

‣ External Tx test plus also tests PLX Recovered Clock Circuit

Recovered Data Loopback

‣ External Tx test plus also tests PLX Logic

Easy Debug via I 2 C

‣ Customers recommended to design in a I 2 C connector

• Allows easy connection to PLX RDK via laptop or PC (USB)

• Enables systems w/o Windows or Linux OS to run SDK thru

remote system (i.e. laptop)

• Convenient for debug purposes

• Ideal for on-site FAE support

USB

PLX RDK

PORT 8 – X16

PORT 12 – X16

Port 8 Status

LED

Port 12 Status

LED

Port 0 Status

LED

Manual

Reset

HD Power

Connector

I 2 C

EEPROM

JTAG

RefClk

PEX 8548

DIP Switch

for Config

I 2 C

PERST#

Port 0 – x16

(Upstream Port)

System Chassis

performancePAK Suite

Performance Enhancing Features

Exclusive to PLX 8600 Switches

PEX 8600 Performance

‣ PLX Leadership in Gen 2 Performance

‣ Achieving >99% theoretical max throughput

in Host-Centric and Peer-to-Peer environments

‣ Proven in simulations & actual measurements

Stay ahead of the pack with PLX!

‣ Featuring performancePAK

• Read Pacing

• Dual Cast & Multicast

• Dynamic Buffer Allocation

• DMA-engine inside

Throughput (MB/s)

7000

6000

5000

4000

3000

2000

1000

0

Throughput (GB/s)

10

8

6

x16 Gen 2 Host-Centric Throughput

100.0

10GE Throughput

12

80.0

60.0

100.00

40.0

80.00

20.0

60.00

0.0

40.00

64 4 128 256 512 1024 2048

Packet Size (B)

2

20.00

Theoretical Max PEX 8600 Bidirectional % Theoretical Max

% of Native

% Theoretical Max

Native

w/ PEX 8624

% of Native

0

One Thread

Multi-Thread

0.00

Read Pacing

‣ Problem Reduced endpoint performance caused by:

• Unbalanced upstream/downstream link-widths

• Uneven number of Read Requests made by endpoints

• Leads to one endpoint dominating Root Complex queue

• Other endpoints get starved

‣ Solution PLX Read Pacing*

• Read Pacing Queues manage incoming Read Requests

• Prevents one endpoint from dominating Root Complex queue

• Ensures no endpoint is starved

• Allows for optimized performance of endpoints

* Patents pending

Without Read Pacing

‣ Performance bottleneck due to mixing of slow

and fast I/Os

1. FC HBA makes multiple 2KB Read Requests

2. Root Complex queues FC HBA requests

3. Ethernet NIC makes one 1KB Read Requests

4. Root Complex queues Ethernet NIC request

5. Ethernet NIC must wait for RC to service FC HBA

requests before serving Ethernet NIC request

Root Complex

IN

2KB RR

1KB RR

x8

Switch

1KB Data

2KB Data

OUT

Ethernet NIC packets

at the end of the line

6. Ethernet NIC is starved

Reduced Ethernet NIC Performance!

Sends 2KB

Read Requests

FC

HBA

x4

2KB RR

x4

1KB RR

Sends 1KB

Read Requests

Ethernet

NIC

With PLX Read Pacing

‣ Increased performance due to fair allocation of

bandwidth to downstream ports

1. FC HBA makes multiple 2KB Read Requests

2. Switch allows one FC HBA request at a time to

pass through based on programmable thresholds

3. Ethernet NIC makes one 1KB Read Requests

4. Switch allows one requests to pass through the

switch based on programmable thresholds

5. Switch continues to allow Ethernet NIC requests to

pass through in front of large FC HBA requests

based on programmable settings

6. Ethernet NIC gets serviced more often with no

impact to FC HBA performance

7. Neither endpoint is starved

Optimized Performance!

FC

HBA

Root Complex

IN

2KB RR

1KB RR

x8

Read Pacing

Queue

Sends 2KB

x4

Read Requests

2KB RR

PEX 8600

2KB Data

1KB Data

2KB Data

1KB Data

2KB Data

OUT

Ethernet NIC packets

fairly queued

Read Pacing

Queue

Sends 1KB

x4

Read Requests

1KB RR

Ethernet

1KB NIC RR

Read Pacing Measured

Read Pacing Comparison

Throughput (MB/s)

498.355 496.315 496.125

112.15

112.24

20.87

Standalone Read Pacing Off Read Pacing On

PLX Packet

Generator

“Port Hog”

Root Complex

x8

Switch

x4

Ethernet

NIC

“Starved Port”

GE NIC Performance

PLX PCIe Packet Generator Performance

Endpoint

Read Pacing OFF Read Pacing ON

GE NIC 18.6 100.1

PLX Packet

Generator

% of Standalone Performance

99.6 99.6

‣ PLX PCIe Packet Generator

• Used to mimic a “fast” I/O

(i.e. FC HBA)

• Sending back to back Memory

Read Requests to Host

PCIe Multicast

‣ Address Based

• Multicast BAR creates a Multicast space

• Posted packets that hit in the Multicast BAR are multicast

‣ Reliability

• PCIe has low error rate and hop by hop error free transmission

‣ Supports Legacy

• Any source can send posted packet in Multicast space

• Legacy devices can be multicast targets

‣ Multicast ECN

• Specifies how – Switches, Root Complex, and End Points implement Multicast

• Improves flexibility and protection for End Points participating in Multicast

• Also allows use of legacy End Points

Multicast – Example

‣ Support for 64 Multicast (MC) Groups

‣ All ports can be programmed as MC source

‣ One source port can have multiple MC groups

‣ One destination port can be part of multiple MC groups

‣ MC can be done across an NT port

CPU

Source Port

Chipset

Destination

Ports

PEX 8680

IO

GPU

IO IO GPU GPU

CPU sending single command to multiple IOs (Group 1)

CPU sending single command to multiple GPUs (Group 2)

Multicast Memory Space

Multicast_Base

2 Multicast_Index_Position

Multicast Group 0

Memory Space

Multicast Group 1

Memory Space

Multicast Group 2

Memory Space

Multicast Group 3

Memory Space

Multicast Group n

Memory Space

Multicast and Address Routing

PCIe Standard Address Route

Multicast Address Route

‣ Request that hits a Multicast address

range is routed unchanged to Ports

that are part of the Multicast Group

derived from Request address

‣ PCIe standard address route not used

for multicast

• Including default upstream route

Peer-to-Peer & Peer Plus Host Multicast

Root

PCIe Switch

P2P

Bridge

Virtual PCI Bus

P2P

Bridge

P2P

Bridge

P2P

Bridge

P2P

Bridge

Endpoint Endpoint Endpoint Endpoint

MC in Graphics & Floating Point Acceleration

‣ Dual-headed graphics

• Each GPU paints ½ the screen

• Multi-Cast commands downstream

• E.g. vector list

• Use peer to peer to transfer bit map

from GPU2 to GPU1

‣ General Floating Point acceleration

• Some GPUs need to see same data

• Push data or commands downstream

to multiple GPUs/FPUs

x16

GPU1

CPU

Root

Complex

x16

PCIe

SWITCH

x16

GPU2

MC in Communication Systems

A 40G (GE) Line Card

• May need to split processing over 4 NPUs via MC

• Service card on AMC may need to MC packets to FPGA & RegEX

RegEx

FPGA

NPU 1

AMC

PEX 8696

RGMII

CPU

NPU

NPU 4

MC in Storage

CPU

Up to 8 Processor Boards

CPU

Chip Set

Mem

Chip Set

PEX 8624

MC Enabled Ports

PEX 8664

MR Switch

IO Drawer

PEX 8664

MR Switch

IO Drawer

Back Plane

IO Cards

PLX PEX 8600 Buffer Allocation

‣ Shared memory pool per 16 lanes

PLX Buffer Allocation

x4

‣ User assigns buffers as per port-width

• Set minimum buffers per ports

• Also creates a common pool

‣ Ports dynamically grab buffers as needed

• Grab when utilization of assigned

buffers exceeds user-assigned

thresholds (25% by default)

• Return empty buffers to the pool

x4

x2

Assigned

Buffers

Common

Buffer

Pool

Assigned

Buffers

Dynamic Allocation Appropriate Buffers

‣ Static Buffers/port

• Unused buffers

• Can not assign based on traffic load

• Can not move buffers between ports

• …LOWER PERFORMANCE

Static Buffers/port

x8

Unused buffers

Fixed 5 packet buffers

for each ports for all

port widths

‣ PLX Shared Memory Pool

• All buffers usable

• Assign based on traffic load

• Move buffers around between ports

• …HIGHER PERFORMANCE

x1

x4

x8

Shared Memory Pool

x8

Buffers assign

as needed

x1

x4

x8

Multi-Root Architecture

What is Multi-Root?

‣ A root (upstream) port connects in the direction of the CPU

‣ A multi-root device has more than one upstream port

• Provides connection for Two or more CPUs

‣ Note: Endpoint sharing between CPUs is not supported

One upstream port

(single root)

Two upstream ports

(multi-root)

CPU

Upstream port

Switch

PCIe

PCIe PCIe PCIe PCIe

Multi-Root Benefits in PEX Switch

‣ Up to Eight upstream ports

• Eight independent subsystems

Host

‣ Efficient use of interconnect

• Mix-and-match number of

endpoints to CPU

• Based on performance needs

• Up to 24-ports supported

Manager

Root

Complex

PCIe

End-Point

Root

Complex

PLX MR

Switch

PCIe End-

Point

Root

Complex

PCIe

End-Point

‣ Failover/Redundancy

• Re-assign endpoints of a failed

host

Switch

PCIe

End-Point

‣ Smaller footprint

• Lower power

PCIe

End-Point

PCIe

End-Point

Transparent and Multi-Root Modes

‣ PEX MR switches support two functional modes

• Mode 0 – Transparent Switch

• Same as today’s switches

• Mode 1 – Multi-Root Switch

• Advanced architecture with multiple upstream ports

CPU

Mode 0 Mode 1

CPU

Cygnus

PCIe

PCIe PCIe PCIe PCIe

Generic Rack Mount Server

CPU

Chip

Set

PEX 8696

I/O

Storage Server with Failover

CPU

Chip

Set

NT

PEX 8696

I/O

Servers Sharing PCIe MR Switch

Each CPU* has its own dedicated IOs isolated from other CPUs

CPU

Chip

Set

Chip

Set

Chip

Set

PEX 8696

* Up to 8 CPUs

supported

I/O

Use of Mode 2 for Fail-over

CPU

Chip

Set

Chip

Set

PEX 8664 PEX 8664

55

x4 & x8

Endpoint

Endpoint

Packet-Ahead

Two Virtual Channels Implementation

Traffic Class re-mapping

Traffic Classes and Virtual Channels

‣ Traffic Class (TC)

• Specifies priority for a given PCI Express packet

• PCI Express supports Eight TCs: TC0 – TC7

• TC7 Highest Priority

• TC for a Request/Completion pair must be the same

‣ Virtual Channel (VC)

• Buffer entity used to queue PCIe packets

• 1 VC = 1 Buffer entity

• 2 VC = 2 Buffer entities

• VC assignment for a given TC according to priority

• Low Priority Traffic shares one VC

• High Priority Traffic has its own VC

One Wire, Multiple Traffic Flows

TC6-TC4/VC1

TC7/VC2

TC3-TC0/VC0

PCI Express Wire

‣ Traffic Class determines priority for packets

‣ Packets mapped to VCs according to Priority

• Highest Priority TC7 VC2

• Lower Priority TC6-TC4 VC1

• Lowest Priority TC3-TC0 VC0

‣ One VC Same priority for ALL packets

TC – VC Mapping in PCIe Hierarchy

‣ Each Device supports different number of VCs

‣ TC/VC mapping according to device capabilities

• On a per link basis

‣ VC arbitration schemes enable QoS

‣ No ordering between VCs; independent buffers

Packet-Ahead Feature

‣ Allows the NT port to modify the original Traffic Class

(TC) of a PCIe packet

• From TC0 to TCx

• where TCx is TC1 – TC7

‣ Benefits

• Provides two separate data paths for memory traffic

• Low priority, High priority

• Enhanced QoS regardless of CPU single VC limitation

• Differentiation of traffic in single VC systems

• Available in PEX8618, PEX8614 and PEX8608

Example Without Packet-Ahead

‣ CPU supports VC0 and TC0 only

• No differentiation of traffic

CPU

‣ Endpoints and Switch support two VCs

and at least two TCs

‣ Single path to CPU

NT

PEX 8618

‣ System is limited by CPU capabilities

ASIC

Example with Packet-Ahead

‣ Same CPU Limitations

• VC0 and TC0 only

‣ Same Endpoint and Switch capabilities

• VC0 – VC1; TC0 – TC1

CPU

‣ Two paths to CPU

• Via Upstream Port and NT Port

NT

PEX 8618

‣ For packets received on NT Port

• TC is changed from TC0 to TC1

• Are mapped to High Priority VC1

ASIC

‣ Packets received on upstream port are

unaffected

Packet-Ahead Transaction Details

‣ Posted Traffic (mem writes)

• CPU generates Posted Packet with TC0

• NT port modifies packet with TC1

‣ Non-Posted (Read Requests)

• CPU generates Read Request with TC0

• NT port modifies packet with TC1

• Endpoint sinks requests and provides completion with TC1

• NT port modifies completion packet to original TC0

Direct Memory Access (DMA)

Inside PLX PCIe Switches

DMA Benefits

‣ Independent Data Mover

• Can transfer small and large blocks of data

• No CPU involvement

• Can transfer data between all switch ports

‣ Centralized DMA Engine

• Processor/chipset no longer needs to support DMA

• More selection lower cost

• Software consolidation through multiple platforms

• Software code for one DMA engine

‣ Improves system performance

• Low latency transfers while sustaining Gen 2 speeds

PCIe Switch with DMA

‣ PCIe Switch is a Multi-Function device

• Function 0 P-to-P bridge

• Transparent Switch model

• No Driver Required

• Function 1 DMA endpoint

• Type 0 Configuration Header

• Memory mapped registers

• Requires DMA Driver

• Provided by PLX

‣ Available now

• PEX8619: 16-lane/16-Port

• PEX8615: 12-lane/12-Port

• PEX8609: 8-lane/8-Port

P-P

Bridge

DMA

F1

Upstream Port

P-P

Bridge

P-P

Bridge

Downstream Ports

Virtual Bus

P-P

Bridge

P-P

Bridge

DMA Implementation

‣ Four DMA Channels – Each Channel:

• Works on one descriptor at a time

• Has a unique Requester ID (RID)

• Has a programmable traffic class (TC) for QoS

‣ DMA descriptor

• Specifies source address, destination address, transfer size, control

• Internal or external

‣ DMA Read Function

• Initiates Read Requests and collects completions by matching RID

and Tag number

‣ DMA Write Function

• Converts Read Completion streams into Memory write streams

DMA Descriptor Overview

‣ Descriptors are instructions for DMA

• Written by CPU

• Stored in a ring in Host Memory

OR

• Stored internal to the PCIe Switch

Descriptor

Ring

Descriptor N – 1

32b

‣ 16B standard format

• Supports 32-bit Addressing

• Control/Status information

Descriptor 2

Descriptor 1

DstAddr

SrcAddr

SrcAddrH, DstAddrH

Transfer Size, Control

‣ 16B extended format

• Supports 64-bit Addressing

• Control/Status information

Descriptor 0

DMA Descriptor Prefetch

‣ DMA channel prefetches 1 to 4 descriptors at a time when in

external descriptor mode

‣ Internal buffer support for up to 256 descriptors

Active Channels Descriptors per Channel

1 256

2 128

4 64

‣ Descriptors are prefetched to internal buffer until filled

• Control in place for # of descriptors to be prefetched

• 1, 4, 8, or max per channel

‣ Invalid descriptors are dropped

• no further descriptor fetch until software clears status

• interrupt optionally enabled

DMA Runtime Flow – Host to I/O

Memory

FPGA

CPU

RC

Switch

ASIC

ORANGE text describes CPU tasks

1. CPU Programs Descriptors in RAM

2. CPU enables DMA

3. DMA reads Descriptors in RAM

a. DMA prefetches 1-256 descriptors

4. DMA works on 1 descriptor at a time

a. DMA reads source

b. Completions arrive in switch

c. Completions are converted to Writes

d. DMA Writes to Destination

e. (There can be multiple read/write per

descriptor)

f. Clears Valid Bit on Descriptor after last

write (optional)

g. Interrupt CPU after descriptor (optional)

h. Start next descriptor

5. End of Ring (DMA Done)

6. CPU receives and handles interrupts

DMA Performance

‣ Ordering enforced within a DMA channel only

• Descriptors are read in order from Host Memory

• Data within a descriptor is moved in order

• Read Requests (MRd) are strictly ordered

• Partial completions per MRd follow PCIe Spec – Out of order tags

will be re-ordered on the chip

• Write Requests (MWr) are strictly ordered

‣ Full Line Rate Throughput

• One channel can saturate one link in one direction

• x8 at 5GT/s with 64B Read Completions

• Two channels can saturate both directions

• x8 at 5GT/s with 64B Read Completions

‣ Programmable Interrupt Control

• Less interrupts Less CPU utilization

‣ Data Rate Controls in place to control the maximum read BW

• Transfer Max Read Size every X clocks (programmable)

Data Integrity

Data Integrity and Error Isolation

‣ PLX supports protection against PCIe errors

• Providing robust system through data integrity & error isolation

‣ PCIe Error Types

• Malformed packets

• EP (Poisoned TLPs) & ECRC errors

• 1-bit ECC & 2-bit ECC

• LCRC

• PHY Errors: Disparity, 8b/10b Encoding, Scrambler & Framing

• Receiver Overflow, Flow Control Protocol Error

• PLX Device Specific : ECC, UR overflow

Data Integrity

‣ Internal data path protection from ingress to egress

• Complete data protection through ECC

• ECRC on ingress and egress ports

‣ Higher performance by reducing re-transmission

Error Isolation – Fatal Errors

‣ User selectable behavior on fatal errors

‣ Malformed Packet or Internal Fatal Error handling

• Mode 1 (Default)

• Assert FATAL_ERR# pin and send Error Message

• Mode 2

• Generate Internal Reset (equivalent to in-band hot reset)

• Mode 3

• Block all packet transmissions

• Cancel packets in transit with EDB

• Mode 4

• Block all packet transmission

• Bring upstream link down to cause surprise down

Error Isolation – EP/ECRC

‣ User selectable behavior with packet error

‣ Poisoned packet (EP) & ECRC error handling

• EP/ECRC Mode 1 (Default)

• Forwarded with appropriate logging

• EP/ECRC Mode 2

• Drop EP/ECRC packet

• Not forwarded, only logged

• EP/ECRC Mode 3

• Block violating device

• Drop and Block EP/ECRC packet

Gen 1 Electrical & Mechanical Summary

Device

Package Size*

(mm 2 )

Typical Power

Consumption (Watts)

Max. Power

Consumption (Watts)

PEX 8505 15 x 15 mm 2 0.8 W 1.4 W

PEX 8508 19 x 19 mm 2 1.6 W 2.5 W

PEX 8509 15 x 15 mm 2 1.2 W 1.8 W

PEX 8511 15 x 15 mm 2 1.0 W 1.6 W

PEX 8512 23 x 23 mm 2 2.2 W 3.1 W

PEX 8513 19 x 19 mm 2 1.3 W 2.6 W

PEX 8516 27 x 27 mm 2 3.2 W 4.3 W

PEX 8517 27 x 27 mm 2 2.6 W 3.6 W

PEX 8518 23 x 23 mm 2 2.6 W 3.6 W

PEX 8519 19 x 19 mm 2 1.7 W 2.6 W

PEX 8524 31 x 31 mm 2 3.9 W 6.1 W

PEX 8525 31 x 31 mm 2 2.6 W 3.8 W

PEX 8532 35 x 35 mm 2 5.7 W 7.4 W

PEX 8533 35 x 35 mm 2 3.3 W 4.8 W

PEX 8547 37.5 x 37.5 mm 2 4.9 W 7.1 W

‣ Voltages – 3 sources

• 1.0 V Core

• 1.5 V SerDes I/O

• 3.3 V I/O

‣ Thermal

• Industrial Temp

available on most

products

* All PBGA, 1.0mm Pitch

PEX 8548 37.5 x 37.5 mm 2 4.9 W 7.1 W

Typical: 35% lane utilization, typical voltages, 25 o C ambient temperature

Maximum: 85% lane utilization, max operating voltages, across industrial temperature

Device

** Preliminary Estimates

Gen 2 Electrical & Mechanical

‣ Voltages – 2 sources

• 1.0 V Core & SerDes I/O

• 2.5 V I/O

Package Size*

(mm 2 )

Typical Power

Consumption (Gen 2

Mode)

‣ Thermal

• Commercial Temp

Typical Power

Consumption

(Gen 1 Mode**)

Max. Power

Consumption

(Gen 2 Mode)

Max. Power

Consumption

(Gen 1 Mode**)

PEX 8648 27 x 27 mm 2 3.7 W 3.0 W 8.6 W 7.9 W

PEX 8647 27 x 27 mm 2 2.8 W 2.1 W 7.4 W 6.7 W

PEX 8632 27 x 27 mm 2 2.7 W 2.2 W 6.4 W 5.9 W

PEX 8624 19 x 19 mm 2 1.9 W 1.5 W 4.6 W 4.3 W

PEX 8616 19 x 19 mm 2 1.7 W 1.5 W 4.3 W 4.1 W

PEX 8619 19 x 19 mm 2 1.80 W 1.58 W 4.53 W 4.08 W

PEX 8618 19 x 19 mm 2 1.75 W 1.54 W 4.47 W 4.04 W

PEX 8617 19 x 19 mm 2 1.6 W 1.4 W 4.26 W 3.85 W

PEX 8612 19 x 19 mm 2 1.6 W 1.4 W 4.2 W 4.0 W

PEX 8615 19 x 19 mm 2 1.6 W 1.37 W 4.01 W 3.65 W

PEX 8614 19 x 19 mm 2 1.5 W 1.33 W 3.96 W 3.60 W

PEX 8613 19 x 19 mm 2 1.4 W 1.2 W 3.80 W 3.4 W

PEX 8609 15 x 15 mm 2 1.33 W 1.16 W 3.51 W 3.20 W

PEX 8608 15 x 15 mm 2 1.31 W 1.15 W 3.51 W 3.20 W

PEX 8606 15 x 15 mm 2 1.25 W 1.09 W 3.32 W 3.04 W

PEX 8604 15 x 15 mm 2 1.18 W 1.03 W 3.13 W 2.89 W

Typical: 35% lane utilization, typical voltages 25C° ambient. L0s mode

Maximum: 85% lane utilization, L0 mode, max operating voltages

End of Presentation

Thank You

www.plxtech.com

PCI Express

PCI Express ... View more PCI Express

Delete template?

Save as template ?

PCI Express PCI Express