DCC2 Specification Table of Contents

DCC2 Specification
This document describes the operation of the 2 nd generation CMS HCAL Data Concentrator Card
(DCC2). This version is updated as of firmware version 0x3022.
Last edited on October 9, 2012.
Table of Contents
DCC2 Specification...................................................................................................................................1
Operational Summary................................................................................................................................2
Detailed Functional Description................................................................................................................2
TTCrx....................................................................................................................................................2
Triggers, Event Count and Bunch Count..........................................................................................2
TTC Broadcast Commands...............................................................................................................3
Orbit Gap Triggers (New).....................................................................................................................4
Trigger Rules.........................................................................................................................................4
TTCrx Event Number FIFO..................................................................................................................5
Link Receiver Blocks............................................................................................................................6
Event Builder.........................................................................................................................................6
Resynchronization Logic..................................................................................................................7
VME Interface.......................................................................................................................................7
Flash Memory........................................................................................................................................7
Monitoring Counters.............................................................................................................................7
Global Counters................................................................................................................................7
Global Timers...................................................................................................................................8
Per-Spigot Counters..........................................................................................................................9

Operational Summary
The DCC receives L1A from the TTCrx, and simultaneously receives event fragments from up to 15
HTR boards. The DCC builds events by matching the event numbers from all sources, adds header and
trailer information to conform to the CMS common data format (CDF). The output event stream may
be error-checked and discarded, sent to an Slink transmitter and/or a VME spy buffer. Extensive errorchecking
and monitoring data is collected and may be read over the VME bus or Ethernet.
The DCC logic is divided into several major sections: Five link receiver blocks (3 channels each), an
event builder and Slink and VME interfaces. The link receiver blocks are called LRBs (after the
daughterboards on the DCC1) and the event builder FPGA is referred to as the “DSP chip” in Wu's
documentation. The sections are described in more detail below.
Detailed Functional Description
TTCrx
RJ-45
HTR
inputs
(total 15)
Xilinx
XC3S200A
FPGA
32MByte
DDR SDRAM
Xilinx
XC3SD1800A
FPGA
32MByte
DDR SDRAM
Illustration 1: DCC2 Block Diagram
Triggers, Event Count and Bunch Count
Xilinx
XC3S200A
FPGA VME
DAQ Data
LVDS Tx
TTCrx
UMD
board
The primary function of the TTCRx interface[TTCRx] is to receive and process L1A triggers. In
response to each L1A, the DCC captures the following three quantities and stores them in a FIFO for
use by the event builder:
● Event Number (EvN) is a 24-bit number which increments by 1 for each event. The EvN is set
to '1' upon receipt of an Event Count Reset TTC command.
Buffers
Future backplane
Connector board
at rear of crate
TP Out
Readout bits in
TTS
TTC
VME64x
Buffers
Out
In
S-Link
LSC
RJ-45
to/from
Readout
Processor

● Bunch Count Number (BcN) is a 12-bit number which increments each bunch crossing clock
and is reset to a programmable constant upon receipt of BC0 TTC command.
● Orbit Number (OrN) is a 32-bit number which increments once each orbit when BcN=0. It is
reset to a programmable constant upon receipt of a Reset Orbit Counter TTC command.
Note that the TTCrx can provide the EvN and BcN, but these are ignored by the DCC.
TTC Broadcast Commands
The DCC must respond to the following broadcast commands received by the TTCrx.
Command Brcst Meaning
ResetOrbitCounter 001X1XXX Reset orbit count to zero
Reset calibration laser position to 0
ReSync 010X1XXX Reset the DAQ path – not enabled
HardReset 011X1XXX Reset the DAQ path – not enabled
Start 100X1XXX Start accepting L1As – not enabled
Stop 101X1XXX Stop accepting L1As – not enabled
Gap-Trigger 010X0XXX Expect calibration trigger in next gap
Gap-Pedestal 011X0XXX Expect pedestal trigger in next gap
Gap-Set-Laser 100X0XXX Advance laser position (see Page 4)
EvtCntReset XXXXXX1X Reset EvN
BC0 XXXXXXX1 Reset BcN
ResetOrbitCounter: Reset orbit counter to a fixed value near zero. This should probably be a
programmable constant with ~4 bits.
ReSync, HardReset: Clear the DAQ path and reset all data-containing buffers in response to any of
these. L1A will be suspended well in advance of ReSync. VME (PCI) programmable registers should
not be affected. TTS output should go to 'BUSY' when ReSync is received, and return to 'READY'
when appropriate. Note: this feature is not implemented
Start, Stop: Enable and Disable accepting L1A. Should be an 'or' with corresponding VME control
bits for testing. Note: these features are not implemented
StatReq: Indicates that the LRB status request header bit was set for this event. LRB monitoring data
should be captured.
Gap-Trigger: Indicates that in the gap at the end of this orbit a laser trigger will occur.
Gap-Pedestal: Indicates that in the gap at the end of this orbit a pedestal trigger will occur.
Gap-Set-Laser: Controls the current laser setting. Please see the section below on Page 4 below for

details of orbit gap calibration triggers.
EvtCntReset (aka ECR): Reset the event number to 1. The TTCrx will reset it's internal event
number to 0, so if the TTCrx event number is used, a constant 1 must be added for each event.
BC0 (aka BCR): Reset BcN (BX_id) to zero after a (programmable) pipeline delay of a few clocks.
When reset occurs, check that BcN = 3563. If not, generate error condition.
Orbit Gap Triggers (New)
HCAL calibration will be performed using two special trigger types which occur during the “gap”
portion of the LHC orbit at about BX=3490. These triggers will be preceded by a Gap-Trigger TTC
broadcast command for laser triggers or a Gap-Pedestal TTC command for pedestal triggers. They
use the standard L1A event numbering sequence and data format, except for the addition of a new
“calibration trigger type” field in the DCC header. Please see the new calibration window register
described on Page Error: Reference source not found and additions to the table of TTC broadcast
commands on Page 3.
For L1A to be recognized as a calibration trigger, it must occur in the same orbit as a Gap-Pedestal or
Gap-Trigger command, within the bunch crossing range programmed in the calibration window
register.
The calibration laser may be set to various configurations (details not important here). The DCC
maintains a laser position register which may be read (but not written) as part of the calibration window
register. It is managed as follows by TTC broadcast commands.
• OCR resets the laser position to 0
• A series of (n) Gap-Set-Laser commands sets the position to (n)
• If a gap of > 8192 orbits (about 730ms) occurs between Gap-Set-Laser commands, the
counting starts over
Two event header fields are set as follows in calibration events.
Event Type
(TTC command)
Pedestal
(Gap-Pedestal)
Laser
(Gap-Trigger)
Normal
(None)
Trigger Rules
Evt_ty
(bits 56-59 of 1 st 64b word)
2 1
Cal_ty
(bits 56-59 of 2 nd 64b word)
2 Current laser setting
1 0
The estimated L1A rate is 100 kHz. There are ``trigger rules'' which limit the L1A rate further. From
the CMS Trigger TDR[TDR] (see section 16.4). These are provided only for reference... the DCC does
not need to check for violations of these, but may assume that they are implemented as stated.
i. No more than 1 L1A per 3 BX (75ns)

ii. No more than 2 L1A per 25 BX (625ns)
iii.No more than 3 L1A per 100 BX (2.5μs)
iv. No more than 4 L1A per 240 BX (6μs)
The DCC implements a “fifth trigger rule” to protect the HTR logic against overflow as follows:
No more than 30 L1A per 12000 BX (300μs)
This rule is implemented using a 5 bit up-down counter which increments once pr L1A, and
decrements once per 10μs. If the count reaches 30, TTS BSY is asserted until the count drops to 15.
This feature may be disabled by setting a configuration register bit. A counter counts the number of
times this rule is invoked. NOTE: This feature is in violation of guidance from the CMS trigger group,
and should not be enabled for normal running.
The L1A with bunch count and event number are stored in a FIFO. An overflow of this FIFO is a fatal
error and should be reported as a loss-of-sync.
The TTCRx is programmed with an I2C Interface which should be controllable from the DCC logic
board.
TTCrx Event Number FIFO
The DCC provides a circular buffer which contains information about the most recent 1k L1A received
by the DCC. Each event results in a total of 16 bytes of information written, as detailed below.
Offset Bits Description
0 23-0 Event Number
4 27-24 Current laser setting
23-20 TTS state
19-16 Cal_ty (calibration type) as in event header
15 Gap-Trigger TTC command received
14 Gap-Pedestal TTC command received
13 Trigger within calibration BX window
12 Valid orbit gap calibration trigger
11-0 BcN
8 31-0 OrN
0xc Not used (all zero)
The circular buffer is contained in the address range 0x4000-0x7ffc. The address of the most recent
L1A received is at the address:

0x4010 + (TTCrx_fifo_read_pointer)
where TTCrx_fifo_read_pointer is documented on Page Error: Reference source not found. Older L1A
are found at increasing addresses (add 0x10 for each L1A). If address 0x8000 is reached, wrap address
back to 0x4000. Note that since the FIFO is never explicitly cleared, one should not read more than the
number of L1A indicated by the event builder block count register (documented on Page Error:
Reference source not found).
Link Receiver Blocks
There are 3 identical input link receiver blocks (LRB), each consisting of three RJ-45 type connectors,
a Spartan-3A FPGA, and a 32 Mbyte DDR SDRAM. Input data is received on each RJ-45 connector as
serial LVDS data at a rate of 80MBytes/sec (40MHz * 16-bit words) on each link. The protocol is
compatible with the National Semiconductor Channel Link (and in fact is transmitted by a National
DS90CR285 transmitter on the HTR). One data block is expected from the HTR per L1A. Data are
de-serialized by the Spartan-3AN FPGA . Headers and trailers for each event are forwarded
immediately to the event builder FPGA . Event fragments are stored in their entirety in the Event FIFO
(one FIFO per input spigot) implemented in the SDRAM. The HTR input data are protected with
hamming codes on each word and a 16-bit CRC on each event fragment. The LRB checks these codes
and keeps track of any errors seen.
The low-level input protocol is given in the table below.
HTR Input Function Notes
0 H0 Hamming code bit 0
1 H1 Hamming code bit 1
2 D0 Data bit 0
3 H2 Hamming code bit 2
4 D1 Data bit 1
5 x Not used due to PCB layout crosstalk issue (ignored
by DCC). Set to '0' at HTR.
6 D3 Data bit 3
7 H3 Hamming code bit 3
8 D4 Data bit 4
9 D5 Data bit 5
10 D6 Data bit 6
11 D7 Data bit 7
12 D8 Data bit 8
13 S1 Framing bit, set to '1' for header and trailer words

HTR Input Function Notes
only
14 D10 Data bit 10
15 H4 Hamming code bit 4
16 D11 Data bit 11
17 D12 Data bit 12
18 D13 Data bit 13
19 D14 Data bit 14
20 D15 Data bit 15
21 D2 Data bit 2
22 D9 Data bit 9
23 P Parity bit
Event Builder
The Event Builder consists of a larger Spartan-3A FPGA with associated 32 Mbyte DDR SDRAM. A
FIFO implemented internally in the FPGA stores header and trailer information for up to 256 events
from each of the 15 HTR inputs. A separate L1A FIFO stores up to 1k L1A. In each FIFO the event
number (EvN), bunch crossing number (BcN) and orbit number (OrN) are stored.
The event builder retrieves the next entry from all enabled FIFOs, and if the EvN match the event is
built according to the documented event format. If the EvN do not match, an optional procedure
described below is entered in an attempt to recover synchronization.

HTR Inputs
(15 total)
Rx
TTCrx
Resynchronization Logic
If the DCC and HTR event numbers either (a) match or (b) their 4 LSBs match and the 4 LSBs of the
recent HTR event numbers are consecutive, the event is accepted. Otherwise, the DCC EvN is
compared with that of the next HTR EvN in the FIFO. If the next HTR EvN is (a) equal or (b) smaller
and the difference is less than 16 and the four LSBs of the next few HTR EvN are consecutive, the
event builder will skip the current HTR event. If the next HTR EvN is larger and the difference is less
than 16 and the four LSBs of the next few HTR EvN are consecutive, the DCC will insert (pad) the
event with an empty block for the corresponding HTR input. In any case, as long as the errors to not
persist, the DCC will eventually resynchronize the input streams. This logic may be completely
disabled if desired.
VME Interface
The VME interface on the DCC2 is implemented using a separate FPGA.
Flash Memory
Configuration information for the 7 FPGAs on the DCC2 is held in a...
Monitoring Counters
The DCC2 contains a large number of counters which keep track of the number of times a particular
condition has occurred. These counters may be read at any time via the VME or Ethernet interfaces
without affecting data taking.
Global Counters
Input
Logic
Event FIFO
256 evt
1K
L1A FIFO
256 evt
Header FIFO
OFW, BSY, SYN
OFW, BSY, SYN
Event
Builder
Illustration 2: DCC2 Buffering
The following counters record per-DCC quantities. All are 32 bits unless otherwise stated.
Event builder block counter No. events processed through event builder
128 evt VME
FIFO SLink

S-LINK block counter events written to S-LINK transmitter card
Monitored event counter events written to VME spy buffer
L1A counter L1A received by TTCrx on DCC2
Calibration trigger counter L1A received as valid orbit gap trigger (see section xxx)
CT EvN mismatch counter Obsolete?
CT BcN mismatch counter Obsolete?
L1 EvN mismatch counter L1A received where one or more HTR EvN do not match DCC
EvN
L1 BcN/OrN mismatch counter L1A received where one or more HTR BcN or OrN do not match
DCC BcN or OrN
Bcnt error counter BC0 received where current bunch count was not 3563
Trigger rule 5 violation counter times trigger rule 5 (see Page 5) was violated
S-LINK CRC error counter times an SLINK CRC was detected on output.
This counter is for internal DCC debugging only.
Event builder HTR CRC error
counter
HTR CRC errors detected in event builder chip.
This is a double-check on the similar logic in the LRB
Event builder word count (64 bit) 32-bit words processed by event builder
S-Link word counter (64 bit) 32-bit words sent to S-Link transmitter
TTCrx single errors (64 bit) Single bit errors detected by TTCrx
TTCrx multi bit errors (64 bit) Multi-bit errors detected by TTCrx
Global Timers
The following counters measure the time the DCC2 logic has been in particular states since last reset.
These counters count at 64MHz based on an on-board crystal in the DCC2 module (confirm this!)
Run time counter Time DCC2 has been in run mode
READY time counter Time DCC2 TTS output has been in READY state
BUSY time counter Time DCC2 TTS output has been in BUSY state
(not able to accept L1A)
Overflow warning time counter Time DCC2 TTS output has been in overflow warning state

Sync lost time counter Time DCC2 has been in “lost synchronization” state
If this counter is non-zero data has been lost/corrupted
Per-Spigot Counters
LRB Counters
These counters are located in the LRB logic. They typically provide accurate counts even if the event
builder logic has stopped for some reason. They are 48 bits long unless otherwise noted.
Word counter 32-bit Words received at LVDS input
Corrected hamming code errors Words received with correctable errors (16 or 32 bit words?!)
Uncorrected hamming code errors Words received with uncorrected errors (32 bit words)
Truncated events Event fragments received longer than maximum (512 32-bit
words) from HTR
Bad ID counter Event fragments where 8 bits of EvN in header did not match
trailer
CRC error counter Event fragments with incorrect HTR CRC
Event Builder Counters
These counters are located in the Event Builder logic. They provide accurate counts for events which
have cleared the Event Builder. Counters are 32 bits unless otherwise noted.
EvN mismatch counter Event fragments where HTR EvN DCC EvN
BcN mismatch counter Event fragments where HTR BcN DCC BcN
OrN mismatch counter Event fragments where HTR OrN DCC OrN
Skipped event counter Event fragments discarded by event builder to maintain
synchronization (see Page 8)
Padded event counter Event fragments inserted by event builder to maintain
synchronization (see Page 8)
Events with CERR counter Event fragments with one or more corrected HTR-DCC link
errors
Events with UERR counter Event fragments with one or more uncorrected HTR-DCC link
errors
Truncated event counter Event fragments truncated because > 512 (32-bit) words long

Short event counter Event fragments shorter than 32 bytes
Events with structure error counter Event fragments where header/data/trailer sequence on HTR-DCC
link is violated
Odd 16bit word count Event fragments with odd 16-bit word count on HTR-DCC link
HTR CRC error in event builder Event fragments with HTR CRC error detected in event builder
HTR CRC error disagreement Event fragments where LRB logic and event builder disagree on
CRC value (indicates internal DCC logic malfunction)
Word count (64 bits) 32-bit words received by event builder input
Event count (64 bits) Event fragments received by event builder input
Note: The following counters monitor the HTR status bits in the 2 nd 32-bit HTR payload word.
The meanings are current as of this writing, but please consult the HTR documentation for updates.
Events with HTR OW flag Event fragments with HTR status bit 0 set (overflow warning)
Events with HTR BZ flag Event fragments with HTR status bit 1 set (busy)
Events with HTR EE flag Event fragments with HTR status bit 2 set (empty event sent)
Events with HTR RL flag Event fragments with HTR status bit 3 set (rejected L1A)
Events with HTR status bit4 set Event fragments with HTR status bit 4 set (bad data sent)
Events with HTR LW flag Event fragments with HTR status bit 5 set (latency warning)
Events with HTR OD flag Event fragments with HTR status bit 6 set (optical data error)
Events with HTR CK flag Event fragments with HTR status bit 7 set (clock problem)
Events with HTR BE flag Event fragments with HTR status bit 8 set (wrong orbit length)
Events with HTR status bit9 set Event fragments with HTR status bit 9 set (unused)
Events with HTR status bit10 set Event fragments with HTR status bit 10 set (unused)
Events with HTR status bit11 set Event fragments with HTR status bit 11 set (unused)
Events with HTR status bit12 set Event fragments with HTR status bit 12 set (test mode)
Events with HTR CT flag Event fragments with HTR status bit 14 set (calibration trigger)
Events with HTR status bit15 clr Event fragments where HTR status bit 15 is zero (error)
Note: This bit is '1' as defined in the HTR data format.
Events with HTR US bit set Event fragments marked as “unsuppressed” from HTR
(Bit 15 in the 4 th 32bit payload word from the HTR set to '1')

Front Panel LEDs
The DCC2 has a number of front-panel LEDs. The meaning of each is indicated below. Note that each
LED has a steady-state meeting (always on) and an error meaning (blinking). Any occurrence of the
error condition will cause the LED to blink for several seconds, or if the error is recurring it will blink
continuously
Top Group
LED
Name
Steady-state function Error Function
VME Indicates VME activity -none-
TTC Indicates TTCrx ready
(TTC input signal present)
L1A Blinks when L1A seen -none-
Single or double bit TTC error or
orbit length error
DAQ SLINK enabled for output Link down active when SLINK enabled
DCC DCC in run mode -none-
TST Currently not used; proposed to blink to
indicate power-up memory test error
Middle Group (TTS)
RDY TTS 'Ready' -none-
BSY TTS 'Busy' output (not accepting L1A) -none-
OFW TTS 'Overflow Warning' output (L1A buffer
almost full)
SYN TTS 'Sync Lost' output (L1A buffer
overflowed or persistent EvN mis-match)
HTR0 ...
HTR14
HTR input enabled, data present, and data
valid (passes simple checks in DCC)
-none-
-none-
HTR input enabled and:
no HTR data or data fails format checks