03m-wp2-040.170.010-td-001-1-pulsar SP on Uniboard.pdf

PULSAR SIGNAL PROCESSING ON UNIBOARDDocument number .................................................................. WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision ........................................................................................................................... 1Author .......................................................................................................... A AhmedSaidDate ................................................................................................................. 2011‐04‐01Status ............................................................................................... Approved for releaseName Designation Affiliation Date SignatureAdditional AuthorsRob Ferdman, Ben StappersSubmitted by:A AhmedSaid UMAN 2011‐04‐01Approved by:W. Turner Signal ProcessingDomain Specialist<strong>SP</strong>DO 2011‐03‐26

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1DOCUMENT HISTORYRevision Date Of Issue Engineering ChangeNumberCommentsA 1 st April 2011 ‐ First draft release for internal review1 1 st April 2011DOCUMENT SOFTWAREPackage Version FilenameWordprocessor MsWord Word 2003 <strong>03m</strong>‐<strong>wp2</strong>‐<strong>040.170.010</strong>‐<strong>td</strong>‐<strong>001</strong>‐1‐<strong>pulsar</strong> <strong>SP</strong> on Uniboard‐2003Block diagramsOtherORGANISATION DETAILSNamePhysical/PostalAddressSKA Program Development OfficeJodrell Bank Centre for AstrophysicsAlan Turing BuildingThe University of ManchesterOxford RoadManchester, UKM13 9PLFax. +44 (0)161 275 4049Website www.skatelescope.org2011‐04‐01 Page 2 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1LIST OF FIGURESFigure 1 The design methodology ........................................................................................................... 9Figure 2 The FFT Dedispersion method ................................................................................................ 10Figure 3 : The Polyphase filter bank dedispersion method .................................................................. 11Figure 4 Pulsar data folding .................................................................................................................. 12Figure 5 Incoherent dedispersion for 1 DM .......................................................................................... 14Figure 6 Incoherent dedispersion architecture .............................................................................. 15Figure 7 4 inputs Taylor tree ................................................................................................................. 15Figure 8 Rearranged 4 inputs Taylor tree ............................................................................................. 16Figure 9 Radix 2 incoherent dedispersion architecture ........................................................................ 16Figure 10 Radix 4 incoherent dedispersion architecture ...................................................................... 17Figure 11 Parallel Pulsar Searching algorithm ...................................................................................... 17Figure 12 Computation time Vs The total number of sub bands processed using the tree algorithm 22No table of figures entries found.LIST OF TABLES2011‐04‐01 Page 4 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1LIST OF ABBREVIATIONSDDR ............................... Double Data RateDM ................................. Dispersion MeasureFFT ................................ Fast Fourier TransformFPGA ............................. Field Programmable Gate ArrayHDL ............................... High Definition LanguageIFFT ............................... Inverse Fast Fourier TransformISM ................................ Inter Stellar MediumRAM .............................. Random Access MemoryRTL ................................ Register Transfer LevelSKA ............................... Square Kilometre ArrayTDD ............................... Total DeDispersion time in clock cyclesTFFT .............................. Total FFT computation time in clock cycles2011‐04‐01 Page 5 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1cases, it has been possible to do so in real time, depending on the required FFT lengths. Otherwise,the baseband data is recorded to disk for offline processing.The real‐time observation and coherent de‐dispersion of <strong>pulsar</strong> data is becoming a necessity, asthere are ongoing efforts towards obtaining larger and larger bandwidths of data and in the case ofthe SKA there will be multiple <strong>pulsar</strong>s observed simultaneously, up to tens. The data rate requiredto obtain such volumes of raw data is incompatible with offline storage capabilities (and costs),whereas folded, de‐dispersed pulse profile data (to be discussed later) is, by comparison, negligiblein size. However, the ability of computer processors to perform real‐time coherent de‐dispersion insoftware amongst nodes on a computer cluster, or by using graphics processing units (which hasrecently seen a surge in popularity), is also becoming limited, due in part to disk input/outputconstraints. This is why the concept of UNIBOARD is a great match for the requirements of <strong>pulsar</strong>observations.5 Design MethodologyFigure 1 The design methodologyThe adopted methodology consists of creating high level algorithm simulations, primarily within theMATLAB environment. These algorithms are then converted into accurate system models, withfurther simulation. Finally the models are converted into hardware implementations using anappropriate methodology.Figure 1 illustrates this methodology.5.1 Algorithmic SimulationUsing the Mathworks Matlab software, it is possible to simulate complex algorithms relativelyquickly and easily. This algorithmic simulation allows the rapid evaluation of the different D<strong>SP</strong>methods that can be used and the exploration of the parameter space.5.2 Architectural ModellingThe next step after the algorithmic simulation is the translation of the high level algorithms intosystem accurate Simulink models. This modelling stage takes into consideration the hardwarearchitecture, its capabilities and limitations. For example fixed point integer arithmetic is usedinstead of floating point arithmetic.5.3 Hardware ImplementationFinally, the models are converted into hardware using a combination of automatic netlist generationtools and HDL coding to structural and RTL level.2011‐04‐01 Page 9 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1This method, illustrated in Figure 2 consists of taking the signal into the frequency domain using anFFT, multiplying it with the dedispersion coefficients (phase shifts) then returning the signal into thetime domain using an inverse FFT.The FFT/IFFT lengths are relatively high and might range from 256M‐1G points. For this reason, it isnecessary to develop an FPGA architecture for large FFTs and IFFTs. Such architecture would usemultiple ‘standard’ FFT cores to deal with these sizes and a sufficient amount of off chip memory. Itis estimated that 4 FFT cores will be needed per large FFT/IFFT pair and for every 150‐250MHz ofbandwidth (depending on the FGPA clock). The off chip memory requirement in bytes is at least 12times the size of the large FFT in order to store 2x3 blocks of data with 2 bytes per complex sample.The total bandwidth requirement is at least 12 times that of the FFT input data (1.8 GB/s‐3 GB/s)because 6 complex sample need to be read/written every clock cycle.6.3 The Polyphase Filter Bank MethodFigure 3 : The Polyphase filter bank dedispersion methodIn this method, illustrated in Figure 3, the signal is split into a number of frequency channels using apolyphase filter bank, and then the FFT method is applied to de‐disperse each channel.The number of channels is likely to be in the range 64‐256 channels of 4MHz bandwidth. With 4‐8coefficients per channel, the filter bank prototype filter will have 256‐1024 coefficients and eachcoefficient will be represented by 8‐12 bits.In this method, the size of the FFTs in the dedispersion part of the processing is smaller than in theprevious method. It is in fact divided by the number of channels. Therefore, the FFT length will be atmost 4M points.2011‐04‐01 Page 11 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1Let T be the time duration of a buffer, BW the signal’s bandwidth and Ffpga the FPGA’s clockfrequency. There will be T x BW = N x M complex samples in a buffer and they have to be processedin no more than T x Ffpga cycles. Therefore, in order to be able to process data in real time:4 x T x BW x log 2 (T x BW / N)/(P x B) BW x log 2 (T x BW / N)

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1Figure 8 Rearranged 4 inputs Taylor treeThe Taylor tree algorithm is similar in its structure to the FFT algorithm. A common basic structure inthis algorithm is a Delay‐Add Butterfly as highlighted in Figure 8. It is very similar to the FFT’s‘Multiply‐Add Butterfly’.Inspired by FFT architectures, a possible incoherent dedispersion architecture is presented in Figure9.Figure 9 Radix 2 incoherent dedispersion architectureIn this architecture, data is stored in the off chip memory, it is then loaded into internal inputbuffers, passed through a Delay‐Add Butterfly and the result is stored in internal output buffer. Fromthere, output data is stored back in the off chip memory and the process is repeated until all theoperations are completed. The Delay‐Add Butterfly consists of two adders and a ‘d’ samples delay.The delay ‘d’ could be equal to 1 or whatever is deemed appropriate. The delay ‘d’ can also bevariable to allow the generalisation of the Taylor tree algorithm which assumes unit delays and isonly applicable to small bandwidths where the dispersion curve is linear. By having a variable delay itmight be possible to apply the Taylor tree algorithm to large bandwidths if a good schedule of delayaddoperations can be found (using a computer program).2011‐04‐01 Page 16 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1Figure 12 Computation time Vs The total number of sub bands processed using the tree algorithmThe best case scenario is when D = 1 and K =N. In this case, the de‐dispersion computation time isminimised and more importantly, there is no need to have extra storage for intermediatecomputations. The input data can be overwritten because it will not be needed anymore thereforethe memory size requirement drops to:TotalExtMemSize = Nt x M x WsAnd the computation times becomes:TDD = M x (Nt x (1+2 x log2(Nt/B)) / B+ 2 x (B+1) x Nt/B) x (Ws x Ffpga) / BWmemTFFT = 2 x Nt/B x M x log2(M) x (Ws x Ffpga) / (BWmem x B)Now the memory size is divided by 3 and assuming the same values for the other parameters asabove, the memory requirement is 8.73 GB per FPGA which is more likely to be feasible andaffordable.2011‐04‐01 Page 22 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 18.6 Mapping a search system into UniboardA Uniboard has 8 FPGAs and each one of them has access to two DDR3 banks of 2GB each capable of800 M transfers/s of 64 bits.The parameters values that have been used in the previous section match specifically the Uniboardcase. Therefore, for D x K =Nt with D < 64 and: ObservationTime = 600 sec; SamplingFrequency = 2GHz; Nt = 8192; M = 9.16E6; Ffpga = 200MHz; Ws =8 bits.The memory required per FPGA is 26.2 GB which is higher than the available 4GB. In the second casewhere D = 1 and K =N, 8.73 GB per FPGA are required. If Ws is reduced to 4 bits then 4.4 GB perFPGA will be required and with a 10 % reduction of the signal bandwidth 4 GB per FPGA will berequired. Thus, using one Uniboard it is possible to implement a real time <strong>pulsar</strong> search engine for abandwidth of 900MHz and 1024 DMs.Another possibility is to use several UNIBOARDS, for example 4 boards, with Ws =4, would be able toprocess 1GHz and 8192 DMs.8.7 Summary and Looking AheadWe have shown that it is likely possible to coherently dedisperse a full 1 GHz of dual‐polarisationbandwidth, appropriately divided into manageable subbands, with a single Uniboard. The number ofsubbands required results in a bandwidth per subband which is still large enough to achieve the timeresolution desired for the high precision timing experiments. Based on today’s prices a Uniboard inproduction will cost about 10,000 euros. Given that in SKA phase one we are looking at trying totime a few tens of <strong>pulsar</strong>s simultaneously this should be achievable for something on the order of250,000 euros. The expected power consumption of 350W per board would then result in a total ofabout 8kW. Each board would likely need a control PC and the data would also need to be archived.In the case of the <strong>pulsar</strong> searching the current Uniboard could likely deal with the expectedbandwidth required for the Galactic plane search at a central frequency of about 800 MHz. Thebandwidth for that search is likely to be less than that assumed for the calculations above (basedmore on the 1‐2 GHz range) and by adjusting the number of bits required we could make it possibleto dedisperse and search about 500 MHz of bandwidth. A more detailed analysis of therequirements for the Aperture Array search would be required, but the total number of dispersiontrials is likely to be similar to the above, and the total data rate less, so it will probably also fit.Assuming 1GHz beams and a price per Uniboard of 10,000 euros per beam, and considering thenumber of dispersion trials we can estimate then it will cost about 5 euros per beam per DM. Thismetric is useful as one of the limiting factors is the number of dispersion measures which needs tobe calculated. As above assuming 350W power consumption per board then it will cost 0.17W perbeam per DM. However, these estimations are specific to Uniboard and should be considered asupper limits because it is possible that the Uniboard resources and the FPGAs will not be fullyutilised.2011‐04‐01 Page 23 of 24

WP2‐<strong>040.170.010</strong>‐TD‐<strong>001</strong>Revision : 1It is estimated that something like 4000 beams will be needed in SKA Phase 1 and in this model oneUniboard is needed for each beam. However it may be possible that the increased processingcapabilities, and hopefully increased amount of memory on Uniboard 2, or a subsequent board, willimprove the efficieny and reduce cost. As noted above it is possible that the capability of the fpgas inUniboard will not be fully utilised and so a next step would be to consider how we might be able toimplement the computationally costly acceleration search as part of this package.2011‐04‐01 Page 24 of 24