Low-Frequency, All-Sky Imaging with the PAPER Array Using AIPY

Low-Frequency, All-Sky 

Imaging with the PAPER 

Array Using AIPY 

A. Parsons 1 , D. Backer 1 , G. Foster 1 , 

R. Bradley 2,4 , C. Parashare 2 , N. Gugliucci 2 , 

E. Mastrantonio 4 , J. Aguirre 3 , D. Jacobs 3 , 

C. Carilli 5 , A. Datta 5 , 

M. Lynch 6 , D. Herne 6 , T. Colegate 6 

1 U. of California, Berkeley, 2 U. of Virginia, 3 U. of Pennsylvania, 

4 NRAO, Charlottesville, 5 NRAO, Socorro, 

6 Curtin U., Perth, Australia

The PAPER Architecture 

Flexible FPGA-based 

Packetized Correlator 

Full-Stokes 

Large # Ants (scalable)! 

Wide Band (up to 200 MHz)! 

2048 Channel Polyphase 

Filter Banks 

4-bit Cross-Multipliers 

Non-tracking Crossed Dipoles 

Wide Bandwidth (125-205 MHz)! 

Movable (unburied TV cable)! 

Smooth Beam 

AIPY: Model-based Imaging/Calibration 

Bootstrap Imaging/Calibration 

W Projection, Multi-frequency Synthesis 

Python-Based, Ties in MIRIAD, HEALPIX 

Measurement Equation-based Parameter Fit

PAPER Antennas/Analog Electronics 

Developed by the Charlottesville (NRAO,UVA) Team 

! Smooth beam (spatially and spectrally) 

! Gain sensitive to ambient temperature

PAPER/CASPER Packetized Correlator 

Two Dual 

500 Msps 

ADCs + IBOB 

Berkeley Wireless Research Center's BEE2

Two Calibration/Imaging Paths... 

Parameter Space: 

Bootstrap: Direct, fast, imperfect 

“Sometimes you can't get started because you can't get started” – Don Backer 

Model-Fit: Clean, correct, expensive 

AIPY: Astronomical 

stronomical 

Interferometry nterferometry in Python Python 

an open-source toolkit targeting 

low-frequency interferometry 

http://pypi.python.org/pypi/aipy

AIPY is: 

AIPY isn't: 

What AIPY Is (and Isn't)! 

! A module adding tools for interferometry to Python (not visa versa)! 

! An amalgam of pure-Python and wrappers around C++ and Fortran 

! Object-Oriented 

! A collection of low- to mid-level operations (you write the program)! 

! A toolkit (i.e. a grocery store, not a restaurant)! 

! In beta release (frequent updates/bug-fixes/changes)! 

! A solution (it just helps you find it)! 

! A one-stop shop (it makes use of other open-source projects)! 

! A replacement for other interferometry packages 

! Wedded to a file format (but only MIRIAD-UV, FITS currently supported)! 

Philosophical Pedantry: 

! Follow the (abbreviated) Zen of Python: 

! Explicit is better than implicit 

! Simple is better than complex 

! Complex is better than complicated 

! Special cases aren't special enough to break the rules 

! Practicality beats purity 

! There should be one (and preferably only one) obvious way to do it, 

although that way may not be obvious at first unless you're Dutch 

! Now is better than never 

! In the face of ambiguity, refuse the temptation to guess 

! Don't handicap the programmer; allow them all the rope they want 

! Don't hide data; return it (or at least grant access) at every turn

MODULES: 

ant/sim/fit 

const 

coord 

deconv 

healpix 

img 

loc 

map 

miriad 

optimize 

src 

----3rd party---ephem 

pyfits 

numpy 

pylab/matplotlib 

The Toolbox 

geometry, phase / gain, amplitude / parameter fitting 

physical constants 

coordinate transforms (eq, ec, ga, precession, etc.)! 

deconvolution (clean, lsq, mem, anneal)! 

wrapper around healpix C++ (Max-Planck-Institut)! 

imaging (gridding/weighting, W projection, beam calc)! 

machinery for importing user-defined AntennaArrays 

generation of all-sky maps, faceting 

interface to MIRIAD files 

parameter fitting, optimization code 

lists of known sources and parameters 

-----------------------------------------------------------------------------astrometry 

(relied upon heavily)! 

reading/writing FITS files 

numerical python (fundamental to everything)! 

plotting, map projection 

and example scripts for: 

manipulating/visualizing MIRIAD files, flagging RFI, filtering out crosstalk, 

fitting parameters, simulating arrays, making/viewing all-sky maps, 

phasing to/filtering out sources, imaging, etc.

AntennaArray 

! set_jultime(...)! 

! gen_phs(...)! 

! gen_uvw(...)! 

! sim(...)! 

Antenna 

! x,y,z 

! dly,off 

! bp_r,bp_i 

Beam 

! response(...)! 

! 

Building a Simulator 

Antenna 

! x,y,z 

! dly,off 

! bp_r,bp_i 

RadioFixedBody 

! compute(...)! 

! ra,dec 

! str,index 

! angsize 

SrcCatalog 


! get_crd(...)! 

YourModule.py 

! get_aa(...)! 

! get_catalog(...)! 

RadioSpecial 


! str,index 

! angsize

PAPER Deployments 

PGB: PAPER, Green Bank PWA: PAPER, Western Australia 

! PGB-4: 2004-6, ants w/o flaps, 1 brd correlator 

! PGB-8: 2006-8, upgraded ants, packet correlator 

! PGB-16: 2008-?, testing & characterization 

! Prototype facility for PWA deployments 

38:25:59.24 N 

-79:51:02.1 W 

Green Bank, WV 

! PWA-4: 2007, ants w/o flaps, 1 brd correlator 

! (PWA-32): 2009, ants w/ flaps, packet correlator 

! (PWA-128): 2010? 

! Low-RFI, full-capability facility 

-26:44:12.74 S 

116:39:59.33 E 

Boolardy, WA

Antenna Configuration 

! Inner circle is PWA-4 configuration 

! Outer circle is PGB-8 configuration 

! Shaded circles have radius = 10x elevation (dark is negative)!

Sample Data: 1 Day, 1 Baseline, PGB 

Fringes (Cas,Cyg,Sun)! 

Phase Amplitude (log)! 

Crosstalk 

Intermittent TX 

Beating sources 

Satellite TX 

TV/Aircraft TX

1-D “Delay” Imaging 

XRFI: 

! Remove model sky 

! Statistical thresholding 

! Manual flagging 

DELAY TF: 

! iFFT of passband 

! Convolved by 

“delay beam” 

1D CLEAN: 

! Compensate for 

holes in passband 

! Retain passband 

shape for self-cal

Delay/Delay-Rate 

Transform 

! 1 hour of data with Cas A, Cyg A, Tau A 

! Phase to a source (here, Cas A)! 

! FFT of frequency axis = “Delay Image” 

! FFT of time axis = “Delay/Delay-Rate” 

! Cas A is confined to a region near origin 

! PSF determined by bandpass + time variabliity 

! Can recover bandpass + beam functions

Delay/Delay-Rate Transform 

! Two sources can show up at same delay for a given baseline 

! Over a time interval (say, 1 hr), can be differentiated by fringe rates (see below)! 

! To prevent excessive blurring (changing fringe rate w/ time) best to phase to source before filtering 

! Clean delay axis before iFFT of time axis 

! Clean fringe rate axis to account for flagging of entire integrations 

! Top to bottom: 3 baseline orientations 

! Left to right: delay bins, delay-rate bins, and combined 

filter bins 

! DDR filters have 2 lobes of response: one that tracks 

your source, one that sweeps across the sky

Recovering Bandpass and Beam 

! Top plot: 

Kernels from DDR Transform 

! '+' = uncleaned bandpass recovered 

! heavy dots = cleaned bandpass 

! light dots = autocorr for comparison 

! Bottom plot: 

! '+' = uncleaned delay image 

! line = cleaned delay image 

! Cas A response versus time, Cyg A removed, 143 

MHz (top) and 165 MHz (bottom)! 

! '+' = beam response recovered using delay filter 

! dots = beam response recovered using DDR 

! lines = prediction of beam model

Modeling Beam of Dipole + Flaps 

138 MHz 156 MHz 174 MHz 

40dB zenith to horizon, 60 degree FWHM 

Smooth spatially and vs. frequency

Model Beam vs. Source Strengths 

Black: response toward Cas A 

Blue: response toward Cyg A 

Cyan: response toward Tau A 

Green: response toward Vir A 

Red: response toward Sun

Imaging with Non-Tracking Beams 

! Only possible to weight for one track through primary beam 

! Dirty beam weighted for phase center imperfectly deconvolves away from center 

! Can evade problem with snapshot imaging 

Position-Dependent Weighting in Data Dirty Beam with Incorrect Weighting

138.8-174.0 MHz PWA-4/PGB-8 

! Scale: log(Jy/beam)! 

! black->cyan = -1 to 1 

! yellow >= 2, red >= 3 

! Equatorial Mollweide Projection, RA=0 on right 

! Confusion at 80mJy/beam RMS at positive dec 

! Sun at (1:10,+7:27), removed with DDR filter 

! Cyg,Cas,Vir,Tau peeled with ionospheric-dependent 

solutions. 

Using: 

8 Antennas at pos dec, 4 at negative 

3 days of data, 14s integrations 

W projected, ~15 deg phase centers

! 1 day of 8-element single polarization data 

! Spans 138.8 to 174.0 MHz 

! Model subtraction of Cyg,Cas,Vir,Crab 

! Fit for ionospheric defraction 

! Delay/F subtraction of Sun 

! Map has constant flux scale 

! Changing noise level due to beam response 

150 MHz PGB-8

Tsys = 250K 

# FoV = 0.425 strradians 

$ 0 = 162.5 MHz 

%$ = 35.2 MHz 

PGB-8 Thermal Noise Map 

# bm 

Nant &int RMS Noise Map, log10(Jy/bm)! 

= 2.2e-5 strradians 

= 8 antennas 

= 3 day observing 

= 3.5 hrs/pointing 

Integrations added with alternating signs 

RA,DEC = 12:00, 40:00 

< "T > = 625 mK (10 mJy/bm)!

PGB-8 System Temperature 

Tsys (and predicted synchrotron)! 

Tsky (confusion + synchrotron)! 

Thermal noise (3 day integration)! 

Using RMS pixel values from 4 patches near 

RA=12:00,DEC=40:00 as a function of frequency, we 

determine a perceived Tsky and thermal noise level. 

Backing out integration, this gives us Tsys 

Tsky 

Thermal noise

Power Spectra and Foregrounds 

“Beware the Jabberwock, my son! 

The jaws that bite, the claws that catch!” 

-Lewis Carrol (1872)! 

21 cm EoR (z=9.5)! 

Power spectra at 146.6, 155.7, 164.5, and 173.3 MHz 

(top to bottom w/ error bars), from 4 fields near 

RA=12:00,DEC=40:00. 

Point source dominated, starting to see synchrotron 

at lower frequencies, wavemodes 

Galactic Synchrotron 

Confusion Noise 

Galactic Free-Free 

Extragalactic Free-Free 

21 cm EoR (z=9.5)! 

CMB Background

Summary 

!16 antennas deployed in PGB, 32 in PWA this fall 

!Analog system working well, going into production 

!Packetized Correlator enabling build-out in staged deployments 

! Working on QADC to lower digitizing cost, reducing to 100 MHz BW 

! Lowering integration time to ~8s 

!AIPY-based processing pipeline maturing 

! Available at http://pypi.python.org/pypi/aipy 

! Working on cluster/parallel computing 

!Ionospheric refraction noticeable in data, compensated per source 

!Have a confusion/synchrotron-limited map (8.4K), ready for next tier of foreground removal 

!Thermal noise in map (600 mK) continuing to integrate down 

!Source removal currently limited by primary beam model 

! Using satellites to characterize per-antenna beam shapes 

!Have not started investigating polarization or galactic synchrotron


Need to balance array configuration between resolving and removing confusing sources (long 

baselines), and optimizing for a power spectrum detection (concentrated core). Below, a map with no 

sources removed (left) and with the top 5 sources removed (right), with corresponding pixel histograms.

Ionosphere 

Time scale: 10s Length scale: few km Refraction scale: few arcminute 

Pushes up data rates, need for instantaneous UV coverage (snapshot imaging)! 

Avoid dawn/dusk when ionosphere is highly variable 

Affects degree to which confusion noise can be suppressed

Ionosphere 

Time scale: 10s Length scale: few km Refraction scale: few arcminute 

Pushes up data rates, need for instantaneous UV coverage (snapshot imaging)! 

Avoid dawn/dusk when ionosphere is highly variable 

Affects degree to which confusion noise can be suppressed 

Black: Cas A, Magenta: Cyg A, 3 days of data, nighttime is shaded

38:25:59.24 N 

-79:51:02.1 W 

Green Bank, WV

38:25:59.24 N 

-79:51:02.1 W 

Green Bank, WV

-26:44:12.74 S 

116:39:59.33 E 

Boolardy, WA

-26:44:12.74 S 

116:39:59.33 E 

Boolardy, WA

Radio Frequency Interference 

Solution 1: Discretion is the better part of valor (run away)! 

Solution 2: Statistical thresholding of raw data 

Solution 3: Manually flag contaminated channels 

Solution 4: Delay/Delay-Rate Filtering 

Solution 5: Statistical thresholding after model subtraction 

Boolardy, Australia Site Characterization 

Blue:Peak, Green:Average, Red:Minimum 

Light: PWA, 3 days of data 

Dark: PGB, 3 days of data

Interferometry Fundamentals 

Our Parameterized Measurement Equation: 

Flat Field (Small Angle) Approximation: 

! (u,v) coordinates represent the Fourier transform of the angular sky coordinates (l,m)! 

! Angular power spectrum directly measured in UV (visibility) domain 

! Inverse 2D FFT of gridded UV visibilities yields dirty image (convolved with synthesized 

PSF)! 

! Synthesized PSF is inverse 2D FFT of visibility weights

Comparing with Other Experiments 

MWA PAPER LOFAR 21CMA 

Advantages/Philosophies: 

! Hands on: collect data early, address problems as they are encountered in data 

! Flexibility to adapt to new theory and new knowledge about foregrounds 

! Optimize smoothness of parameter space to allow faster, more precise fitting 

! Wide simultaneous bandwidth, full correlation of large numbers of antennas 

! Both fully functional prototype environment and low RFI location 

Disadvantages: 

! Collecting area 

! Simulation 

Unsure: 

! Spectral resolution vs. bandwidth 

! Group size

Actual 

Primary 

Beam 

Restricted 

Visibilities 

Sky-Vis Relationships 

Sky |Vis| ø(Vis)! 

Implications for 

measuring PS 

Field of view (FoV) 

determines coherence region 

in visibility domain. 

Signal strength depends on 

FoV 

For incoherent signal 

summation, scaling of SNR with 

# samples depends on SNR of 

samples:

Sample UV Optimized for 3 Scales 

Nested Triangular Configuration 

with Outlying Calibrators 

UV coverage Dirty Beam

Slices Through Image 

Virgo A 

Galactic Plane/Her A 

Confusion Sun Confusion 

2008 May 13 Harvard-Smithsonian: 21cm Cosmology 38

Power Spectra 

What an interferometer measures: 

Related measures used in literature:

Sensitivity in Visibility Domain 

Tsky ~ 160K 

#beam ~ 60 deg x 60 deg 

%$ ~ 5 MHz 

#patch ~ 10 deg x 10 deg 

Nsamples ~ 81 baselines 

&s ~ 40 min (varies with patch size and baseline orientation)! 

To hit SNR~1 in a UV pixel: 

' ~ 2 months for 0.5 mK 

Minimum to constrain cosmic variance: 

- 3 sigma detection ~ 9 pixels around circle. 

- Using nested triangles with 3x3 antennas at each virtex, 

~ 64 antennas (being clever). 

- Want to sample multiple rings to select for 

shape of power spectrum, so more like 128 antennas. 

- Need calibrator/imaging antennas, so more like 

160 antennas. 

BUT... 

** UV pixel sampled is frequency dependent, 

undermining ability to subtract foregrounds. 

!^ 

v 

u

What we can learn in first experiments: 

! Time of EoR degenerately constrains 

What We Can Learn 

! Dark Energy/Dark Matter (time for structure to collapse) + star formation rate 

! Confirm inside-out structure formation 

! X-ray heating by first quasars 

What we can learn with SKA-class study: 

! Matter power spectrum through Dark Ages 

! Constrains dark energy, dark matter 

(growth factor in collapse)! 

! Constrains time of radiation-matter 

equality (transfer function)! 

! Geometry of Reionization 

! Star formation 

! Galaxy formation 

! Supermassive black hole formation 

! Feedback of reionization on 

structure formation 

“Richest of all datasets”

Interaction Distance Scale (log)! 

Evolution of Perturbations 

Density Perturbation (log)! 

Inflationary Radiation Dominated Matter Dominated... 

Time (log)! 

Recombination 

Matter-Radiation Equality

Simulated Power Spectra

21 cm Spin Temp 

Wouthuysen Field Effect

Bounds on EoR (6

Raw Sensitivity in Image Domain 

Tsys = Tsky + Trx ~ 170K + 110K 

%$ ~ 1 MHz 

N ~ 256 

Aeff ~ 2 

#obs ~ 1 to 0.1 degrees 

For necessary sensitivity: 

' ~ 1 month for 1 mK at 

1 degree (l ~ 100)! 

or more likely: 

' ~ 10 yrs for 1 mK at 

0.1 degree (l ~ 1000)! 

Galactic Synchrotron 


Galactic Free-Free 

Have added flaps to 

! increase collecting area 

! reduce sky temp 

(z=9.5)! 

! suppress sidelobes of sources 

EoR cm 

! supress RF 21 

Extragalactic Free-Free 

CMB Background

Challenge: Starting from Scratch 

The Problem: 

“Sometimes you can't get started because you can't get started” – Don Backer 

! Self-calibration needs single-source data 

! Source isolation requires calibration 

! Parameter fitting needs a good starting guess 

! How do we get to first base? 

Bootstrapping: 

! Direct, imperfect, iterative 

! Do not rely (excessively) on priors 

! Take advantage of wide bandwidth 

! Address degeneracies one at a time

Polarized Galactic Synchrotron 

150 MHz Power Spectrum Predicted 

from Komolgorov Turbulence 

Faraday Rotation Screen: 

! Smooth with frequency, but improperly calibrated linear 

polarization can produce frequency-dependent structure. 

! Smooth power spectrum can allow further rejection. 

! Need a factor of 1000 suppression 

325 MHz Measured Polarization

Low-Frequency, All-Sky Imaging with the PAPER Array Using AIPY

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?