Wavefront Coding

Computational Imaging:More pixels, less costAndy Harvey

Summary• The role of Computational imaging• Hybrid imaging (wavefront coding)• The quid pro quo• Noise• Artefacts• Optimisation• Assuming artefacts can be removed• Artefacts• Origin and removal• Example optimization applications• Thermal infrared singlet• Ultra-miniature single-moving element zoom• Other computational imaging2

Why computational imaging?• Imaging optics are imperfect• Aberrations• Aliasing and detector space-bandwidth product• Reducing imperfection costs• Complexity• £• Volume• Weight• Logistics• All modern imaging systems involve optics and computation• Combined design and optimisation can enhance trade off and offernew capabilities• Diffraction-limited imaging through aberrations/conformal optics• Imaging in 4π steradians• Lensless imaging….3

Computational imaging: Taxonomy• Wavefront coding• Anti-symmetric phase mask.• Rotationally symmetric phase mask• Multi-aperture imaging systems (incoherent)• Multi-scale lens systems• Coded aperture systems (coherent)• Phase coded and amplitude coded systems.

Hybrid imaging5

The Image recovery problem:loss of information6

The additional parameter space with hybridimagingNoiseFixedparametersTarget metricsLens VariablesMetricsNoiseFixedparametersTarget metricsLens VariablesMask variables*-1Metrics7

Increased depth of field:The good and the uglyConventional imageHybrid imaging: cubicphase profileCarles et al, J. Mod. Opt. in press (2010)8

Effect of defocus when usingsymmetric and antisymmetric masksWithoutphase maskCircularly symmetricModerate Extreme MinordefocusAntisymmetric⎡OH o'(x, y) = I −1 W20 opt⎢⎣⎢H W20( ) H dl( res)⎤⎥⎦⎥Withphase mask9

Artefacts – generalised cubic maskConventional imageRecovered image10

Optimisation of artefact-free imaging(assuming the optimal recovery kernel)11

Phase-function optimisation• Phase function?• Antisymmetric• Pure Cubic• Generalised Cubicθ ( x, y) = α ( x 3 + y 3)θ ( x, y) = α ( x 3 + y 3) + β ( x 2 y + xy 2)• Radially symmetricθ ( x, y) = γ ( ρ 2 + δ ) 2• Phase amplitude: α, β, γ, δ?• Metric: mean square error in image

Through-focus imaging fidelity13

Through-focus imaging fidelityLow defocus toleranceNoise gain / regularizationVettenburg et al Opt. Exp. 18, pp 9221 (2010) 14

Expected imaging error• errorε 2 = E( 2I r− I ) DL= E( 2H wH ab− H DLP ) Sfx, f y+ H W2P Nfx , f yP s• Obtained from images or using a noise modelP Nα 1 fW 20( max) = 5λ15

Generalized cubic image fidelity1[ dB]EMSE16

Relative performance of cubic, purecubic and quartic masksθ ( x, y) = α ( x 3 + y 3) − 3α ( x 2 y + xy 2)θ ( x, y) = γ ( ρ 2 + δ ) 2θ ( x, y) = α ( x 3 + y 3)• Order of preference based on MSE for pure defocus aberrration1. Pure cubic2. Generalised cubic3. Radially symmetric 17

Depiction of image errorθ ( x, y) = α ( x 3 + y 3)( )( )θ ( x, y) = α x 3 + y 3−3α x 2 y + xy 2θ ( x, y) = γ ( ρ 2 + δ ) 2• Order of preference based on MSE for pure defocus aberrration1. Pure cubic & Generalised cubic2. Radially symmetric18

Optimum phase amplitudeα optimum≈ W 20 (max)2β optimum≈ 0α optimum≈ W 20 (max)5β = −3α• So for minimum error in image when W 20 (max)=5λ optimumα is ~2.5λ • Compare previous reports α~15λ• A 6-fold reduction in α ⇒SNR increased by ~3 • But some variation in PSF so kernel should be optimised19

Optimisation of image-recovery kernel20

Artefacts⎡OH o'(x, y) = I −1 W20 opt⎢⎣⎢H W20( ) H dl( res)⎤⎥⎦⎥α=0In-focusDefocusα=5λ21

Origin of artefactsDecomposing the OTF

Decomposition of OTF• Defocus:• For a given spatial frequency,OTF components curl followinga circleH ( f ) = ∫ P( r + f ) P * ( r − f )dr• Cubic phase modulation:• For a given spatial frequency, OTFcomponents depict a Cornu spiral• Defocus unwinds the spiral andproduces no zeros in MTFOTFHnL w 20 20 = = 1ê2 1ê6 1ê3 5ê6 01nOTFOTFReImG. Muyo and A. R. Harvey, Opt. Lett. 30, 2715-2717 (2005)23

Hybrid imaging with a cubic-phase functionperformance parametersCurvature κ of Generalised Cornuspiral=+ ( ) κπν 16 wx203αNote that AF∞*AFy (,)(/2)(/2)exp(2) νννπ =+ Pu Pu − j uydu∫−∞replaced by linear relationship• Explanations andexpressions are obtainedfor• Mean value of MTF• W 20 (max)=3α(1-f)• Cf optimisation• Amplitude and phasemodulation of the OTFΔMTFΔMTFΔӨNet rotation of spiral removed24

The origin of image replicationsOTF Kernel SystemRecovered Original imageimageW 20 (optical)=2λW 20 (kernel) =0 =4λ =3λ =1λ =2λΔθ(v) = 4πv(W 220,0− W 2 20)+3α⎛3α sin ⎡4πv(ΔW 20) 2 / (3α) ⎤⎜ ⎣⎦−2π v ⎜ ΔW⎝20I ' res(v) ≈∞∑n=−∞⎛⎝⎜∞∑m=−∞J nsin ⎡4πv(ΔW 20,0) 2 / (3α)⎣ΔW 20,0⎞( A )J m ( C)exp ⎡⎣i(nB − mD)v⎤⎦I⎠⎟ diff(v)⎤ ⎞⎦ ⎟⎟⎠M. Demenikov & A R Harvey Optics Express 18, pp 8210 (2010)25

Removal of artefacts

Parametric blind-deconvolution:Calculation of optical W 20∇(i(W 20 o )) = MAD{HL 2,1(G(i(W 20 o )))}+ MAD{LH 2,1(G(i(W 20 o )))}MAD: Median Absolute DeviationG(i): Gaussian filter applied to restored image i27

Determination of optical W 20§ Artefact metric, ∇(i(W o 20)) minima when the restoration kernelmatches the imaging kernel:1SNR= 80 60 40 dB0.31575lNormalizedartefact metric0.80.60.41.39658l1.80218l2.76166l0.24.30275l0D0 1 2 3 4 5Defocus W 20 @l§ |W 20 -W 20 W 20(W 20= 4.30275λ)|

Accuracy of optical W 20DError in W 20 D @l (λ)1.0000.5000.1000.0500.0100.005áìçàæòôíóæçôæòíàìáóæàìçòôíæ ç íæíá æóô æòôàìáòçáæóìàóôíçáôæ òæòáíçæìàìàóDó10 20 30 40 50 60 70 80SNR SNR (dB) @dBôòçæáæíìàóæàìòôçáíóæBoatCameramanLenaManMandrillPlasticSpokeStrawsUS airforce testAverage• Optical defocus can be determined with sufficient accuracy• Low levels at high SNR• Masked by noise at low SNR• Optimisation of α is possible29

Application:Lens simplification for cost reductionand weight reduction

Simplification of LWIR F/1 f=75mm Geoptical system• From TWO ELEMENTS• Field flattener• Diffraction-limited• To SINGLET• Corrected for coma and sphericalaberration• Off-axis performance limited by• Field curvature• Astigmatism31142mm78mm

Correction of aberrations across FoV• 10 waves of aberrations• FoV of 9º• Presence of zeros in the MTFs• Large field curvature and astigmatism32

Pure and Generalised Cubic SolutionsPure CubicGeneralised cubic• Generalised cubic higher MTF for compensation ofoff-axis astigmatism and field curvature33

PSFs across FoV34

IR singlet and phase maskDesignManufactured35

Correction of field curvature, astigmatismand defocus• Singlet only• 150 300 Optimal mm defocus• Decoded Wavefront-coded WC imageimage• 150 300 Best mm focusdefocusG Muyo, A Singh, M Andersson D Huckridge,A Wood and A R Harvey, Opt. Exp., 23, 21118 (2009)36

And as a movie37

Miniaturisation of zoom lenses

Single-moving-element zoom lenses:the problem• Conventional zoom lenses have atleast two moving elements:• A zoom element• A compensator• Single-moving element zoom lensescannot be miniaturised due toexcessive defocus:• Hybrid imaging can mitigatedefocus….10mm39 39

• M. Demenikov, E. Findlay, and A. R. Harvey, “Miniaturization of zoom lenses with a single moving element,” Opt. Express 17, 6118-6127 (2009).SME lens: conventional and hybridConventionalCoded• Hybrid imaging enables a ten-fold reduction in length of SME zoom (now 1000xshorter than with 35mm film), or• Enabled two moving elements to be reduced to one, but…• SNR reduced by 13.6dB• Constant PSF and no aliasing reduce SNRG. Muyo and A. R. Harvey, Opt. A: Pure Appl. Opt. 11, (2009)• M. Demenikov, E. Findlay, and A. R. Harvey, Opt. Express 17,6118-6127 (2009).40

No Phase maskGeneralised CubicPure Cubic41

Conclusions• Hybrid imaging offers an additional parameter for optimisationof imaging systems• Reduced cost• Reduced weight and complexity• Miniaturisation• Optimal phase-modulation• Generalised cubic, cubic, quartic• It has been common to use strong optical coding• Use of weak coding with adaptive image recovery yields optimalSNR• Significant additional demands on• Adaptive recovery algorithms• Artefacts removal• Computational power42