A Tutorial on Wavelets from an Electrical Engineering ... - IEEE Xplore

A <strong>Tutorial</strong> on Wavelets from an ElectricalEngineering Perspective, Part 1:Discrete Wavelet Techniques’T. K. Sarkar’, C. Su’, R. Adve’, M. Salazar-Palma’, L.Garcia-Castillo’, Rafael R. B og‘Department of Electrical and Computer EngineeringSyracuse University121 Link HallSyracuse, New York 13244-1240 USATel: +1 (315) 443-3775Fax; +1 (315) 443-4441E-mail: tl

One of the goals of this paper is to illustrate how one can generatethe scaling functions and the wavelets, specially tailored to one'sneeds.Consider a signal, x(t), the Fourier transform of which isX(w). In this paper, we deal only with discrete wavelet techniques.Discrete wavelet techniques are quite suitable for discretesignal processing, for example, in speech and image processing. Inparticular, their applications are very desirable in data compression.Since a complex matrix is a two-dimensional system, thesolution of a matrix equation may be posed as an image-analysisproblem. As we shall observe, discrete wavelet techniques may besuitable for the solution of large complex matrix equations.Continuous techniques, on the other hand, may be suitablefor time-domain processing, where the wavelet transform can beinterpreted as a windowed Fourier transform. This we shall dealwith in the second part of the paper.3. Development of the discrete wavelet methodology fromfilter-theory concepts3.1 PreliminariesConsider the signal ~(t) that is discrete, so that it is representedby the sequencex( .): Iz = 0,1,2,. . . (1)Then, its Fourier transform is best handled by the z transform (thelowercase letters are for functions in the original domain, and theuppercase letters are used for the z transform):where w=2$ is the angular frequency, and z=eJo =eJ2~Consider the sampled signal x(n) to be bandlimited, and assume itis sampled (say) at f = 1Hz. Thus, the sampling interval isAt = 1 sec. From the Nyquist sampling criterion, it is then necessaryfor the signal x(n) to be bandlimited to 112 Hz, so that it canbe sampled at two times its bandwidth at the Nyquist frequencywithout aliasing. The bandwidth of the signal (in angular fieqUency),mband > isand the sampling angular frequency, w,a,,p, is= 2z. (4)Now, let us filter the signal .(a) by a low-pass filter, H(z), ofbandwidth 7112 [Le., 0 5 w 5 z 121, and by a high-pass filter, G(z) ,of bandwidth w = n 12 to z. Let u'(n) be the low-pass-filteredsignal [Le., ~'(n) has been obtained by passing x(n) through thelow-pass filter h(n) 1, and let ~'(n) be the high-pass-filtered signal[i.e., ~'(n) has been obtained by passing x(n) through the highpassfilter g(n)]. This is shown in Figure 1. Now, since the bandwidthof the signals ~ '(n) and ~'(n) has been reduced by a factorTransmitterReceiverFigure 1. The principles ok sub-band filtering.of two, these signals can be decimated by a factor of two withoutany aliasing. This is equivalent to reducing the sampling rate.Decimation or down-sampling by a factor of two implies thataltemate samples are dropped, and the data are compressed, asshown in Appendix 1 (Section 8). The purpose of decimation is toreduce the sampling rate and, thereby, the bandwidth of the signal.This down-sampling is possible because .I(.) and ~'(n) have aneffective bandwidth of f = 1 I4 or w = z 12, because they havebeen filtered.Next, both ~'(n) and ~'(n) are down-sampled by a factor oftwo, resulting in u(n) and v(n). This sub-sampling can continuefurther, as we shall see later on. Here, we will restrict ourselves tothe two stages of filtering of x(n) by h(n) and g(n), for illustrationpurposes. Since both u(n) and v(n) have a smaller bandwidth,they can be easily sampled, quantized, coded, and transmitted.In the receiver, the quantized, sampled, and coded .(.) andv(n) are received. The down-sampled versions, .(a) and ~(n),can be transmitted at a much lower bit rate than the original signal,without any loss of information, as they have a smaller bandwidththan the original signal. The problem is how to reconstruct the signalx(n) back again, given the sub-sampled versions u(n) andv(n) of x(n), without any aliasing or distortion.Using this type of sub-band splitting has many advantages:1. This methodology results in a "maximally decimated" signal(Le., some of the sample values of the signal have been deletedor set to zero), namely, the sampling rate can be reduced, withoutany loss of information. [Note that v(n) of x(n) have a lowerbandwidth than the original signal x(n) and, hence, the samplerate has been reduced by a factor of two]. This is equivalent tosaying that u(n) and v(n) have been decimated by a factor of two.2. Even if .(E) and v(n) are quantized in a very rough fashion(say, quantized into two bits, rather than the conventional eightbits or 16 bits), then the reconstruction is really remarkable, eventhough such large quantization errors are introduced in the codingof u(n) and v(n) [3-81. This has been shown, at least in the area ofimage processing. We will also demonstrate that this happens inthe compression of matrices arising in electromagnetic-field problems.Our interest in this is due to the fact that a matrix is essentiallya discretized version of an image. So, can we apply thismethodology to the efficient solution of operator equations? Wewill address this issue later on.3. Mother Nature, in many cases, performs processing in sucha way. For example, human ears and eyes, at least in the first stageof decoding sound and sight, perform processing by constant-Qfilters that are very closely related to wavelets.50 /€€E Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998

For example, Daubechies [ 11 originally developed this methodologyby constraining the filter N(z) to be smooth, by enforcingall derivatives to be zero at w = 0, up to order p, or, equivalently,G‘P(l) = 0, where the superscript p denotes the pth derivative ofG’ . The actual value for p is determined from the number of equationsneeded to solve for the values of h(n) , n = OJ,. ..,A’. Thisleads to taking the various moments of g’(n) and setting themequal to zero. Daubechies chose this procedure because enforcingthe above conditions guarantees smooth wavelets, which we willdefine later. However, for the present case, where the number ofdiscrete wavelet coefficients is finite, the smoothness of the waveletsis a moot point. This is true since for the discrete case, thewavelet transform can be implemented (for the examples that weare interested in) without explicitly specifying the wavelets. Thesmoothness of the wavelets at this point, then, is of no concern tous! Another approach, for example, is given in [lo, 111.In mathematical terms, setting the derivative of G’(1) to zeroleads to-Oh(O) - Ih(1) + 2h(2) - 3h(3) = 0 . (37)Solution of Equations (30), (35), (36), and (37) leads toI+&h‘(0) = h(3) = __4Jz ’3+&h‘(l) = h(2) = __4Jz ’3-Ah’(2) = h(l) = __4Jz ’1-43h‘(3) = h(0) = __44‘5The magnitude and the phase responses of H(z), H‘(z) , G(z) ,G‘(z) are shown in Figures 4a and 4b, respectively. Note thatthese filters have no ripples, with many zeros at n. The slope at the-150 --200 I0 0.5 1 1.5 2 2.5 3 3.5-wFigure 4b. The phase responses of the fourth-order filters.center is proportional to fi . The transition from jH(z)\ = 0.98&to jH(z)l= 0.02&IIis over an interval of length 4/&.Instead of having a two-stage decomposition of the signalx(n) into ~’(n) and ~’(n) , one can perform a multistage decompositionby applying the filters successively to each stage of thedecomposition, as shown in Figure 5. The electrical-filter-theoryequivalent is shown in Figure 6. This is the decomposition part (orthe part labeled “transmitter” in Figure I). The coefficients at theoutput of Figure 5 are then thresholded. This is equivalent tokeeping only those coefficients that are bigger in magnitude thansome constant E. The values smaller than E are set equal to zero,resulting in the approximate filter output. Now, the interesting partis that these “filtered” thresholded coefficients can now be used torecover the original signal with an accuracy better than E (!). (Wewill see this feature later on). The reconstruction algorithm isdepicted in Figure 7, where the approximated coefficients are upsampled,and then filtered the way that is depicted in the receiverof Figure 1.The type of decomposition outlined in Figure 5 is identical toa wavelet decomposition, as the next section will illustrate. Forexample, filtering a function by h(n) is equivalent to fitting ascaling function at a certain scale, and filtering by g(n) is equivalentto curve fitting x(n) by wavelets at the same scale as thescaling function. The mathematical connection is now establishedbetween wavelet theory and filter theory.3.2 Connection between the filter theory and the mathematicaltheory of wavelets. -0 0.5 1 1.5 2 2.5 3 3.5Figure 4a. The magnitude responses of the fourth-order filters.-wTo establish a connection between the filter theory and themathematical theory of wavelets, we will deal with functions of areal continuous variable. Discrete samples then will be viewed as a“sample and hold” of the these functions of a continuous variable.Such an approach is absolutely necessary, because the waveletscannot be expressed in terms of discrete samples. Moreover, thedilation equation, which is at the heart of wavelet theory, has nosolutions for the discrete samples. So, the discrete wavelet transformimplies that the functions we are dealing with are functions of/€€€Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998 53

and these are the discrete wavelet coefficients of the function ~ (t) .These are the same values shown in Figure 5. Our objective in thissection is to establish that isomorphism.Now, if we have to carry out the inner products in Equation(59), it will be extremely time consuming, as the inner productshave to be carried out for all values of n and k. Here, thestrength of the wavelet techniques comes in, as they provide a fastand accurate way to recursively evaluate the inner products. This isaccomplished through the introduction of the scaling functionsand from Equation (45),It is interesting to note that a byproduct of Equations (66) and (67)is that@l,j(t) = 2-"24(2-'t - j). (60)We further assumeiutilize the orthogonality relationships betweenthe scaling functions and the wavelets, and between the scalingfunctions themselves, i.e.,which can be generalized toandWe now define the coefficients qn throughIt is now shown how the dk,n are evaluated recursively throughthe Ck,n.We have, from Equations (58), (61), and (62), that the followingorthonormal setNext, observe thatandmm2(n) = jp(t)@(t - n)dt = j p(t)C hF(t)&@(2t - 2n - k)dt-m -m k(68)= Ch'(k)a(-')(2n+k)=Co(-l)(k)h'(k-2n),kkbo(.)= Ca(-l)(k)g'(k-2?2).ICGiven the existence of relationships like Equations (68) and (69),and drawing the isomorphism betweenrepresents a basis, as the functions involved in the set are orthogonal.From the dilation equation, Equation (41), and from Equation(45), we note that(&5(2t- k)Im k=-malso form an orthonormal set for the same space. This is for scalen = -1, as defined by Equation (56). Therefore, we can expand anyfunction p(t) bywe can generalize the expressions of Equations (68) and (69) toc ~ , = ~ E~,,~h'(n--2k)=+ ~Ccn,,h(N+2k-n) for j2-1, (70a)nnHere, u(-')(k) are the coefficients used to represent the functionp(t) by @(2t - k) . The superscriptsj on the coefficients d (k) andbj(k) represent the value of the scale j at which they arerepresented. We have from Equation (41),These results have been derived utilizing Equations (15) and (16).The above recursive relations show that we need to compute theinner product of Equation (63) at the highest scale, n = -1, onlyonce (instead of using Equation (59)), and then the wavelet coefficientsdk,n of the XDwr(n, k) ( n = 0,1,2,. ..) are computed recursivelyfrom Equations (70a) and (70b). From a filter-theory pointof view, Equations (70a) and (70b) show that the c,,~ need to beconvolved with h(n) and g(n) , then down-sampled by a factor oftwo. Through Figure 5 and from the above development, it is clearthat for the computation of the discrete wavelet transform (DWT),/€E€ Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998 57

it is not necessary to even know what the scaling functions andwavelets are, as one can directly use Equations (70a) and (70b)without going through the mathematical derivations, as Figures 5and 6 illustrate. One starts with c~,-~, which is the coefficient generatedby correlating 4(2t) with the function ~ (t) , and then recursivelycomputes the discrete wavelet transform mathematicallythrough Equations (70a) and (70b), and graphically using Figure 5,which is easier to visualize from a filter-theory perspective. Themethodology is the same. The process described so far is similar tothe transmitter part as labeled in Figure 1. If the 4(2t) are theimpulse scaling functions, then q - 1 will be equivalent to thesampled version of ~ (t), namely x(n).There is another subtle point that one should introduce now!So far, in the approximation in Equation (55), the limits are infinity.This is good from a mathematical perspective. However, froma practical reality, the limits have to be finite. Hence, in addition todk,n, we also have c~,,~. The approximation of ~(t) from a practicalstandpoint is done byHence, we have both wavelets and scaling functions in theapproximation. This very important feature is generally not clearlydelineated in many presentations. Now, observe that Equation (71)is the representation of ~ (t) .Figure 9b. A compressed version of the picture of “Lena,”utilizing QMF compression (40:l compression).In summary, the evaluation of the discrete wavelet coefficientsin Equation (55) is equivalent to filtering the coefficientc~,-~ obtained from ~(t) (or, equivalently, the sampled values of~(t) for a certain class of scaling functions) by a cascade of mutuallyorthogonal filters, as shown in Figure 5. The bandwidth of thefilters is reduced by a factor of two as one goes towards the DCvalue. For the infinite sum in Equation (59, the scaling functiondoes not enter into the final sum, except in the intermediate com-Figure 9c. A compressed version of the picture of “Lena,” utilizingJPEG compression (40:l compression).putations. However, if the sum is finite in Equation (55), then oneobtains Equation (71), and the scaling functions are needed in thesummation.Figure 9a. The original picture of “Lena.”In practice, once the coefficients d,c,n and qm have beenobtained, those coefficients with magnitudes below a certainthreshold value of E (eg, E = are set equal to zero. At thispoint, the transmitter then sends out the approximate coefficients-dk,n and ck,M, which are the values obtained after thresholdingdk,n and qm by E. Now the problem is, how does one recover~(t) from these approximate coefficients in a fast efficient way?58 IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998

asic philosophy of the two methodologies is the same: namely, toobtain a sparse complex matrix instead of a full one, as resultsutilizing MOM and sub-sectional basis functions. However, theadvantage of forming a sparse matrix is offset by the problem thatthe condition number of the transformed matrix may be worse.This is particularly significant for the second method, where thetransformation utilizing a wavelet-like basis, from a complex-fullmatrix to a sparse-complex matrix, is orthogonal, from a strictlymathematical point of view. However, from a purely numericalperspective, as will be shown later, the results do not confirm thisorthogonal transformation, as the condition number of the systemchanges and, in some cases, the change is by an order of magnitude.5. 1 Application of wavelet basis for the solution of operatorequationsIn [22], a Galerkin method, utilizing a wavelet basis, hasbeen used for the analysis of radiation for TM-scattering problems.There, a method is also proposed to compress the impedancematrix, utilizing wavelet techniques. This is accomplished througha compression process in which only the significant terms in theexpansion of the (yet unknown) current are retained and subsequentlyderived.Figure lob. A compressed version of the simulated halftonepicture of Figure loa, utilizing QMF 30:l compression.Wavelet bases have also been used to solve the Fredholmintegral equation of the first kind, which leads to the TM-scatteringfrom conducting cylinders [23]. Here, the compactly supportedsemi-orthogonal bases have been used. The wavelets have beenspecially constructed for the bounded interval for solving first-kindintegral equations. It has been observed that the use of cubic-splinewavelets almost diagonalizes the matrix. Explicit closed-formpolynomial representations for the scaling functions and waveletsare given.In [24], the wavelet-expansion method, in combination withthe boundary-element method (BEM), has been used to solve theintegral equation for the surface currents. The unknown current isFigure 10c. A compressed version of the simulated halftonepicture of Figure loa, utilizing JPEG 30:l compression.expanded in terms of a basis derived from a periodic, orthogonalwavelet in a finite interval. Because the geometrical representationof the BEM is employed to establish the mapping between thecurved computational domain and the interval [0,1], it would bevery interesting to find out how this can be extended to threedimensionalproblems, where an arbitrary surface needs to be discretized.Figure loa. The original, simulated halftone picture printed ona 400 dpi digital printer.An adaptive multi-scale Moment Method is presented in [25], forthe solution of the Fredholm integral equation of the first kind. Anadaptive procedure is outlined, which refines the unknown solutionin regions utilizing multi-scale wavelet-like functions. Even though68 IEEEAntennas and Propagation Magazine, Vol. 40, No. 5, October 1998

the matrix is highly sparse, the deterioration in the condition numberof the matrix-as opposed to the condition number of the originalmatrix utilizing a conventional sub-sectional basis-is clearlyvisible. One disturbing fact is that the results are not consistentwith the increase of the order of the filter, nor with the level ofthresholding used. The methodology can also be used for solutionof the differential form of Maxwell’s equations, utilizing finiteelements [26]. Similar wavelet-like functions can also be used tocompress the elements of the impedance matrix, utilizing a thresholdingoperation.In [27], the problem of electromagnetic scattering from perfectlyconducting strips, coated with thin dielectric material, isanalyzed, utilizing an adaptive multi-scale Moment Method. Themethod of a non-uniform grid and the multi-scale technique, whichgenerates a locally finer grid, are usually used when the solutionsof the integral equations or the differential equations are known tovary widely in different domains. By non-uniform gridding, onecan reduce the size of the problem and improve the accuracy. Themulti-level or the multi-grid technique has been widely used insolving differential equations and integral equations [28-321.Kalbasi and Demarest [33, 361 applied multilevel concepts to solvethe integral equation by the Moment Method on different levels,which has been called the Multilevel Moment Method. No matterwhat the multi-grid technique is, the basis functions for animproved approximation have to be reconstructed again. By usingthe multi-scale technique in one dimension, the basis functions forthe new scale have to be reconstructed. The new approximationgrids formed by the multi-scale technique are the same as those forthe multilevel technique; however, the constructions for the functionsare different.In addition to these numerical concems, there are some philosophicalconcems that need to be addressed in the selection of anappropriate set of basis functions. Consider the solution of anoperator equationAX=Y, (75)where, in general, A is a known integro-differential operator, andX is the unknown to be solved for a given excitation, Y. For thesolution of boundary-value problems, the most widely used techniques,like MOM, FEM, and BEM, convert the functional equationto a matrix equation. The matrix equation is then solved forsome unknown constants, instead of obtaining the solution in termsof unknown functions. The solution procedure starts by expandingthe unknown, X, in terms of known basis functions e,(t), withsome unknown constant multipliers a, in front, i.e.,N~(t) z Calei(t).,=IWe then substitute Equation (76) into Equation (75), and form theerror or the residual (R), defined byThen, the objective is to make the residuals zero with respect tosome weighting functions, Wj. From Equation (77), it is quiteclear that Aej must approximate Y in some sense. Hence, it isrequired that Aei must he linearly independent, and must form acomplete set. It is immaterial whether the ei are linearly independ-ent, or even orthogonal! This point was illustrated by [13-171,through the solution of the following differential equation:-=sinx+2d2Ydx2for 01x

finite elements has been illustrated. Initial results illustrate how touse the wavelet techniques to generate a sparse matrix, utilizing awavelet-like basis. A similar methodology can also be used inmaking sparse matrices-arising in the implementation of finiteelements in the solution of differential forms of Maxwell’s equations-stillsparser.In the next Section 5.2, how to use the discrete wavelet techniquesto transform a complex dense matrix, arising in conventionalMOM, into a sparse matrix, utilizing a set of orthogonaltransformations, is illustrated. Some work [35-381 has already beendone to address this topic. This is described next.5.2 Solution of large matrix equations by the discrete wavelettransformConsider the solution of a matrix equation [A][X]=[Y],where [A] is a known Q x Q matrix, [Y] is a known Q x 1 vector,and [XI is the unknown to be solved for. Note that Q has to be aninteger power of two for the wavelet techniques to be applicable. Ifthe original matrix [A] is not of size 2m, then the matrix [A] canbe augmented by a diagonal identity matrix to make it 2m. First,we illustrate how to obtain the wavelet transform of a vector [Y],and then we will illustrate how to take the wavelet transforms of amatrix [A]. We first select h‘(n). We then use h’(n) and g‘(n) tofind the discrete wavelet coefficients, utilizing Equations (70a) and(70b). What is going to be different is that we are going to expressEquations (70a) and (70b) as circular correlations with respect toh’(n) and gf(n), or as circular convolutions with respect to h(n)and g(n) , respectively. This is a computationally efficient way tocarry out convolution by utilizing the FFT. As an example, we firstcreate the following 8 x 8 orthogonal matrix [PI asg‘(n) (see Equation (17) for the relationship between h‘(n) andg‘(n)). Therefore, multiplying a vector (say [Y]) by [PI isequivalent to filtering (in the transmitted portion) in Figure 1, followedby a sub-sampling of two. The matrix-vector product yieldsu(n) and v(n) . The first four elements are equivalent to u(n) andthe last four are v(n). The matnx-vector product has alreadyincorporated the sub-sampling by a factor of two. The sub-samplingby a factor of two is accomplished by the shift between theelements of each row of the matrix [PI. Now, for [PI to be anorthogonal matrix, it is necessary that the following three equations,similar to Equation (29) [the normalization constant is set tounity] and Equation (30)[the filter is orthogonal to its two-shiftedversion] hold:Ch2(i) = 1,1h(O)h(2) + h(l)h(3) + h(2)h(4) + h(3)h(5) = 0, (83)h(O)h(4) + h(l)h(5) = 0. (84)Finally, from the boundary conditions for the filter,H(z =I)=& and G(z =1)= 0, we have1h(0) + h(2) + h(4) = - = h(1) + h(3) + h(5) .Jz(85)Equations (82)-(85) provide four independent equations, and oneneeds two more to uniquely solve for the h(i)s. If one followsDaubechies’ procedure for making the wavelets smooth, one wouldneed the derivatives of G’P(z = 1) = 0 for p = 1 and p = 2, leadingto (from Equation (18))-0-h(O)+lh(l)+2h(2)+3h(3)-4h(4)+5h(5)= 0, (86)I0J0 -h, +h, -h2 +h3 4 4 +h5-h4 +A5 0 0 -h, +h, -h, +h3-h2 +h3 -h4 +hs 0 0 -h, +hi[In Equation (81), we have used hk to represent h(k) , to conservespace]. This equation is the matrix form of the discrete wavelettransform of Equations (70a) and (70b). Inside [PI we have thefilter coefficients, six in number. Please note that the first fourrows are due to the filters h’(n) , and the last four rows are due to-O*h(O)+lh(l)-4h(2)+9h(3)-16h(4)+25h(5) = 0. (87)The setting of the higher-order moments of g’(n) to zero guaranteesthe smoothness of the wavelets. This, in tun, provides a recipeso that the discrete-wavelet coefficients of the transform drop offrapidly, as one goes to the dilated scales from a fine scale. Thenumber of zeros of G’P(z) at z = 1 tells us how many basis functionsare needed in Equation (55) for approximating ~(t). Thesmoother the function and the higher the order of zeros, the fasterthe expansion coefficients go to zero, and the fewer coefficients weneed to keep. For piecewise functions, a wavelet basis is better.These piecewise functions may have jumps. They may be smoothand then suddenly rough. We keep more Coefficients in the roughneighborhoods by going to a smaller scale TJ. The mesh adaptsto ~ (t) in a way that Fourier methodology finds difficult.If x(t) has p derivatives, its wavelet coefficients decay like2-np [38]:where J is a constant, and &’)(t) represents the pth derivative of4) ’62 IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998

dimensional discrete wavelet transform is applied to the systemmatrix [A], the resulting system matrix will be sparse if all theelements below a threshold are set to zero. Typically, one would(:Ihave only 10Qloglo elements in the sparse system, where Q isthe size of the matrix and E is the truncation level, i.e. elements ofthe resultant matrix whose absolute value is less than E will bediscarded. So, for a 2048x 2048 matrix, the resultant systemmatrix would be sparse by a factor of about 30 [20].Consider a real matrix [A], of dimension Q x Q, where Q islarge. Let the elements of [A] be defined by( 1 if i=jOrder YO ofof Filter Nonzero~ + 1 Elements4 7.58%8 6.28%16 6.41%32 6.54%Error inReconstruction6Cond[B,],58 10-4 4.66 x 106,54 10-4 5.56 x lo6,52x10-4 5.74 107,5iX10-4 1.57 107We now apply a wavelet transform to the matrix [A]. This isequivalent to pre- and post-multiplying [A] by a number oforthogonal matrices. Let [SI be the product of the orthogonalmatrices [Pi], as explained earlier for the one-dimensional discretewavelet transform explained by Figure 11 : Order YO of Error inof Filter Nonzero ReconstructionCond[B,]Even though the sizes of the various [Pl]s are not the same, wemake them the same by supplementing, say, [P2] by a diagonalunity matrix, [I], to make it the same size as [P,]. Thus,(92)Since the product of all orthogonal matrices is an orthogonalmatrix, [SI is an orthogonal matrix. When the wavelet transform isapplied to the system of equations [A][X] = [Y], one obtains(93)where T denotes the transpose of a matrix. Since [SI is anorthogonal matrix, we havewhere [I] is the identity matrix. So, we take the wavelet transformof [XI and [Y] to form [X'] and [Y'], and we take the wavelettransform of [A] to form [B] . Hence, Equation (93) reduces to[B][X'] = [Y'] with [B] = [S][A][S]T. (95)The unknown [XI is solved for fi-om this byrows and columns, which is similar to carrying out a two-dimensionalFFT.Now, we consider [A] to be of the form in Equation (go),and choose various order filters (i.e., M = N + 1) for h(n) , with 4,8, 16, or 32 terms. Next, we compute [B] = [S][A][SIT ([SI beinggiven by Equation (91)), and then apply a threshold to the elementsof [B] to obtain matrix [B,]. The matrix [B,] is an extremelysparse matrix. For example, if the threshold is set at and thesize of [A] is Q = 5 12, and if we then apply a fourth-order filter(Le., M = N+1= 4) h(n) , we find that only 7.58% of the ele-ments of [Ba] are nonzero (see Table 1).Next, take that sparse matrix [Ba], and try to reconstruct [A]by computingDefine an average value, 6, of the error between the elements of[Aal and [AI by[XI = [sIT[xr]. (96)[B] is the two-dimensional wavelet transform of [A], and hasbeen computed by a series of one-dimensional transforms to itsFrom Tables 1-6, it is seen that the matrix [A] can be recoveredfrom [B,] to provide [A,] with an average error that is lower than64 IEEEAntennas and Propagation Magazine, Vol. 40, No. 5, October 1998

Table 3. The sparseness, error in reconstruction, and conditionnumber of the two-dimensional wavelet transform of a matrix[A] of the form in Equation (go), as a function of the order ofthe filter, with a threshold of 0.001, Q = 2048, and the condi-tion number of [A] given by Cond[A] = 6.80 x 10’.Error in Cond[B,][-:o 1 Reconstruction 1Table 4. The sparseness, error in reconstruction, and conditionnumber of the two-dimensional wavelet transform of a matrix[A] of the form in Equation (90), as a function of the order ofthe filter, with a threshold of 0.0001, Q = 512, and the condi-tion number of [A] given by Cond[A] = 4.45 x 10‘.ElementsTable 5. The sparseness, error in reconstruction, and conditionnumber of the two-dimensional wavelet transform of a matrix[A] of the form in Equation (90), as a function of the order ofthe filter, with a threshold of 0.0001, Q = 1024, and the condi-tion number of [A] given by Cond[A] = 3.02 x lo7.Error in Cond[B,]=:io I Reconstruction 1the value of the threshold used to eliminate the wavelet coefficientsof [A], (or, equivalently, the elements of matrix [B] ). The numberof [P,] matrices (see Equation (92)) used for obtaining the resultsappearing in Tables 1-6 is INT [ log2 [21)]__ (LNT stands for “theinteger part of’). Please note that the wavelet transform of a realfunction is always real.The other interesting property to note is that utilizing ahigher-order filter does not necessarily produce larger sparsity, asshown in Tables 2,4, and 5.The computational time in the computation of the wavelettransforms in the compression of a matrix is now investigated. Weconsider a filter of length L = N + 1, and the data matrix is oflength Q. Then the one-dimensional wavelet transform of [Y] maybe done (we carry out the initial product at the highest stage ofresolution and then down-sample) by the following number ofmathematical operations:QL 1+-+-+ ... =2QL(99)G:)To carry out the two-dimensional wavelet transform of [A], werequire (2QL)* operations. Therefore, to produce the sparse systemof Equation (95), we require 4Q2L2 + 2QL operations, resultingin a matrix [B] that contains, at most, of the order of O(Q)elements. If we now apply the conjugate-gradient method to solvethe sparse system, per iteration we will require O(2Q) multiplicationsto carry out two matrix-vector products. Even if the conjugate-gradientmethod converges in at most Q steps (where Q is thenumber of unknowns), then we have solved Equation (95) in anoperation count of Q0(2Q), in addition to 4Q2L2 + 2QL opera-tions. Observe that this O( Q2) is significantly lower than the con-ventional Q3/3 operations typically required in a solution of amatrix equation of size Q. This is essentially the contribution ofBeylkin, Coifman, and Rokhlin [18].However, it is interesting to note that the nature of the varia-Table 6. The sparseness, error in reconstruction, and conditionnumber of the two-dimensional wavelet transform of a matrix[A] of the form in Equation (90), as a function of the order ofthe filter, with a threshold of 0.0001, Q = 2048, and the condi-tion number of [A] given by Cond[A] = 6.80 x lo7.ElementsError inCond[B,]tion ____ may also be the result from a convolution. In thatli - jl“case, the FFT would be much faster than the discrete wavelet transform,as the FFT essentially diagonalizes the operatorimatrix.However, for other cases, even when the variation is not due to aconvolution, the wavelet result still holds.The most disturbing fact about the discrete wavelet transformis that the condition number of the matrix changes after the transform.It has been shown earlier that the wavelet transform of thematrix [A], which is [B] , has been formed through a series oforthogonal transformations. Therefore, by definition, the conditionnumber of the matrix should not change as one goes from [A] to[B] . However, as is clear from the six tables, there is a change inthe condition number of the transformed matrix when one uses adifferent-order filter. Also, the condition number is dependent onthe threshold used to truncate the elements. There appears to be alack of systematic change in the results. In this case, the matrix[A] is real.IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998 65

As a final example, consider the electromagnetic scatteringfrom an array of wires, randomly spaced. We considered 56 thinwireantennas. Six were of length 2.7A and radius 0.0051. Theremaining 50 were 32 long and of the same radius. The 56 wireswere located inside a parallelepiped of dimensions271xX5/2x21/2. The usual MOM application led to a2096 x 2096 matrix. The matrix was compressed, utilizing a filterh(n) of length 16. The compression for the real part of the impedancematrix was 17.8%, Le. 749289 of the elements of the matrixwere above a threshold of For the imaginary part of theimpedance matrix, only 3.28% of the elements were nonzero. Thissparse impedance matrix was then used in a conjugate-gradientroutine, to solve the transformed Equation (93). Convergence inthe residuals of was obtained in 95 iterations. This simpleexample demonstrates the potential for the solution of large matrixequations.There are a few points that are worth mentioning. First of all,if the compression was not done on the real and the imaginaryparts of the matrix separately, then the degree of compression wasmerely 35%, as opposed to 17.8%. This is significant. Secondly,the size of the impedance matrix has to be a power of two. Thirdly,the conjugate-gradient method takes the same number of iterations(95) to converge to the solution when applied to the original densematrix, or to the sparse matrix, as the transformation is presumablyorthogonal, from a strictly theoretical point of view. However, nowas the entire compressed matrix is in memory, the number of pagefaults is small, and so the result can be obtained quite efficiently.Note that one of the disadvantages with this procedure is thatthe size of the matrix Q has to be a power of two for efficientimplementation of the wavelet transform.cx (4YD(4Figure 12. The symbol used to represent decimation by a factorof two.0 14 >36. ConclusionThe discrete wavelet transform has been presented from firstprinciples, utilizing the basic concepts of filter theory. It has beenshown how to construct the filters h(m) that produce the waveletsand the scaling functions. However, for the discrete case, the introductionof wavelets and scaling functions are not at all necessary.Finally, it has been shown how to apply this technique to the solutionof large matrix equations. This was accomplished by compressinga large matrix by means of the discrete wavelet transform.The disturbing point is that the condition number before and afterthe transform and thresholding is quite different as a function ofthe order of the filter.Figure 13. The waveform obtained in the sampled domainfrom decimation by a factor of two.P"'Figure 14. The spectrum of the original signal.8. AcknowledgmentThe contributions of the Editor-in-Chief in helping to preparethis paper for publication are gratefully acknowledged.9. Appendix 1. The principle of decimation by a factor of twoPictorially, decimation by a factor of two is represented bythe symbol shown in Figure 12, where the decimated signal,yD(n), has been generated from the original signal, x(n). In thesampled domain, this is equivalent to obtaining the waveformshown in Figure 13. Note that altemate sample values have beendropped. Therefore, from Figure 13, in the sampled domainFigure 15. The spectrum of the down-sampled signal.or in the z transform domain,Yo(,) = -yyD(n)z-n =nform evenCX(M)Z-"266 IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998

= ‘[x(S) + Xj-&)I.2If we observe the spectrum, then one observes that the original signalhas the spectrum given in Figure 14. Once the signal is downsampled,the spectrum is as given in Figure 15. Hence, the spec-trum of YD(e’”) is aliased, and it is the sum ofFigure 18. The spectrum of the original signal.10. Appendix 2. The principle of expansion by a factor of twoPictorially, up-sampling can be represented by the symbolshown in Figure 16. In the sampled domain, this is equivalent toinserting a zero between the sampled signals, as shown in Figure17. Mathematically, this is equivalent toFigure 19. The spectrum of the up-sampled signal.or, in the transform domainIf we observe the spectrum of YL(z), then we observe that theoriginal signal in Figure 18 is transformed into the spectrum of Y2 ,as shown in Figure 19.11. ReferencesFigure 16. The symbol used to represent up-sampling by a factorof two.P1. Ingrid Daubechies, Ten Lectures on Wavelets, CBMS-NSFRegional Conference Series in Applied Mathematics, Philadelphia,SIAM, 1992.2. C. K. Chui, An Introduction to Wavelets, New York, AcademicPress, 1992.3. P. P. Vaidyanathan, Multirute Systems and Filter Banks,Englewood Cliffs, NJ, Prentice-Hall, 1993.4. J. Morlet, G. Arens, I. Fourgeau and D. Giard, “Wave Propagationand Sampling Theory,” Geophysics, 47, 1982, pp. 203-236.5. Special issue ofIEEE Proceedings, 84,4, April 1996.6. D. Esteban and C. Galland, “Application of Quadrature MirrorFilters to Split-Band Voice Coding Schemes,” Proceedings of theIEEE International Conference on Acoustics, Speech, and SignalProcessing, Hartford, Connecticut, 1977, pp. 191-195.7. M. J. T. Smith and T. P. Barnwell, 111, “Exact ReconstructionTechniques for True Structured Subband Coders,” IEEE Transactionson Acoustics, Speech, and Signal Processing, ASSP-39,1986, pp. 434-441.8. M. Vettereli, “Multidimensional Subband Coding: Some Theoryand Algorithms,” Signal Processing, 6, 1984, pp. 97-1 12.Figure 17. Up-sampling by a factor of two is equivalent toinserting a zero between the sampled signals in the sampleddomain.9. T. H. Koomwinder (ed.), Wavelets: An Elementary Treatment ofTheory and Applications, New Jersey, World Scientific, 1993.10. S. Schweid and T. K. Sarkar, “A Sufficiency Criteria forOrthogonal QMF Filters to Ensure Smooth Wavelet Decomposi-/E€€ Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998 67

tion,” Applied and Computational Harmonic Analysis, 2, 1995, pp.61-67.11. S. Schweid and T. K. Sarkar, “Iterative Calculating and Factorizationof the Autocorrelation Function of Orthogonal Waveletswith Maximal Vanishing Moments,” IEEE Transactions on Circuitsand Systems, CAS-42, 11, November 1995, pp. 694-701.12. R. Ansari, C. Guillemot and J. F. Kaiser, “Wavelet ConstructionUsing Lagrange Half-Band Filter,” IEEE Transactions on CircuitsandSystems, CAS-38, 1991, pp. 1116-1118.13. M. A. Krasnoselskii, G. N. Vainikko, P. P. Zabreiko, Ya B.Rutitskii and V. Ya Stetsenko, Approximate Solution of OperatorEquations, Groningen, The Netherlands, Walters-Noordhoff, 1972.14. S. G. Mikhlin, The Numerical Performance of VariationalMethods, Groningen, The Netherlands; Walters Noordhoff, 1971.15. K. Rektorys, Variational Methods in Mathematics, Science andEngineering, Dordrecht, Holland, D. Reidel Publishing Company,1975.16. T. K. Sarkar, “A Note on the Choice of Weighting Functionalin the Method of Moments,” IEEE Transactions on Antennas andPropagation, AP-33,4, April 1985, pp. 436-441.17. T. K. Sarkar, A. R. Djordjevic and E. h as, “On the Choice ofExpansion and Weighting Functions in the Numerical Solution ofOperator Equation,” IEEE Transactions on Antennas and Propagation,AP-33, 9, September 1985, pp. 988-996.IS. G. Beylkin, R. Coifman and V. Rokhlin, “Fast Wave Transformsand Numerical Algorithms,” Communications on Pure andAppliedMathematics, 44, pp. 141-183.19. A. P. Calderon, “Intermediate Spaces and Interpolation, theComplex Method,”Stud. Math., 24, 1964, pp. 113-190.20. W. H. Press, S, Teukolsky, W. T. Vetterling and B. P. Flemery,Numerical Recipes-The Art of Scientijic Computing, Cambridge,Cambridge University Press, 1992.21. S. C. Schweid, “Projection Method Minimization Techniquesfor Smooth Multistage QMF Filter Decompositions,” PhD thesis,Syracuse University, May 1994.22. B. Z. Steinberg and Y. Leviatan, “On the Use of WaveletExpansions in the Method of Moments,” IEEE Transactions onAntennas and Propagation, AP-41, 5, 1993, pp. 610-619.23. J. C. Goswami, A. K. Chan and C. K. Chui, “On Solving First-Kind Integral Equations Using Wavelets 011 a Bounded Interval,”IEEE Transactions on Antennas and Propagation, AP-43, 6, June1995, pp. 614-622.24. G. Wang, “A Hybrid Wavelet Expansion and Boundary ElementAnalysis of Electromagnetic Scattering from ConductingObject,” IEEE Transactions on Antennas and Propagation, AP-43,2, February 1995, pp. 170-178.25. C. Su and T. K. Sarkar, “A Multiscale Moment Method forSolving Fredholm Integral Equation of the First Kind,” in J. A.E=mg (ed.), Reports on Progress in Electromagnetic Research,Volume 17, Cambridge, Massachusetts, EMW Publishing, 1997,pp. 237-264.26. T. K. Sarkar, R. S. Adve, L. Castillo and M. Salazar, “Utilizationof Wavelet Concepts in Finite Elements for an Efficient Solutionof Maxwell’s Equations,” Radio Science, 29, July 1994, pp.965-977.27. C. Su and T. K. Sarkar, “Electromagnetic Scattering fromCoated Strips Utilizing the Adaptive Multiscale Moment Method,”in J. A. King (ed.), Reports on Progress in ElectromagneticResearch, Volume 18, Cambridge, Massachusetts, EMW Publishing,1998, pp. 173-208.28. A. Brandt, “Multi-level Adaptive Solutions to Boundary ValueProblems,” Mathematics of Computation, 31, 1997, pp. 330-390.29. W. Hackbusch, Multigrid Methods and Applications, NewYork, Springer-Verlag, 1985.30. S. F. McCormick, Multigrid Methods: Theoly, Applicationsand Supercomputing, New York, Marcel Dekker, 1988.3 1. J. Mandel, “On Multilevel Iterative Methods for Integral Equationsof the Second Kind and Related Problems,” Numer. Math.,46, 1985, pp. 147-157.32. P.W. Hemker and H. Schippers, “Multiple Grid Methods forthe Solution of Fredholm Integral Equations of the Second Kind,”Mathematics of Computation, 36, 153, 1981.33. K. Kalbasi and K. R. Demarest, “A Multilevel Enhancement ofthe Method of Moments,” in Seventh Annual Review of Progress inApplied Computational Electromagnetics, March 1 99 1, Monterey,California, Naval Postgraduate School, pp. 254-263.34. K. Kalbasi and K. R. Demarest, “A Multilevel Formulation ofthe Method of Moments,” IEEE Transactions on Antennas Propagation,AP-41, 5, May 1993, pp. 589-599.35. Z. Baharav, Y. Leviatan, “Impedance Matrix CompressingUsing Adaptively Constructed Basis Functions,” IEEE Transactionson Antennas and Propagation, AP-44, 9, 1996, pp. 1231-1238.36. R. L. Wagner and W. C. Chew, “A Study of Wavelets for theSolution of Electromagnetic Integral Equations,” IEEE Transactionson Antennas and Propagation, AP-43, 8, 1995, pp. 802-810.37. F. X. Canning, J. F. School, “Diagonal Preconditioners for theEFIE Using a Wavelet Basis,” IEEE Transactions on Antennas andPropagation, AP-44, 9, 1996, pp. 1239-1246.38. G. Strang and T. Nguyen, Wavelet and Filter Banks, Cambridge,Wellesley-Cambridge Press, 1996.68 IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998

Introducing Feature Article AuthorsPhD degree at Syracuse University. His research interests includenumerical electromagnetics and the use of signal-processing techniquesin numerical electromagnetics.Tapan Kumar Sarkar received the B. Tech. degree from theIndian Institute of Technology, Kharagpur, India, the MScE degreefrom the University of New Brunswick, Fredericton, Canada, andthe MS and PhD degrees from Syracuse University, Syracuse, NewYork in 1969, 1971, and 1975, respectively.From 1975 to 1976, he was with the TACO Division of theGeneral Instruments Corporation. From 1976 to 1985, he was withthe Rochester Institute of Technology, Rochester, NY. From 1977to 1978, he was a Research Fellow at the Gordon McKay Laboratory,Harvard University, Cambridge, MA. He is now a Professorin the Department of Electrical and Computer Engineering, SyracuseUniversity; Syracuse, NY. He has authored or co-authoredmore than 170 journal articles, and has written chapters in tenbooks. His current research interests deal with numerical solutionsof operator equations arising in electromagnetics and signal processing,with application to system design.Dr. Sarkar is a registered Professional Engineer in the Stateof New York. He was an Associate Editor for Feature Articles ofthe IEEE Antennas and Propagation Society Newsletter, and hewas the Technical Program Chairman for the 1988 IEEE Antennasand Propagation Society International Symposium and URSIRadio Science Meeting. He was the Chairman of the IntercommissionWorking Group of URSI on Time-Domain Metrology. He is amember of Sigma Xi and USNC/URSI Commissions A and B. Hereceived one of the “best solution” awards in May, 1977, at theRome Air Development Center (RADC) Spectral EstimationWorkshop. He received the Best Paper Award of the IEEE Transactionson Electromagnetic Compatibility in 1979, and at the 1997National Radar Conference. He is a Fellow of the IEEE. Hereceived the degree of Docteur Honoris Causa from the UniversiteBlaise Pascal, Clermont-Ferrand, France, in 1998.Chaowei Su was born in Jiangsu, People’s Republic ofChina, on April 23, 1961. He received the BS and MSc degreesfrom Northwestern Polytechnical University (NPU), both in theDepartment of Applied Mathematics, in 1981 and 1986, respectively.He joined the Department of Applied Mathematics at NPUin 1982. Mr. Su was appointed as an Associate Professor and a fullProfessor at NPU in 1993 and 1996, respectively. He was a VisitingScholar at SUNY, Stony Brook, New York, USA, betweenDecember, 1990, and June, 1992, supported by a Grumman Fellowship.Since August, 1996, he has been a Visiting Professor atSyracuse University, IJSA. He is an author of Numerical Methodsin Inverse Problems of Partial Differential Equations and TheirApplications (published by NPU Press). His main research interestsare in the area of numerical methods of electromagnetic scatteringand inverse scattering, and inverse problems of partial differentialequations.Raviraj S. Adve was born in Bombay, India. He received theBTech degree in electrical engineering from the Indian Institute ofTechnology, Bombay, in 1990. He is currently working toward theMagdalena Salazar-Palma was born in Granada, Spain. Shereceived the PhD degree in Ingeniero de Telecomunicacion fromthe Universidad Politecnica de Madrid (Spain), where she is a ProfesorTitular of the Departamento de Seiiales, Sistemas y Radiocomunicaciones(Signals, Systems and RadiocommunicationsDepartment) at the Escuela Tecnica Superior de Ingenieros deTelecomunicacion of the same university. She has taught courseson electromagnetic field theory, microwave and antenna theory,circuit networks and filter theory, analog and digital communicationsystems theory, and numerical methods for electromagneticfield problems, as well as related laboratories. Her research withinthe Grupo de Microondas y Radar (Microwave and Radar Group)is in the areas of electromagnetic field theory, and computationaland numerical methods for microwave structures, passive components,and antenna analysis; design, simulation, optimization,implementation, and measurements of hybrid and monolithicmicrowave integrated circuits; and network and filter theory anddesign.She has authored a total of 10 contributions for chapters andarticles in books, 15 papers in international journals, and 75 papersin international symposiums and workshops, plus a number ofnational publications and reports. She has lectured in several shortcourses, some of them in the framework of European CommunityPrograms. She has participated in 19 projects and contracts,financed by international, European, and national institutions andcompanies. She has been a member of the Technical ProgramCommittee of several international symposiums, and has acted asreviewer for international scientific journals. She has assisted theComision Interministerial de Ciencia y Tecnologia (NationalBoard of Research) in the evaluation of projects. She has alsoserved in several evaluation panels of the Commission of theEuropean Communities. She has acted in the past and is currentlyacting as topical Editor for the disk of references of the Review ofRadio Science. She is a member of the editorial board of two scientificjournals. She has served as Vice Chair and Chair of theIEEE MTTIAP-S Spanish joint Chapter, and is currently serving asChair of the Spain Section of the IEEE. She has received two individualresearch awards from national institutions.Luis-Emilio Garcia-Castillo was born in 1967 in Madrid,Spain. In 1992. he received the degree of Ingeniero de Telecomu-IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998 69

nicaci6n from the Universidad PolitCcnica de Madrid (Spain). In1993, he became a Research Assistant in the Departaniento deSeiiales, Sistemas y Radiocomunicaciones (Signals, Systems andRadiocommunications Department) in the Escuela TBcnica Superiorde Ingenieros de Telecomunicaci6n of the same university.Since 1997, he has been an Associate Professor of the Departamentode Ingenieria Audiovisual y de Comunicaciones (Audiovisualand Communications Engineering Department) in the EscuelaUnivesitaria de Ingenieria TCcnica de Telecomunicaci6n of thesame university, where he teaches microwave theory and therelated laboratory. His research activities and interests are focusedon the application of numerical methods, mainly finite elements, toelectromagnetic problems, including research on curl-conformingfinite elements, the characterization of multiconductor andwaveguide structures, analysis of scattering and radiation problems,and the use of wavelet theory in computational electromagnetics.Other research areas of his interest are network theory andfilter design.He has authored four contributions for chapters and articlesin books, six papers in international joumals, and 29 papers inintemational symposiums, plus a number of national publicationsand reports. He has participated in seven projects and contractsfinanced by international, European, and national institutions andcompanies.Rafael Rodriguez Boix received the BSc, MSc, and PhDdegrees in physics from the University of Seville, Spain, in 1985,1986, and 1990, respectively. Since 1985, he has been with theElectronics and Electromagnetics Department at the University ofSeville, where he became an Associate Professor in 1994. Durmgthe summers of 1991 and 1992, he was at the Electrical EngineeringDepartment of UCLA as a Visiting Researcher Also, dungthe summer of 1996, he was at the Computer and Electrical EngineeringDepartment of Syracuse University as a VisitingResearcher. His current research activities are focused on thenumerical electromagnetic analysis of planar microwave circuitsand printed circuit antennas. %’August 13-21,1999, University of Toronto, Canadairst Call for Papers Now AvailableThe first announcement booklet and call for papers for the XXVIth General Assembly of theIntemational Union of Radio Science is now available. It includes the schedule for the sessionsof the 10 URSI Commissions, as well as the instructions and format for submitting papers. Thereis also information on the Young Scientists Program and a Canadian Student Competition.The information in this first announcement booklet is essential for anyone wishing to submit anabstract of a paper to be presented at the General Assembly. The booklet can be obtained bysending a request with full address and contact information toURSI GA ‘99 Management OfficeNational Research Council CanadaMontreal Road, Building M-19Ottawa, Ontario, Canada K1A OR6Tel: (613) 993-7271; Fax: (613) 993-7250E-mail: URSI99@nrc.caThose interested can also obtain the information, and request being placed on the mailing list, atThe deadline for receipt of abstracts is January 15,1999.70 IEEE Antennas and Propagation Magazine, Vol. 40, No. 5, October 1998