Greville's Method for Preconditioning Least Squares ... - Projects

More documents

Recommendations

Info

2 1 Introduction Consider the least squares problem, min ‖b − Ax‖ 2, (1.1) x∈R n where A ∈ R m×n , b ∈ R m . To solve least squares problems, there are two main kinds of methods : direct methods and iterative methods [4]. When A is large and sparse, iterative methods, especially Krylov subspace methods are preferred for solving (1.1), since they require less memory and computational work compared to direct methods. In [14], Hayami et al. proposed using GMRES [16] to solve least squares problems by using some preconditioners. By using a preconditioner B ∈ R n×m preconditioning (1.1) from the left, we can transform problem (1.1) to min ‖Bb − BAx‖ 2. (1.2) x∈R n On the other hand, we can also precondition problem (1.1) from the right and transform the problem (1.1) to min ‖b − ABy‖ 2. (1.3) y∈R m In [14], necessary and sufficient conditions for B such that (1.2) and (1.3) are equivalent to (1.1) were given, respectively. There are different methods to construct the preconditioner B, the simplest choice of B is to choose B = A T . If rank(A) = n, one may use B = CA T where C = diag(A T A) −1 or better, use Robust Incomplete Factorization(RIF) [2,3] to construct C[14]. Another possibility is to use the incomplete Givens orthogonalization method to construct B [19]. However, so far no efficient preconditioners has been proposed for the rank deficient case. In this paper, we will construct a preconditioner which also works when A is rank deficient. The idea is to use the approximate inverse of the coefficient matrix of a linear system Ax = b (1.4) with A ∈ R n×n as a preconditioner. This kind of preconditioners were originally developed for solving nonsingular linear systems [10,15]. Since now A is a general matrix, we construct matrix M ∈ R n×m which is an approximation to the Moore-Penrose inverse [9,17] of A, and use M to precondition the least squares problem (1.1). When A is rank deficient, the preconditioner M can also be rank deficient. Note, in passing, that in [20] it was shown that for singular linear systems, singular preconditioners are sometimes better than nonsingular preconditioners. The main contribution of this paper is to give a new way to precondition general least squares problems from the perspective of the approximate Moore-Penrose inverse. Similar to the RIF preconditioner[2,3] our method also includes an A T A- orthogonalization process when the coefficient matrix is full column rank. When A is rank deficient, our method tries to orthogonalize the linearly independent part in A. We also give a theoretical analysis on the equivalence between the preconditioned problem and the original problem, and discuss the possibility of breakdown when using GMRES to solve the preconditioned system. The rest of the paper is organized as follows. In Section 2, we first introduce some properties of the Moore-Penrose inverse of a rank-one updated matrix and the original
3 Greville’s method. In Section 3, based on the Greville’s method, we propose a global algorithm and a vector-wise algorithm for constructing the preconditioner M which is an approximate generalized inverse of A. In Section 4, we show that for full column rank matrix A, our algorithm is similar to the RIF preconditioning algorithm[3] and includes an A T A-orthogonalization process. In Section 5 and Section 6, we prove that under a certain assumption, using our preconditioner M, the preconditioned problem is equivalent to the original problem, and the GMRES method can determine a solution to the preconditioned problem before breakdown happens. In Section 7, we consider some details on the implementation of our algorithms. Numerical results are presented in Section 8. Section 9 concludes the paper. 2 Greville’s Method Given a rectangular matrix A ∈ R m×n , rank(A) = r ≤ min{m, n}, assume that the Moore-Penrose inverse of A is known. We are interested in how to compute the Moore- Penrose inverse of A + cd T , c ∈ R m , d ∈ R n , (2.1) which is a rank-one update of A [9,12]. In [9], the following six logical possibilities are considered 1. c ∉ R(A), d ∉ R(A T ) and 1 + d T A † c arbitrary, 2. c ∈ R(A), d ∉ R(A T ) and 1 + d T A † c = 0, 3. c ∈ R(A), d arbitrary and 1 + d T A † c ≠ 0, 4. c ∉ R(A), d ∈ R(A T ) and 1 + d T A † c = 0, 5. c arbitrary, d ∈ R(A T ) and 1 + d T A † c ≠ 0, 6. c ∈ R(A), d ∈ R(A T ) and 1 + d T A † c = 0. Here R(A) denotes the range space of A. For each possibility, an expression for the Moore-Penrose inverse of the rank one update of A is given by the following theorem [9]. Theorem 1 For A ∈ R m×n , c ∈ R m , d ∈ R n , let k = A † c, h = d T A † , u = (I−AA † )c, v = d T (I − A † A), and β = 1 + d T A † c. Note that, c ∈ R(A) ⇔ u = 0 (2.2) d ∈ R(A T ) ⇔ v = 0, (2.3) then, the generalized inverse of A + cd T is given as follows. 1. If u ≠ 0 and v ≠ 0, then (A + cd T ) † = A † − ku † − v † h + βv † u † . 2. If u = 0 and v ≠ 0, and β = 0, then (A + cd T ) † = A † − kk † A † − v † h. 3. If u = 0 and β ≠ 0, then (A + cd T ) † = A † + 1¯β v T k T A † − ¯β σ 1 p 1 q1 T , where p 1 = ( ‖k‖ 2 ) ( 2 − ¯β vT + k , q1 T ‖v‖ 2 ) 2 = − ¯β kT A † + h . 4. If u ≠ 0, v = 0 and β = 0, then (A + cd T ) † = A † − A † h † h − ku † . 5. If v = 0 and β ≠ 0, then (A + cd T ) † = A † + 1¯β A † h T u T − ¯β σ 2 p 2 q2 T , where p 2 = ) ( ‖h‖ 2 ( ‖u‖2 − ¯β A† h T + k , q2 T 2 = − ¯β uT + h , and σ 2 = ‖h‖ 2 2‖u‖ 2 2 + |β| 2 . 6. If u = 0, v = 0 and β = 0, then (A + cd T ) † = A † − kk † A † − A † h † h + (k † A † h † )kh. )
Page 1: Noname manuscript No. (will be inse
Page 5 and 6: 5 we can express A † i in a unifi
Page 7 and 8: 7 Remark 4 If k i is sparse, when w
Page 9 and 10: 9 6. k j = k j + uT i aj f i (e i
Page 11 and 12: 11 where C is a nonsingular matrix.
Page 13 and 14: 13 Hence, [ ] M = (I − K)F −1 I
Page 15 and 16: 15 where ∆E is an error matrix. H
Page 17 and 18: 17 Theorem 10 Let A ∈ R m×n . If
Page 19 and 20: 19 Table 1 Information on the test
Page 21 and 22: 21 preconditioner is usually much d
Page 23: 23 problems are equivalent to the o

Greville's Method for Preconditioning Least Squares ... - Projects

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?