THESE de DOCTORAT - cerfacs

3.4 GMRES 55
Stopping criteria of GMRES
In order to analyze the convergence of GMRES, it is important to know which mathematical
properties of A determine the size of ‖r‖. There are two main observations. The first is that
GMRES converges monotonically ‖r m+1 ‖ ≤ ‖r m ‖. The reason is that ‖r m ‖ is as small as possible
for the subspace K m . By enlarging K m to the space K m+1 , it is only possible to decrease the
residual norm, or at worst leave it unchanged. The second is that after at most n steps the
process must converge, at least in the absence of round errors ‖r n ‖ → 0.
Many algorithms of numerical linear algebra satisfy a condition that is both stronger and simpler
than stability. We say that an algorithm ˜f for a problem f is backward stable if ˜f (x) = f ( ˜x)
for some approximated solution ˜x = x m . Here ˜x = x + δx and ˜f = f + δ f are a perturbed solution
and a perturbed function respectively. The normwise backward error η is then defined
as:
η = ‖x m − x‖
‖x‖
= ‖A−1 (b − r m ) − A −1 b‖
‖A −1 b‖
= ‖r m‖ 2
‖b‖ 2
(3.56)
The backward error measures the distance between the data of the initial problem and those of
the perturbed problem. The best η one can require from an algorithm is a backward error of
the order of the machine precision. In practice, the approximation of the solution is acceptable
when its backward error is lower than the uncertainty on the data.
3.4.1 Preconditioning
The convergence of a matrix iteration depends on the properties of the matrix, i.e., the eigenvalues,
the singular values, or sometimes other information. It is interesting to comprehend
that in many cases, the problem of interest can be transformed so that the properties of the matrix
are improved drastically. This process of ‘preconditioning’ is essential to most successful
applications of iterative methods.
Suppose we wish to solve an n × n nonsingular system Ax = b. For any nonsingular n × n
matrix M, the system MAx = Mb has the same solution. If we solve this system iteratively, the
convergence will depend on the properties of MA instead of those of A. If this preconditioner
M is well chosen, the problem may be solved much more rapidly. The best choice ever for a
preconditioner is the inverse of the system matrix A, i.e., M = A −1 . Of course it would be
ridiculous to use A −1 as a preconditioner since this is already the solution of the problem. The
idea is then to make M −1 as close as A as possible. In other words, to make the eigenvalues of
M −1 A be close to 1 and ‖M −1 A − I‖ 2 small where I is the identity matrix. In doing so, a quick
convergence of the solution is expected.
Two well known preconditioning methods are the Diagonal scaling or Jacobi (M −1 = diag(A))