The Steepest Descent Algorithm for Unconstrained Optimization and ...

7 Proof of Kantorovich Inequality
Kantorovich Inequality: Let A and a be the largest and the smallest
eigenvalues of Q, respectively. Then
(A + a) 2
β ≤ .
4Aa
Proof: Let Q = RDR T , and then Q −1 = RD −1 R T , where R = R T is
an orthonormal matrix, and the eigenvalues of Q are
0 < a = a 1 ≤ a 2 ≤ ... ≤ a n = A,
and,
⎛ ⎞
a1 0 ... 0
0 a 2 ... 0 D = ⎜
⎝
.
. . .
⎟ . . ⎠ .
0 0 ... a n
Then
β =
(d T RDR T d)(d T RD −1 R T d) f T Dff T D −1 f
=
(d T RR T d)(d T RR T d) f T ff T f
∑ n
where f = R T f
d.Let λ i = f T i 2
f . Then λ i ≥ 0and
λ i =1, and
i=1
∑ n ( )
1
∑n
∑ n ( ) λ
1 i a i
i=1
β = λ i a i λ i = ⎛ ⎞ .
a
i=1 i=1 i
⎜ 1 ⎟
⎝ ∑ n ⎠
λ i a i
i=1
The largest value of β is when λ 1 + λ n = 1, see the illustration in Figure 8.
Therefore,
24