The Steepest Descent Algorithm for Unconstrained Optimization and ...
The Steepest Descent Algorithm for Unconstrained Optimization and ...
The Steepest Descent Algorithm for Unconstrained Optimization and ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
7 Proof of Kantorovich Inequality<br />
Kantorovich Inequality: Let A <strong>and</strong> a be the largest <strong>and</strong> the smallest<br />
eigenvalues of Q, respectively. <strong>The</strong>n<br />
(A + a) 2<br />
β ≤ .<br />
4Aa<br />
Proof: Let Q = RDR T , <strong>and</strong> then Q −1 = RD −1 R T , where R = R T is<br />
an orthonormal matrix, <strong>and</strong> the eigenvalues of Q are<br />
0 < a = a 1 ≤ a 2 ≤ ... ≤ a n = A,<br />
<strong>and</strong>,<br />
⎛ ⎞<br />
a1 0 ... 0<br />
0 a 2 ... 0 D = ⎜<br />
⎝<br />
.<br />
. . .<br />
⎟ . . ⎠ .<br />
0 0 ... a n<br />
<strong>The</strong>n<br />
β =<br />
(d T RDR T d)(d T RD −1 R T d) f T Dff T D −1 f<br />
=<br />
(d T RR T d)(d T RR T d) f T ff T f<br />
∑ n<br />
where f = R T f<br />
d.Let λ i = f T i 2<br />
f . <strong>The</strong>n λ i ≥ 0<strong>and</strong><br />
λ i =1, <strong>and</strong><br />
i=1<br />
∑ n ( )<br />
1<br />
∑n<br />
∑ n ( ) λ<br />
1 i a i<br />
i=1<br />
β = λ i a i λ i = ⎛ ⎞ .<br />
a<br />
i=1 i=1 i<br />
⎜ 1 ⎟<br />
⎝ ∑ n ⎠<br />
λ i a i<br />
i=1<br />
<strong>The</strong> largest value of β is when λ 1 + λ n = 1, see the illustration in Figure 8.<br />
<strong>The</strong>re<strong>for</strong>e,<br />
24