C++ for Scientists - Technische Universität Dresden

278 CHAPTER 14. NUMERICAL EXERCISES
We use the property that
det(A − λI) = det(R)
Since R is upper triangular, the determinant is the product of the diagonal elements of R.
The matrices A are constructed as follows. Start with a simple case : the diagonal matrix with
elements 1, 2, . . . , n on the main diagonal. Then do the tests for the same matrix multiplied
on left and right by a random orthogonal matrix X, as in A = XDX T where D is a diagonal
matrix.
g the minimum of a convex function
This exercise is a programming exercise on Newton’s method. First, we explain the method for
a function with a single variable, then we discuss the case of multivariate functions, and finally,
we show a small application.
14.6.1 Functions in one variable
For a differentiable function f, the minimum ˜x is attained for f ′ (˜x) = 0. So, we must find the
zero of the first order derivative. When f is a second order polynomial, we have
then an extreme value of f is attained for
which is a minimum when f ′′ ≡ 2γ > 0.
f = p := α + βx + γx 2
f ′ = p ′ := β + 2γx
˜x = − β
2γ
(14.6)
For an arbitrary function, we do not have such simple explicit formulae. We can use an iterative
method, which is called Newton’s method. It is an iterative approach, i.e. we start from an
initial guess ˜x and improve this value until it has converged to the minimum of the function. On
each iteration we approximate the function by a degree two polynomial, for which the simple
formula (14.6) can be used. One way to compute such a degree two polynomial is to start from
the Taylor expansion of f around ˜x :
f(x) = f(˜x) + f ′ (˜x)(x − ˜x) + 1
2 f ′′ (˜x)(x − ˜x) 2 + · · · .
If we approximate f by the first 3 terms (i.e. a degree two polynomial), then we have
f(x) ≈ p(x) := f(˜x) + f ′ (˜x)(x − ˜x) + 1
2 f ′′ (˜x)(x − ˜x) 2 .
If x is close to ˜x, |f(x) − p(x)| is small. The first order derivative is
f ′ (x) ≈ p ′ (x) = f ′ (˜x) + f ′′ (˜x)(x − ˜x) .