(x) is

Nonlinear Equations
The scientist described what is: the engineer creates what never was.
Fall 2010
Theodor von Karman
The father of supersonic flight
1

Problem Description
•Given a non-linear equation f(x)=0, ) find a x*
such that f(x*) = 0. Thus, x* is a root of f(x)=0.
•Galois theory in math tells us that only
polynomials l of fdegree ≤ 4 can be solved with
close forms using +, −, ×, ÷ and taking roots.
•General non-linear equations can be solved with
iterative methods.
•Basically, we try to guess the location of a root,
and approximate it iteratively.
•Unfortunately, this process can go wrong, leading
to another root or even diverge.
2

Methods Will Be Discussed
•There are two types of methods, bracketing and
open. The bracketing methods require an
interval that is known to contain a root, while
the open method does not.
•Commonly seen bracketing methods include
the bisection method and the regula falsi
method, and the open methods are Newton’s
method, the secant method, and the fixed-point
iteration.
3

Bisection Method: 1/6
•Idea: The key is the intermediate value theorem.
•If y = f(x) is a continuous function on [a,b] and r
is between f(a) and f(b), then there is a x* such
that r = f(x*).
•Therefore, f if f(a)×f(b)

Bisection Method: 2/6
•Note that f(a)×f(b) ) ) < 0 guarantees a root in [a,b].
•Compute c = (a+b)/2 and f(c).
•If f(c) = 0, we have found a root.
•Otherwise, either f(a)×f(c) < 0 or f(b)×f(c) < 0 but
not both. Use [a,c] for the former and [c,b] for
the latter until |f(c)| < ε .
1 2
y=f(x) 3
f(a)
4
a
x*
b
f(b)
X
5

Bisection Method: 3/6
•Convergence: g Since the first iteration reduces
the interval length to |b-a|/2, the second to |b-a|/2 2 ,
the third to |b-a|/2 3 , etc, after the k-th iteration,
the interval length is |b-a|/2 k .
•If ε > 0 is a given tolerance value, we have |b-a|/2 k
< ε or |b-a|/ε < 2 k for some k.
•Taking base 2 logarithm, we have the value of k,
the expected number of iterations with accuracy ε:
k = ⎡ ⎢ log 2
b
− a
⎤
ε
⎥
•Convergence g is not fast but guaranteed and steady.
6

Bisection Method: 4/6
• Algorithm:
• If ABS(b-a) < ε,
then stop.
• Note the red framed
IF statement.
Without this, an
infinite loop can
occur if |b-a| is very
small and c can be a
or b due to
rounding.
Fa = f(a)
Fb = f(b) ! Fa*Fb must be < 0
DO
IF (ABS(b-a) < ε) EXIT
c = (a+b)/2
IF (c ==a .OR. c == b) EXIT
Fc = f(c)
IF (Fc == 0) EXIT
IF (Fa*Fc < 0) THEN
b = c
Fb = Fc
ELSE
a = c
Fa = Fc
END IF
END DO
7

Bisection Method: 5/6
•However,,
there is a catch. ABS(b-a)

Bisection Method: 6/6
•The ABS(b-a) < ε may be changed to
ABS(b-a)/MIN(ABS(a),ABS(b)) < ε
•This test t has a potential ti problem: if the initial
iti bracket contains 0, MIN(ABS(a),ABS(b))
can approach h0 which hwould cause a division i i by
zero!
•Nothing is perfect!
9

Bisection Method Example: 1/3
•Suppose we wish to find the root closest to 6 of
the following equation:
f ( x)
=
sin( i( x
+ cos( x
))
1+
•Plotting the equation f(x) = 0 around x = 6 is
very helpful.
•You may use gnuplot for plotting. Gnuplot
is available free on many platforms such as
Windows, MacOS and Unix/Linux.
x
2
10

Bisection Method Example: 2/3
•Function f(x) has a root in [4,6], and f(4) < 0
and f(6) > 0. We may use [4,6] with the
Bisection method.
f ( x)
=
sin( x+
cos( x))
2
1+
x
11

Bisection Method Example: 3/3
• 19 iterations, x * ≈ 5.5441017… f(x * ) ≈ 0.863277×10 -7 .
[5.5,5.5625]
f ( x
)
=
sin( x + cos( x
))
2
1+ x
[555625] [5.5,5.625]
[5.5,5.75]
[5.5,6]
[4,6]
[5,6]
12

Newton’s Method: 1/13
•Idea:Ifa If a is very close to a root of y = f(x) ,the
tangent line though (a, f(a)) intersects the X-axis
at a point b that is hopefully closer to the root.
•The line through (a, f(a)) with slope f’(a) is:
y −
f ( a
) '
= f ( a)
x−
a
y=f(x)
(a,f(a))
a
x*
b
y - f(a) = f ’ (a)×(x-a)
13

Newton’s Method: 2/13
•The line through (a, f(a)) with slope f’(a) is
y
− f ( a)
x −
a
=
f
'
( a)
•Its intersection point b with the X-axis can be
found by setting y to zero and solving for x:
b
= a
−
f ( a)
'
f ( a)
y=f(x)
(a,f(a))
a
x*
b
14

Newton’s Method: 3/13
•Starting with a x 0 that is close to a root x*,
,
Newton’s method uses the following to compute
x 1 , x 2 , … until some x k converges to x*.
.
x
i
+11
= x
i
−
f ( xi
)
'
f ( x )
i
x 0 x 1 x 2 x 3
x*
y=f(x)
15

Newton’s Method: 4/13
•Convergence g 1/2:
•If there is a constant ρ, 0 < ρ < 1, such that
x *
− x
lim | |
k + 1
=
ρ
k→∞
*
| xk
− x |
the sequence x 0, x 1, …, is said to converge
linearly with ratio (or rate) ρ.
•If ρ is zero, the convergence is superlinear.
•If x k+1 and x k are close to x*, the above
expression means
|x k+1 – x*| ≈ ρ|x k – x*|
•The error at x k+1 is proportional to (i.e.,
linear) and smaller than the error at x k since
ρ < 1.
16

Newton’s Method: 5/13
•Convergence 2/2:
•If there is a p > 1 and a C > 0 such that
*
lim | x |
k + 1
− x
= C
k →∞
* p
| x − x |
k
the sequence is said to converge with order p.
•If x k+1 and x k are close to x*, , the above
expression yields |x k+1 - x * | ≈ C|x k - x*| p .
•Since |x k - x*| | iscloseto0andp p >1 1, |x k+1 -x*|
is even smaller, and has p more 0 digits.
17

Newton’s Method: 6/13
•The right shows a
possible algorithm that
implements Newton’s
method.
•Newton’smethod
s converges with order 2.
•However, it may not
converge at all, and a
maximum number of
iterations may be needed
to stop early.
x = initial value
Fx = f(x)
Dx = f’(x)
DO
IF (ABS(Fx) < ε) EXIT
New_X = x – Fx/Dx
Fx = f(New_X)
Dx = f’(New_X)
x = New_X
END DO
18

Newton’s Method: 7/13
•Problem 1/6:
• While Newton’s method has a fast
convergence rate, it may not converge at all no
matter how close the initial guess is.
verify this fact yourself
y
=
x
1/(2n+
1)
x 3 x 1 x 2
x 4
19

Newton’s Method: 8/13
•Problem 2/6:
• Newton’s method can oscillate.
• If y=f(x)=x x 3 -3x 2 +x+3, then y’=f’(x)=3x 2 -6x+1.
• If the initial value is x 0 =1, then we will have x 1
= 2, x 2 = 1, x 3 = 2, …. (see textbook page 78 for
a function plot).
• Note that if x 0 =-1, in 3 iterations the root is
found at -0.76929265!
• Therefore, carefully choosing an initial guess
is important.
20

Newton’s Method: 9/13
•Problem 3/6:
• Newton’s method can oscillate around a
minimal point, eventually causing f’(x) to be
zero.
• In the following, x 1 , x 3 , x 5 , … approach the
minimum i while x 2 , x 4 , …, approach hinfinity.
it
x*
x 1 x 3 x 5 x 2 x 4
21

Newton’s Method: 10/13
•Problem 4/6:
• Since Newton’s method uses x k+1 = x k -
f(x k )/f’(x k ), it requires f’(x k ) ≠ 0.
• As a result, multiple roots can be a problem
because f’(x)=0 at a multiple root.
• Inflection points may also cause problems.
f’(x)=0at = 0 this inflection point
f’(x 2 )=0
f’(x) = 0 at a multiple root
x*
x 2
x 1
22

Newton’s Method: 11/13
•Problem 5/6:
• Newton’s method can converge to a remote
root even though the initial guess is close to the
anticipated i t one.
x 2
x*
x 4 x 3
x 1
4
23
But, we get this one
This is what we want!

Newton’s Method: 12/13
•Problem 6/6:
• Even though Newton’s method has a
convergence rate of 2, it can be very slow if the
initial iti guess is not right. The following solves
x 10 -1=0. Note that this x * = 1 is a multiple root!
slow!
Iteration ti 0 x = 0.5 f(x) = -0.99902343
Iteration 1 x = 51.65 f(x) = 1.3511494E+17
Iteration 2 x = 46.485 f(x) = 4.711166E+16
Iteration 3 x = 41.836502 f(x) = 1.6426823E+16
Iteration 4 x = 37.65285 f(x) = 5.727679E+15
...... Other output ......
Iteration 10 x = 20.01027 f(x) = 1.0292698E+13
...... Other output t ......
Iteration 20 x = 6.9771494 f(x) = 273388500.0
Iteration 30 x = 2.4328012 f(x) = 7261.167
Iteration 40 x = 1.002316 f(x) = 0.02340281
02340281
Iteration 41 x = 1.000024 f(x) = 2.3961067E-4
Iteration 42 x = 1.0 f(x) = 0.0E+0
24

Newton’s Method: 13/13
•A few important notes:
• Newton’s method is also referred to as the
Newton-Raphson method.
• Plot the function to find a good initial guess.
• When the computation converges, plug the x*
back to the original equation to make sure it
is indeed a root.
• Check for f’(x) = 0.
• The use of a maximum number of iterations
would help prevent infinite loop.
25

Newton’s Method Example: 1/3
•Suppose we wish to find the root closest to 4 of
the following equation:
f ( x ) = sin( e − x
+
cos( x
))
•The following is a plot in [0,10]:
f ( x) = sin( e − x + cos( x))
26

Newton’s Method Example: 2/3
•A plot in [4,6] shows that the desired root is
approximately 4.7, and f(x) is monotonically
increasing in [4,6]:
f ( x) = sin( e − x + cos( x))
27

Newton’s Method Example: 3/3
•The following has f(x) and f’(x):
• Initial x 0 = 4.
− x
f ( x) = sin( e + cos( x))
'
−x
−x
f ( x) =− cos( e + cos( x))( e + sin( x))
x f(x) f’(x)
3 iterations
0 4 -0.59344154 0.5943911
x* ≈ 4.703324
1 4.998402 0.2848776 0.9131543
f(x*) ≈ 0.8102506×10 -7 f’(x*) ≈ 0.99089384
2 4.686432 -0.016733877 0.99030494
3 4.703329 0.5750917×10 -5 0.99089395
4 4.703324 0.8102506×10 -7 0.99089384
28

The Secant Method: 1/8
•Idea :Newton’s method requires f’(x). What if
f’(x) is difficult to compute or even not available
•Methods that use an approximation g k of f’(x)
are referred to as quasi-Newton methods. Thus,
x k+11 is computed (from x k ) as follows:
x
k
+ 1
= xk
−
f ( x )
k
g k
•The secant method offers a simple way for
estimating g k .
29

The Secant Method: 2/8
•The secant method uses the slope of the chord
bt between x k-1 and x k as an approximate of ff’( f’(x k ) .
•Therefore, x k+1 is the intersection point of this
chord and the X-axis. i
•Since the secant method uses two points, it is
usually referred to as a two-point method.
(x k-1 ,f(x k-1 ))
(x k ,f(x k ))
x k+1
Newton’s method
x* x k x k-1
30

The Secant Method: 3/8
•Since the slope g k is
g
k
=
f ( x ) − f ( x )
k
x
k
−
x
•Some algebraic manipulations yield the secant
method formula:
k−1
k−1
xk
− xk
1 x −
k+
1
= xk − ×
f ( xk
)
f( x ) − f( x )
k
•It is more complex than that of Newton’s method;
but, it may be cheaper than evaluating f’(x k ) .
k−1
31

The Secant Method: 4/8
•Convergence:
• The secant method, in fact all two-point
methods, are superlinear!
• More precisely, one can show that the
following holds with a 1< p 0
| x *
|
k + 1
− x
lim
k→∞
|
* |
p
x − x
k
• Moreover, p is the root of p 2 – p – 1 =0:
=
C
1 = + 5
p =
2
1618 . .....
32

The Secant Method: 5/8
•Algorithm: a = initial value #1
b = initial value #2
•The right is the secant Fa = f(a)
method.
Fb = f(b)
DO
•Note that Fb and Fa c = b – Fb*(b-a)/(Fb-Fa)
may be equal, causing Fc = f(c)
IF (ABS(Fc) < ε) EXIT
either overflow or
a = b
division by zero!
Fa = Fb
•0/0 is also possible!
•Since there are three
subtractions, ti rewrite
the expression to avoid
cancellation if possible.
b = c
Fb = Fc
END DO
33

The Secant Method: 6/8
•The following shows the output of solving f(x) =
x 10 -1 with initial points x 0 =2 and x 1 =1.5.
•It is faster than Newton’s method (42 iterations).
a f(a) b f(b)
0 : 2.0 1023.0 1.5 56.66504
1 : 1.5 56.66504 1.4706805 46.33511
2 : 1.4706805 46.33511 1.3391671 17.550165
3 : 1.3391671 17.550165 1.2589835 9.004613
4 : 1.2589835 9.004613 1.1744925 1744925 3.994619
5 : 1.1744925 3.994619 1.1071253 1.7667356
6 : 1.1071253 1.7667356 1.0537024 0.68725025
7 : 1.0537024 0.68725025 1.0196909 0.21530557
8 : 1.0196909 0.21530557 1.0041745 0.04253745
9 : 1.0041745 0.04253745 1.0003542 3.5473108E-3
10 : 1.0003542 3.5473108E-3 1.0000066 6.556511E-5
Converged after 11 iterations x* = 1.0 f(x*) = 0.0E+0
34

The Secant Method: 7/8
•Due to subtractions, cancellation and loss of
significant digits are possible.
•If we start with x 0 =0and x 1 =15 1.5, division by
zero can occur! We hit a flat area of the curve.
explain why
Initially:
a = 0.0E+00 f(a) = -1.0 b = 1.5 f(b) = 56.66504
c = 0.026012301 f(c) = -1.0
Iteration 1 :
a = 1.5 f(a) = 56.66504 b = 0.026012301 f(b) = -1.0
c = 0.051573503 f(c) = -1.0
Iteration 2 :
a = 0.026012301 f(a) = -1.0 b = 0.051573503 f(b) = -1.0
Floating exception (Oops!!!)
35

The Secant Method: 8/8
•AA few important notes:
• The secant method evaluates the function
once per iteration.
• Plot the function to find a good initial guess.
• Check if the computed result is indeed a root.
• Check for f(x k) – f(x k-1 1) ≈ 0 (i.e., flat area) as
this can cause overflow or division by zero.
• The secant method shares essentially the
same pitfalls with Newton’s method.
• The use of a maximum number of iterations
can help prevent infinite loop.
36

Handling Multiple Roots: 1/2
•A polynomial f(x) ) has a root x* of multiplicity ypp
> 1, where p is an integer, if f(x) = (x – x*) p g(x)
for some polynomial g(x) and g(x*)≠0.
•Then we have
f(x*) ) = f’(x*) ( ) = f”(x*) ( ) = … f [p-1] (x*) = f [p] (x*) = 0
•For example, f(x) = (x-1)(x-1)(x-2)(x-3) has a
double root (i.e., p = 2) and f(x)=(x-1) ) ( ) 2 g(x) ) and
g(x) = (x-2)(x-3).
•Newton’s method and the secant method,
converge only linearly for multiple roots.
37

Handling Multiple Roots: 2/2
•Two modifications to Newton’s method can still
maintain quadratic convergence.
•If x* is a root of multiplicity p, then use the
following instead of the original:
x = +
x − p ×
k
1 k
'
f ( x
)
f
k
( x )
•Or, let g(x) = f(x)/f’(x) and use the following:
x
k
1
x
k
g
( x k
)
g'( x )
k
k
+ = − 38

The Regula Falsi Method: 1/5
•Idea:TheRegula Falsi,akafalse-aka positions,
method is a variation of the bisection method.
•The bisection method does not care about the
location of the root. It just cuts the interval at
the mid-point, and, as a result, it is slower.
•We may use the intersection point of the X-axis
and dthe chord dbetween (a, f(a)) and (b, f(b)) as an
approximation. Hopefully, this would be faster.
39

The Regula Falsi Method: 2/5
•The intersection point c
(a,f(a))
may not be closer to x* than
the mid-point (a+b)/2 is.
•The line of (a, f(a)) and (b,
(a+b)/2 x*
f(b)) is a c
b
y
− f ( a) f ( b) − f ( a)
=
x−
a b−
a
•Setting y=0 and solving for
x yield c as follows:
c
= a−
b−
a
f ( b) − f ( a)
× f ( a)
= b−
b−
a
f () b − f () a
×
f ( b)
(b,f(b))
40

The Regula Falsi Method: 3/5
•The Regula Falsi
method is a variation
of the bisection
method.
•Instead of computing
the mid-point, it
computes the
intersection ti point of
the X-axis and the
chord.
•Otherwise, they are
identical.
Fa = f(a)
Fb = f(b) ! Fa*Fb must be < 0
DO
c = a-Fa*(b-a)/(Fb-Fa)
Fc = f(c)
IF (ABS(Fc) < ε) EXIT
IF (Fa*Fc Fc < 0) THEN
b = c
Fb = Fc
ELSE
a = c
Fa = Fc
END IF
END DO
41

The Regula Falsi Method: 4/5
•The Regula Falsi method can be very slow when
hitting a flat area.
•One of the two end points may be fixed, while
the other end slowly moves the root.
•Near a very flat area f(b) ) – f(a) ) ≈ 0 holds!
x 1 x 2 x 3 x 4 x 5 x 0
x*
42

The Regula Falsi Method: 5/5
•Solving x 10 -1=0 with a = 0 and b = 1.3. Very slow!
a f(a) b f(b) c
1 0.0E+0 -1.0 1.3 12.785842 0.094299644 -1.0
2 0.094299644 -1.0 1.3 12.785842 0.18175896 -0.99999994
3 0.18175896 -0.99999994 1.3 12.785842 0.26287412 -0.99999845
4 0.26287412 -0.99999845 1.3 12.785842 0.33810526 -0.99998044
5 0.33810526 -0.99998044 1.3 12.785842 0.4078781 -0.99987256
10 0.6395442 -0.98855262 1.3 12.785842 0.6869434 -0.97660034
20 0.94343614 -0.44136887 1.3 12.785842 0.95533406 -0.36678296
30 0.99553364 -0.043776333 1.3 12.785842 0.9965725 -0.03375119
40 0.9996887 -3.1086206E-3 1.3 12.785842 0.9997617 -2.3804306E-3
50 0.99997854 -2.1457672E-4 1.3 12.785842 0.99998354 -1.6450882E-4
60 0.99999856 -1.4305115E-5 5 1.3 12.785842 0.9999989 -1.0728836E-5
f(c)
63 0.9999995 -4.7683715E-6
43

Fixed-Point Iteration: 1/19
•Rewrite f(x) = 0 to a new form x = g(x).
• f(x) = sin(x)-cos(x) = 0 becomes sin(x)=cos(x). Hence,
x = sin -1 (cos(x)) ( or x = cos -1 (sin(x)).
( • f(x) = e -x – x 2 = 0 becomes e -x = x 2 . Hence, x = -LOG(x 2 )
or x = SQRT(e -x ).
•The fixed-point iteration starts with an initial value
x 0 and computes x i+1 = g(x i ) until x k+1 ≈ g(x k ) for
some k.
•We are seeking a x* such that x*=g(x*) ) holds.
This x* is a fixed-point of g() since function g()
maps x* to x* itself, and, hence, a root of f(x).
44

Fixed-Point Iteration: 2/19
•The fixed-point iteration x=g(x) can be
considered as computing the intersection point of
the curve y=g(x) and the straight line y = x.
y = x
x* = g(x*)
y = g(x)
45

Fixed-Point Iteration: 3/19
•Let us rewrite x k+1= g(x k )
as y = g(x k ) and x k+1 = y .
•Thus,, y = g(x k ) means
from x k find the y-value of
g(x k ).
•Then, x k+1 = y means use
the y-value to find the
new x-value x k+1 on y = x.
•Repeat this procedure as
shown until, hopefully, it
converges.
Y
y = x
x k+2 = y y = g(x) ( )
x k+1 = y
y = g(x k ) y = g(x k+1 )
X
x k x k+1
46

Fixed-Point Iteration: 4/19
•Newton’s s method uses x k+1 = x k – f(x k )/f’(x k ).Let
g(x) = x – f(x)/f’(x) and Newton’s method
becomes the fixed-point iteration: x k+1 = g(x k ) .
•The fixed-point iteration converges linearly if
|g’(x)| < 1 in the neighborhood of the correct root.
•Observation: if g’(x)∈(0,1) and g’(x)∈(-1,0), the
convergence is asymptotic ti and oscillatory,
respectively, and it is faster if g’(x) approaches 0.
47

Fixed-Point Iteration: 5/19
•Two convergence cases:
‣ Left g’(x) ∈ (-1,0) and hence oscillatory
‣ Right g’(x) ) ∈ (0,1) and asymptotic tti
g’(x) ) ∈ (-1,0) 10) g’(x) ) ∈ (0,1)
x 1 x 3 x 2
x 1 x 2 x 3
oscillatory
asymptotic
48

Fixed-Point Iteration: 6/19
•Two divergence cases:
‣ Left g’(x) < -1
‣ Right g’(x) )>1
g’(x) < -1
g’(x) > 1
x 3 x 1 x 2 x 4 x 1 x 2 x 3
49

Fixed-Point Iteration: 7/19
•Since f(x) = 0 has to be transformed to a form of
x = g(x) in order to use the fixed-point iteration,
algebraic manipulation is necessary.
•Incorrect transformations can yield
roots we don’t want, diverge, or
cause runtime errors.
•As a result, after transformation, ti itis very
helpful to plot the curve of y = g(x) and the line y
= x, and dinspect tthe slope g’(x) ) near the desired
d
fixed point to find a good initial guess.
50

Fixed-Point Iteration: 8/19
•Consider f(x) = cos(x) – x 1/2 .
a root in the range of [0,2]
f(x) = cos(x) – x 1/2 51

Fixed-Point Iteration: 9/19
•Setting f(x) to zero yields
cos( x) − x = 0
•Rearranging i terms yields
cos 2 (x) = x.
•We use g(x) = cos 2 (x) for
the fixed-point iteration.
•From curve y = cos 2 (x)
and line y = x, we see only
one root at .64171…..
approximately .
y = cos 2 (x)
y = x
06 0.6 08 0.8
52

Fixed-Point Iteration: 10/19
•Setting g f(x) ) to zero yields
cos( x) − x = 0
•Rearranging terms yields
cos(x) = x 1/2 .
•Taking cos
-1 yields g(x) =
cos -1 (x 1/2 ).
•From y = cos -1 (x 1/2 ) and
line y = x, we see only one
root at 0.64171…..
approximately.
y = cos -1 (x 1/2 )
y = x
06 0.6 08 0.8
53

Fixed-Point Iteration: 11/19
•One may transform f(x) in many ways, each of
which could yield a different root. Some may
diverge or cause runtime errors.
•Again, plotting y = g(x) and y = x and inspecting
the slope g’(x) can help determine the
transformation that can yield the best result.
•For example, the following f(x) ) has two obvious
transformations:
2
f( x) = log( x + 1) − 1+
x
54

Fixed-Point Iteration: 12/19
•Plotting this function f(x) shows that there are
roots in [-0.8, -0.7], [2.0, 3.0] and [70, 80].
55

Fixed-Point Iteration: 13/19
•If we transform f(x) in the following way:
2
log( x
1) 1
+ = +
2 1 x
x
+
1 =
e
+
2 1 x
x e +
= −1
x
1
x e +x
= −1
gx e +x
1
( ) = −1
56

Fixed-Point Iteration: 14/19
•The fixed-point iteration converges to x*=
2.2513…… easily since 0 < g’(x) < 1; but, it is
difficult to get the other two roots.
This is the asymptotic case
y = x
gx e +x
1
( ) = −1
57

Fixed-Point Iteration: 15/19
•If f(x) is transformed in a different way:
2
log( x 1) 1
+ = +
x
(
2
x ) 2
log( + 1) = 1+
x
x
(
2
x ) 2
= log( + 1) −1
( ) 2
x
2
gx ( ) = log( + 1) −1
58

Fixed-Point Iteration: 16/19
This is also the asymptotic case
( ( x
2
)
) 2
gx ( ) = log + 1 −1
If the initial guess is larger than x* = 2.2513…
the fixed-point iteration converges to 72.309…
59

Fixed-Point Iteration: 17/19
This is the oscillatory case
If the initial iti guess is less than x* * = 2.2513…
2513
the fixed-point iteration converges to -0.7769…
( ( x
2
)
)
2
g( ( x
) = log + 1 −1
60

Fixed-Point Iteration: 18/19
( ( x
2
)) 2
gx ( ) = log + 1 −1
no way to reach x* = 2.2513…
converge to 72.309…
converge to -0.7769…
61

Fixed-Point Iteration: 19/19
•Conclusion: The fixed-point iteration is easy to
use. But, different transformations
may yield different roots and
sometimes may not converge.
Moreover, e some transformations at o s may
lead to runtime errors.
•One may try different transformations to find
the desired root. Only those transformations
that can compute the desired root are considered
correct.
•Use the fixed-point i iteration ti with care.
62

Fixed-Point Iteration Example: 1/9
•Consider the following function f(x):
f ( x) = e − x −sin( x)
•The following plot shows that t f(x) ) = 0h has four
roots in [0,10]. In fact, f(x) = 0 has an infinite
number of roots.
− x
f ( x) = e − sin( x)
=
0
63

Fixed-Point Iteration Example: 2/9
•Since e -x – sin(x) = 0, we have sin(x) = e -x ,andx x =
sin -1 (e -x ). Therefore, we may use g(x) = sin -1 (e -x ).
•We have a problem.
y = g(x) and y=x have only one
intersection point around 0.5.
As a result, only the root in
[0,1] can be found with the
fixed-point iteration.
y = x
−1
−
y = g( x) = sin ( e x )
0.5885…
64

Fixed-Point Iteration Example: 3/9
•Since 0 ≤ e -x ≤ 1, g(x) = sin -1 (e -x ) can be
computed without problems.
•Since e -x decreases monotonically from 1 =
e -0 to 0 as x→∞, sin -1 (e -x ) also decreases
monotonically from π/2 = sin -1 (e -0 )= sin -1 (1)
to 0 as x→∞.
.
•Therefore, g(x) = sin -1 (e -x ) and y = x has
only one intersection ti point, and the fixedpoint
iteration can only find one root of f(x)
= 0.
65

Fixed-Point Iteration Example: 4/9
•Left is a plot of g(x) ) in [0,1].
•g’(x) is calculated below:
−1
−
gx ( ) = sin ( e x )
−1 1
(sin ( x)) ' =
1−
x
g
( x)
=−
'
− x
e
1−
e
−2x
•Since g’(0.4) = -0.90…, 090 g’(0.5)
= -0.76… , g’(0.6) = -0.66…,
|g’(x)|

Fixed-Point Iteration Example: 5/9
x i
0 0.5
1 0.6516896
2 0.5482148
3 0.616252
4 0.5703949
5 0.6007996
26 iterationsti
x* ≈ 0.58853024
x 0
x 2 x 4
x 5 x 3 x 1
67

Fixed-Point Iteration Example: 6/9
•We may transform e -x –sin(x) = 0 differently.
•Since e -x –sin(x) = 0, we have e -x = sin(x).
•Taking gnatural logarithm yields -x = ln(sin(x))
and hence x = -ln(sin(x)) .
•We may use g(x) = -ln(sin(x)).
•However, this approach has two problems:
• If sin(x) = 0, ln(sin(x)) is -∞.
• If sin(x) < 0, ln(sin(x)) is not defined.
•Since sin(x) is periodic, ln(sin(x)) is undefined on
an infinite i number of fintervals (i.e., [π,2π],
[3π,4π], [5π,6π], …). In general, ln(sin(x)) is
undefined on [(2n-1)π, 2nπ], where n ≥ 1.
68

Fixed-Point Iteration Example: 7/9
y
=
x
A plot shows y = g(x)
and y = x at two
points in (0,π). There
are other roots not in (0,π).
x = 0
g( ( x
) =−ln(sin( ( x
))
x = π
Obviously, the derivative
at this larger root is larger
than 1. Hence, the
fixed-point iteration won’t
be able to find it!
Exercise: Plot g(x) in
[0,20] to see other part.
69

Fixed-Point Iteration Example: 8/9
•Let us examine the smaller root in [0.4,0.6] .
•Since g(x) = -ln(sin(x)), g’(x) is (use chain rule):
'
' cos( x
)
g ( x) = ( − ln(sin( x))
= − = −cot( x)
sin( x)
•We have g’(0.4) = -2.36…, g’(0.5) = -1.83…,
g’(0.6) = -1.46….
•Since |g’(x)| > 1 around the root, the fixed-point
iteration diverges and is not be able to find the
desired root.
70

Fixed-Point Iteration Example: 9/9
•If we start with x 0 = 0.4, we have x 1 = 0.9431011,
x 2 = 0.2114828, x 3 = 1.561077, x 4 = 0.0000472799,
x 5 = 9.960947
•Since sin(x 5 ) = sin(9.960947) = -0.51084643, g(x 5 )
= g(9.960947) 960947) = ln(sin(-0.51084643)) is not welldefined,
and the fixed-point iteration fails.
•Hence, g(x) ( ) = -ln(sin(x)) is NOT a correct
transformation.
•You may follow this procedure to do a simple
analysis before using the fixed-point iteration.
71

The End
72