Errors and Uncertainties in Computations Errors and Uncertainties ...

Errors and Uncertainties in
Computations
Rubin H Landau
With
Sally Haerer and Scott Clark
Computational Physics for Undergraduates
BS Degree Program: Oregon State University
“Engaging People in Cyber Infrastructure”
Support by EPICS/NSF & OSU
Introductory Computational Science 1
© Rubin Landau, EPIC/OSU 2005

Problem: Life + Errors (Uncertainties)
Always part of computation
Finite precision uncertainties = “errors”
Don't be afraid; Don't play with garbage
Errors accumulate with steps U i
start → U1 → U2 → ...→ Un
→ end
(1)
• p = probability U i correct
• P = p n = probability n steps correct
• n= 1000, p = 0.9993 P = 1/2
• (whoops!)
Introductory Computational Science 2
© Rubin Landau, EPIC/OSU 2005

Theory: Types of Errors (4 plagues)
1. Blunders: typos, wrong program, wrong data, …
2. Random errors: electronic fluctuations, cosmic rays, …
– rare, but if 10 8 steps?
– can't control
– repeat calculation
3. Approximation errors, algorithm, approx math:
e −x =
∞X
n=0
(−x) n
n!
'
NX
n=0
(−x) n
n!
= e −x + E(x, N)
(2)
• decreases as N increases
• vanishes as N
4. Round-Off Errors (cont)
Introductory Computational Science 3
© Rubin Landau, EPIC/OSU 2005

Errors (cont)--
Roundoff errors:
Because: numbers via finite # bits
≈ uncertainties in measurement
Some numbers represent exact (2 n )
Accumulates with steps unstable
garbage: RO ≈ result:
2 ¡ 1
3
¢
−
2
3
=0.66666 − 0.66667 = −0.00001 6=0
(3)
Significant figures
• a = 11223344556677889900
= 1.12233445566778899 * 10 19
(4)
• exponent: stored separately
small full precision
Most significant part: 1.12233
Least significant part: 44556677
• error prone
Introductory Computational Science 4
© Rubin Landau, EPIC/OSU 2005

Disaster Model: Subtractive Cancellation
If you subtract two large numbers and end up with a small
result, there will be less significance in the small result
a = b − c ⇒ a c = b c − c c
(5)
a c = b(1 + ² b ) − c(1 + ² c )
(6)
e −x = 1 −x + x2
2 − x3
6 +···= 1 −100 +5000 +··· (9)
⇒
a c
a = 1 + ² b
b
a − c a ² c
(7)
' 1 − b a (² b − ² c ) → ∞
(8)
HW: Cancellation in power series, x = 100:
Introductory Computational Science 5
© Rubin Landau, EPIC/OSU 2005

Model for Multiplicative Errors
• Propagation of error inmultiplication
a = b × c ⇒ a c = b c × c c ,
⇒
a c
a =(1 + ² b)(1 + ² c ) ' 1 + ² b + ² c
(10)
(11)
• How add errors? |² b | + |² c |, |² b | − |² c |?
• Algorithm Model: N steps random walk (Chap.5)
² ro ≈ √ N² m
N
• Each step ' machine precision
If non random: ² ' N² m , N!² m
E.G.: several hour calculation, 10 10 Flops
⇒ ² ' 10 7 ² m
Singles: ² m ' 10 −7
⇒ ² ' 1 (whoops!)
Introductory Computational Science 6
© Rubin Landau, EPIC/OSU 2005
R
1
4
2 3

Experiment: Determine Errors
1. Basic algorithm questions:
a) does it converge?
b) if not, quit
c) how precise are converged results?
2. Converged correct
3. How $$ (time consuming) ?
Introductory Computational Science 7
© Rubin Landau, EPIC/OSU 2005

Experiment: Expected Behavior
• ² apprx
= approximation error = A ns exact -A
ns algorithm
• Algorithmic errordecreases rapidly
² aprx ' α → 0, (N → ∞),
N
β
(12)
• Round oFF error increases slowly
² RO ' √ N ² m
(13)
• Want minimum of sum
² tot = ² approx + ² RO ,
α
'
N + √ N² β
m
(14)
• Want N small ⇒ faster, accurate
Introductory Computational Science 8
© Rubin Landau, EPIC/OSU 2005

Experimental Approach
Determine & via applied math
(see text)
10 -1
Min via comparison to known
answer
• test (similar) problem
If N looks good, assume 2N exact:
A(N) ≈ A + α
N β + RO
² tot = A(N) − A(2N) ≈ α
N β
Plot log 10 () vs log 10 (N)
• ordinate = # decimal places precision
• slope =
|error|
10 -3
10 -5
10 -7
10 -9
Gaussian
Simpson
trapezoid
10 100
N
Look & understand!
converge diverge
Introductory Computational Science 9
© Rubin Landau, EPIC/OSU 2005

Time for Exercises
in Lab
Introductory Computational Science 10
© Rubin Landau, EPIC/OSU 2005

Exercise: Subtractive Cancellation
1. Equivalent solutions to
ax 2 + bx + c =0
x 1,2 = −b ± √ b 2 − 4ac
, x 0 1,2 =
2a
−2c
b ± √ b 2 − 4ac .
Subtractive cancellation if 4ac

Subtractive Exercise (cont)
2. Three equivalent (math) sums:
S (1)
N
=
S (2)
N
=
S (3)
N
=
2NX
n=1
NX
n=1
NX
n=1
(−1) n
2n
2n +1 −
n
n +1 . (1)
1
2n(2n +1)
N X
n=1
2n − 1
2n
(even & odd), (2)
(partial sums). (3)
Write program to calculate S(1), S(2), S(3) 1 N 100
Assume S(3) is exact
Log-log plot relative error: log 10 | S(1) N
−S(3) N
| vs log 10
(N)
Get expected straight line?
S (3)
N
Introductory Computational Science Lab 2
© Rubin Landau, EPIC/OSU 2005

Exercise: error in e -x
(cont)
3. Mathematical definition versus algorithm
e −x =1− x + x2
2! − x3
3! + ··· ' N X
n=0
(−x) n
n!
(1)
a. Examine cancellation of terms for x ≈ 10
b. Convergence terms decrease
new = (n+1)/x old n+1 ≈ x
c. Demo: near-perfect cancellation here?
d. exp(-x) = 1/exp(x) better, yet still RO
e. Plot error vs N for 1 x 100 (see easyPtPlot.java.)
Introductory Computational Science Lab 5
© Rubin Landau, EPIC/OSU 2005