MA409 Continuous-Time Optimisation

Summer 2009 examination
MA409
Continuous-time Optimisation
Half Unit
Suitable for all candidates
Instructions to candidates
Time allowed: 2 hours
This examination paper contains 6 questions. You may attempt as many questions as you wish,
but only your best 4 questions will count towards the final mark. All questions carry equal
numbers of marks.
Please write your answers in dark ink (preferably black or blue) only.
Calculators are not allowed in this exam.
You are supplied with: Maths Answer Booklet
c○ LSE 2009 / MA409 Page 1 of 7

1.
(a) State the Fundamental Lemma of the Calculus of Variations.
(b) Let f : R 3 → R be a continuous function with continuous partial derivatives with respect to
the first and second argument.
Consider the problem of extremizing the functional
F(x) =
∫ T
0
f (x(t),ẋ(t), t) dt,
subject to the end-point restrictions x(0) = a, x(T ) = b, where a,b are constants and x :
[0,T ] → R is a continuously differentiable function.
(i) Use the Fundamental Lemma to derive the Euler-Lagrange equation corresponding to this
problem.
(ii) If the end-point value x(T ) is unrestricted, deduce a condition satisfied by x(T ).
(c) Consider the problem of finding the continuously differentiable function minimizing the
functional
subject to x(0) = 1.
F(x) =
∫ T
0
(
x(t) 4 + x(t) 2 ẋ(t) 2) dt,
Write down the first integral of the Euler-Lagrange equation for this problem and hence, using
(c)(ii), solve the problem.
c○ LSE 2009 / MA409 Page 2 of 7

2.
(a) Let X be a normed vector space of continuously differentiable functions on [0,1]. Let the
function f : R 3 → R have continuous partial derivatives.
Consider the problem of extremizing over X
F(x) =
∫ 1
subject to G(x) = 0,
0
f (x,ẋ,t)dt,
where G is a differentiable function from X to a normed vector space Y . Suppose that the
relative stationarity condition holds at x = ξ , that is, for each h in X
DG(ξ )h = 0 ⇒ DF(ξ )h = 0.
Deduce a Lagrange Multiplier Theorem. You should define any duality notions which you call
upon.
(b) Let X comprise the continuously differentiable functions (x(t),y(t)) on [0,1], i.e. with
values in R 2 , and Y = R.
Apply the method of Lagrange multipliers to solve the constrained problem of maximizing
∫ 1
where r > 0 is a constant, subject to
with x(t) ≥ 0,y(t) ≥ 0.
0
e −rt (x(t) 1/2 + 2y(t) 1/2 ) dt,
∫ 1
0
(x(t) + y(t)) dt = 1,
c○ LSE 2009 / MA409 Page 3 of 7

3.
A dynamical system has governing equation:
ẍ = 4ẋ − 3x + u, |u| ≤ 1.
(a) State the Pontryagin Principle in a form suited to deriving the minimum time trajectory
taking the system from a given initial state to rest at the origin (i.e. x = 0, ẋ = 0).
(b) Use the Pontryagin Principle to show that the time-optimal control is of ‘bang-bang type’
and that at most one switch of control takes place.
(c) Find the singular points in the (x,ẋ) phase plane corresponding to the two constant controls
u = ±1, and the linear trajectories through them. By considering the eigenvalues of the
associated first-order formulation, say what shape of trajectories to expect in general.
(d) Sketch the trajectories u = ±1 which pass through the origin of the phase plane.
(e) Sketch the switching curve and indicate how it is used to obtain optimal trajectories from
controllable initial states (x,ẋ) in the phase plane.
c○ LSE 2009 / MA409 Page 4 of 7

4.
Consider the problem of finding
∫ T
S(c,T ) = min{ (x 4 + x 2 ẋ 2 ) dt : x is continuously differentiable on [0,T ] and x(0) = c}.
0
(a) Show that the Bellman equation for the problem is
0 = min{(c 4 + c 2 v 2 ) + v ∂S
v
∂c − ∂ S
∂T }.
(b) Verify that S(c,T ) = c 4 S(1,T ) and hence find S(1,T ).
(c) Deduce that
lim S(1,T ) = 1
T →∞ 2 .
(d) Consider now the problem of finding
∫ ∞
V (c) = min{ (x 4 + x 2 ẋ 2 ) dt : x is continuously differentiable on [0,∞) and x(0) = c}.
0
Deduce from (c) that the optimal trajectory satisfies ẋ = −x and hence find the optimal trajectory
assuming x(0) = 1.
c○ LSE 2009 / MA409 Page 5 of 7

5.
Consider the problem of finding, for t ≥ 0,
∫ ∞
C(x,t) = min{E[ e −ρs (aXs 2 + bu 2 s )ds] : u s is continuous for s ≥ t},
t
where for s ≥ t the stochastic process X s is defined by the initial condition X t = x and the
equation
dX s = u s ds + σX s dz s ,
with z s a standard Wiener process, and a,b,ρ,σ positive constants.
(a) Assuming that the function C is twice continuously differentiable, show that C satisfies the
Hamilton-Jacobi-Bellman equation.
ρC(x,0) = min[ax 2 + bu 2 + u ∂C(x,0) + 1
u
∂x 2 σ 2 x 2 ∂ 2 C(x,0)
∂x 2 ].
(b) Find C(x,0), assuming it is of the form C(x,0) = Ax γ for some A and γ. You should identify
A and γ.
(c) Express the value function of the optimization problem, C(x,t), in terms of C(x,0).
c○ LSE 2009 / MA409 Page 6 of 7

6.
Let C [0,1] denote the normed vector space of continuous functions on [0,1] with values in
R with the maximum norm ||x|| ∞ . Let C 1 [0,1] denote the subspace of C [0,1] consisting of
functions x whose derivative ẋ is continuous. Define a norm on C 1 [0,1] as follows:
||x|| 1 := ||x|| ∞ + ||ẋ|| ∞ .
(a) Let F be a function from C 1 [0,1] to C [0,1]. Define (i) the Gateaux derivative D h F(x)
(the directional derivative) in direction h, and (ii) the Fréchet derivative (the strong derivative)
DF(x). Show that if the strong derivative exists, then for all h in C 1 [0,1]
D h F(x) = DF(x)h.
(b) For the function S : C 1 [0,1] → C [0,1] defined by
S(x)(t) = ẋ(t) 3 ,
explain why the Gateaux derivative D h S(x) is the function y such that y(t) = 3ẋ(t) 2 ḣ(t).
Show further that if ||x|| 1 ≤ 1 and ||h|| 1 ≤ ε < 1, then
||S(x + h) − S(x) − D h S(x)|| ∞ ≤ 4ε||h|| 1 .
Conclude that the function S(x) has a strong derivative.
(c) The real-valued functions G and F are defined on C 1 [0,1] × R by
G(x,τ) = (x(0),x(τ) − q(τ)),
F(x,τ) =
∫ τ
0
f (x,ẋ,t)dt,
where the function q : R → R is differentiable and the function f : R 3 → R is continuous, and
differentiable in its first two arguments.
(i) Let u denote a vector (h,σ) in C 1 [0,1]×R with 0 < σ < 1. Suppose that 0 < τ < 1 and that
x is in C 1 [0,1]. Compute the directional derivatives D u G(x,τ) and D u F(x,τ).
(ii) For the problem of maximizing F on C 1 [0,1] × R subject to G = 0, it is known that at the
optimum (x,τ) the condition
DG(x,τ) = 0 =⇒ DF(x,τ) = 0,
holds. Deduce the transversality condition at τ :
f − (ẋ − ˙q) fẋ = 0.
You may assume that for an optimal pair (x,τ), the Euler-Lagrange equation is satisfied for
0 ≤ t ≤ τ.
c○ LSE 2009 / MA409 Page 7 of 7