Model Predictive Control

7.3 Solution via Recursive Approach 121
cost q(xj−1, uj−1) and the optimal cost-to-go J ∗ j→N (xj) from time j onwards.
J ∗ j−1→N (xj−1) = min
uj−1
q(xj−1, uj−1) + J ∗ j→N (xj)
subj. to xj = g(xj−1, uj−1)
h(xj−1, uj−1) ≤ 0
xj ∈ Xj→N
(7.13)
Here the only decision variable left for the optimization is uj−1, the input at time
j − 1. All the other inputs u∗ j , . . .,u∗ N−1 have already been selected optimally to
yield the optimal cost-to-go J ∗ j→N (xj). We can rewrite (7.13) as
J ∗ j−1→N (xj−1) = min
uj−1
q(xj−1, uj−1) + J ∗ j→N (g(xj−1, uj−1))
subj. to h(xj−1, uj−1) ≤ 0
g(xj−1, uj−1) ∈ Xj→N,
(7.14)
making the dependence of xj on the initial state xj−1 explicit.
The optimization problem (7.14) suggests the following recursive algorithm
backwards in time to determine the optimal control law. We start with the terminal
cost and constraint
and then proceed backwards
J ∗ N−1→N (xN−1) = min
uN −1
.
J ∗ 0→N (x0) = min
u0
J ∗ N→N(xN) = p(xN) (7.15)
XN→N = Xf, (7.16)
q(xN−1, uN−1) + J ∗ N→N (g(xN−1, uN−1))
subj. to h(xN−1, uN−1) ≤ 0,
g(xN−1, uN−1) ∈ XN→N
q(x0, u0) + J ∗ 1→N (g(x0, u0))
subj. to h(x0, u0) ≤ 0,
g(x0, u0) ∈ X1→N
x0 = x(0).
(7.17)
This algorithm, popularized by Bellman, is referred to as dynamic programming.
The dynamic programming problem is appealing because it can be stated compactly
and because at each step the optimization takes place over one element uj of the
optimization vector only. This optimization is rather complex, however. It is not
a standard nonlinear programming problem but we have to construct the optimal
cost-to-go J ∗ j→N (xj), a function defined over the subset Xj→N of the state space.
In a few special cases we know the type of function and we can find it efficiently.
For example, in the next chapter we will cover the case when the system is linear
and the cost is quadratic. Then the optimal cost-to-go is also quadratic and can be
constructed rather easily. Later in the book we will show that, when constraints
are added to this problem, the optimal cost-to-go becomes piecewise quadratic and
efficient algorithms for its construction are also available.