v2009.01.01 - Convex Optimization

E.10. ALTERNATING PROJECTION 701
E.10.3.1
Dykstra’s algorithm
Assume we are given some point b ∈ R n and closed convex sets
{C k ⊂ R n | k=1... L}. Let x ki ∈ R n and y ki ∈ R n respectively denote a
primal and dual vector (whose meaning can be deduced from Figure 148
and Figure 149) associated with set k at iteration i . Initialize
y k0 = 0 ∀k=1... L and x 1,0 = b (1961)
Denoting by P k t the unique minimum-distance projection of t on C k , and
for convenience x L+1,i ∆ = x 1,i−1 , calculation of the iterates x 1i proceeds: E.22
for i=1, 2,...until convergence {
for k=L... 1 {
t = x k+1,i − y k,i−1
x ki = P k t
y ki = P k t − t
}
}
(1962)
Assuming a nonempty intersection, then the iterates converge to the unique
minimum-distance projection of point b on that intersection; [92,9.24]
Pb = lim
i→∞
x 1i (1963)
In the case all the C k are affine, then calculation of y ki is superfluous
and the algorithm becomes identical to alternating projection. [92,9.26]
[123,1] Dykstra’s algorithm is so simple, elegant, and represents such a tiny
increment in computational intensity over alternating projection, it is nearly
always arguably cost effective.
E.10.3.2
Normal cone
Glunt [130,4] observes that the overall effect of Dykstra’s iterative procedure
is to drive t toward the translated normal cone to ⋂ C k at the solution
Pb (translated to Pb). The normal cone gets its name from its graphical
construction; which is, loosely speaking, to draw the outward-normals at Pb
(Definition E.9.1.0.1) to all the convex sets C k touching Pb . The relative
interior of the normal cone subtends these normal vectors.
E.22 We reverse order of projection (k=L...1) in the algorithm for continuity of exposition.