v2009.01.01 - Convex Optimization

218 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS
variables which are later combined to solve the original problem (510).
What makes this approach sound is that the constraints are separable, the
partitioned feasible sets are not interdependent, and the fact that the original
problem (though nonlinear) is convex simultaneously in both variables. 3.11
But partitioning alone does not guarantee a projector. To make
orthogonal projector W a certainty, we must invoke a known analytical
optimal solution to problem (512): Diagonalize optimal solution from
problem (511) x ⋆ x ⋆T = ∆ QΛQ T (A.5.2) and set U ⋆ = Q(:, 1:k)∈ R n×k
per (1581c);
W = U ⋆ U ⋆T = x⋆ x ⋆T
‖x ⋆ ‖ 2 + Q(:, 2:k)Q(:, 2:k)T (513)
Then optimal solution (x ⋆ , U ⋆ ) to problem (510) is found, for small ǫ , by
iterating solution to problem (511) with optimal (projector) solution (513)
to convex problem (512).
Proof. Optimal vector x ⋆ is orthogonal to the last n −1 columns of
orthogonal matrix Q , so
f ⋆ (511) = ‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) (514)
after each iteration. Convergence of f(511) ⋆ is proven with the observation that
iteration (511) (512a) is a nonincreasing sequence that is bounded below by 0.
Any bounded monotonic sequence in R is convergent. [222,1.2] [37,1.1]
Expression (513) for optimal projector W holds at each iteration, therefore
‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) must also represent the optimal objective value f(511)
⋆
at convergence.
Because the objective f (510) from problem (510) is also bounded below
by 0 on the same domain, this convergent optimal objective value f(511) ⋆ (for
positive ǫ arbitrarily close to 0) is necessarily optimal for (510); id est,
by (1563), and
f ⋆ (511) ≥ f ⋆ (510) ≥ 0 (515)
lim
ǫ→0 +f⋆ (511) = 0 (516)
3.11 A convex problem has convex feasible set, and the objective surface has one and only
one global minimum.