Chapter 3 Geometry of convex functions - Meboo Publishing ...

238 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS
partitioned feasible sets are not interdependent, and the fact that the original
problem (though nonlinear) is convex simultaneously in both variables. 3.14
But partitioning alone does not guarantee a projector. To make
orthogonal projector W a certainty, we must invoke a known analytical
optimal solution to problem (553): Diagonalize optimal solution from
problem (552) x ⋆ x ⋆T QΛQ T (A.5.1) and set U ⋆ = Q(:, 1:k)∈ R n×k
per (1663c);
W = U ⋆ U ⋆T = x⋆ x ⋆T
‖x ⋆ ‖ 2 + Q(:, 2:k)Q(:, 2:k)T (554)
Then optimal solution (x ⋆ , U ⋆ ) to problem (551) is found, for small ǫ , by
iterating solution to problem (552) with optimal (projector) solution (554)
to convex problem (553).
Proof. Optimal vector x ⋆ is orthogonal to the last n −1 columns of
orthogonal matrix Q , so
f ⋆ (552) = ‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) (555)
after each iteration. Convergence of f(552) ⋆ is proven with the observation that
iteration (552) (553a) is a nonincreasing sequence that is bounded below by 0.
Any bounded monotonic sequence in R is convergent. [252,1.2] [41,1.1]
Expression (554) for optimal projector W holds at each iteration, therefore
‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) must also represent the optimal objective value f(552)
⋆
at convergence.
Because the objective f (551) from problem (551) is also bounded below
by 0 on the same domain, this convergent optimal objective value f(552) ⋆ (for
positive ǫ arbitrarily close to 0) is necessarily optimal for (551); id est,
by (1646), and
f ⋆ (552) ≥ f ⋆ (551) ≥ 0 (556)
lim
ǫ→0 +f⋆ (552) = 0 (557)
Since optimal (x ⋆ , U ⋆ ) from problem (552) is feasible to problem (551), and
because their objectives are equivalent for projectors by (548), then converged
(x ⋆ , U ⋆ ) must also be optimal to (551) in the limit. Because problem (551)
is convex, this represents a globally optimal solution.
3.14 A convex problem has convex feasible set, and the objective surface has one and only
one global minimum. [296, p.123]