v2010.10.26 - Convex Optimization

3.5. EPIGRAPH, SUBLEVEL SET 247partitioned feasible sets are not interdependent, and the fact that the originalproblem (though nonlinear) is convex simultaneously in both variables. 3.16But partitioning alone does not guarantee a projector as solution.To make orthogonal projector W a certainty, we must invoke a knownanalytical optimal solution to problem (575): Diagonalize optimal solutionfrom problem (574) x ⋆ x ⋆T QΛQ T (A.5.1) and set U ⋆ = Q(:, 1:k)∈ R n×kper (1700c);W = U ⋆ U ⋆T = x⋆ x ⋆T‖x ⋆ ‖ 2 + Q(:, 2:k)Q(:, 2:k)T (576)Then optimal solution (x ⋆ , U ⋆ ) to problem (573) is found, for small ǫ , byiterating solution to problem (574) with optimal (projector) solution (576)to convex problem (575).Proof. Optimal vector x ⋆ is orthogonal to the last n −1 columns oforthogonal matrix Q , sof ⋆ (574) = ‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) (577)after each iteration. Convergence of f(574) ⋆ is proven with the observation thatiteration (574) (575a) is a nonincreasing sequence that is bounded below by 0.Any bounded monotonic sequence in R is convergent. [258,1.2] [42,1.1]Expression (576) for optimal projector W holds at each iteration, therefore‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) must also represent the optimal objective value f(574)⋆at convergence.Because the objective f (573) from problem (573) is also bounded belowby 0 on the same domain, this convergent optimal objective value f(574) ⋆ (forpositive ǫ arbitrarily close to 0) is necessarily optimal for (573); id est,by (1683), andf ⋆ (574) ≥ f ⋆ (573) ≥ 0 (578)limǫ→0 +f⋆ (574) = 0 (579)Since optimal (x ⋆ , U ⋆ ) from problem (574) is feasible to problem (573), andbecause their objectives are equivalent for projectors by (570), then converged(x ⋆ , U ⋆ ) must also be optimal to (573) in the limit. Because problem (573)is convex, this represents a globally optimal solution.3.16 A convex problem has convex feasible set, and the objective surface has one and onlyone global minimum. [302, p.123]