v2007.09.13 - Convex Optimization

200 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONSwith convex problemminimize x T (I − (1 + ǫ) −1 W)xx∈R nsubject to x ∈ C(477)(a)minimize x ⋆T (I − (1 + ǫ) −1 W)x ⋆W ∈S nsubject to 0 ≼ W ≼ ItrW = k≡maximize x ⋆T Wx ⋆W ∈S nsubject to 0 ≼ W ≼ ItrW = k(478)until convergence, where x ⋆ represents an optimal solution of (477) fromany particular iteration. The idea is to optimally solve for the partitionedvariables which are later combined to solve the original problem (476).What makes this approach sound is that the constraints are separable, thepartitioned feasible sets are not interdependent, and the fact that the originalproblem (though nonlinear) is convex simultaneously in both variables. 3.6But partitioning alone does not guarantee a projector. To makeorthogonal projector W a certainty, we must invoke a known analyticaloptimal solution to problem (478): Diagonalize optimal solution fromproblem (477) x ⋆ x ⋆T = ∆ QΛQ T (A.5.2) and set U ⋆ = Q(:, 1:k)∈ R n×kper (1475c);W = U ⋆ U ⋆T = x⋆ x ⋆T‖x ⋆ ‖ 2 + Q(:, 2:k)Q(:, 2:k)T (479)Then optimal solution (x ⋆ , U ⋆ ) to problem (476) is found, for small ǫ , byiterating solution to problem (477) with optimal (projector) solution (479)to convex problem (478).Proof. Optimal vector x ⋆ is orthogonal to the last n −1 columns oforthogonal matrix Q , sof ⋆ (477) = ‖x⋆ ‖ 2 (1 − (1 + ǫ) −1 ) (480)after each iteration. Convergence of f ⋆ is proven with the observation(477)that iteration (477) (478a) is a nonincreasing sequence that is bounded belowby 0. Any bounded monotonic sequence in R is convergent. [188,1.2]3.6 A convex problem has convex feasible set, and the objective surface has one and onlyone global minimum.