v2010.10.26 - Convex Optimization

174 CHAPTER 2. CONVEX GEOMETRYthat follows from a necessary and sufficient condition for projection on convexset C (Theorem E.9.1.0.2)P(x ⋆ −∇f(x ⋆ )) ∈ C , 〈x ⋆ −∇f(x ⋆ )−x ⋆ , x−x ⋆ 〉 ≤ 0 ∀x ∈ C (2010)Proof of equivalence [Wıκımization, Complementarity problem] is providedby Németh. Given minimum-distance projection problem1minimize ‖x − y‖2x 2subject to x ∈ C(354)on convex feasible set C for example, the equivalent fixed point problemx = P C (x − ∇f(x)) = P C (y) (355)is solved in one step.In the unconstrained case (C = R n ), optimality condition (352) reduces tothe classical rule (p.252)∇f(x ⋆ ) = 0, x ⋆ ∈ domf (356)which can be inferred from the following application:2.13.3.0.1 Example. Optimality for equality constrained problem.Given a real differentiable convex function f(x) : R n →R defined ondomain R n , a fat full-rank matrix C ∈ R p×n , and vector d∈ R p , the convexoptimization problemminimize f(x)x(357)subject to Cx = dis characterized by the well-known necessary and sufficient optimalitycondition [61,4.2.3]∇f(x ⋆ ) + C T ν = 0 (358)where ν ∈ R p is the eminent Lagrange multiplier. [305] [250, p.188] [229] Inother words, solution x ⋆ is optimal if and only if ∇f(x ⋆ ) belongs to R(C T ).Via membership relation, we now derive condition (358) from the generalfirst-order condition for optimality (352): For problem (357)C {x∈ R n | Cx = d} = {Zξ + x p | ξ ∈ R n−rank C } (359)