v2007.09.13 - Convex Optimization

2.13. DUAL CONE & GENERALIZED INEQUALITY 1512.13.3.0.1 Example. Equality constrained problem.Given a real differentiable convex function f(x) : R n →R defined ondomain R n , a fat full-rank matrix C ∈ R p×n , and vector d∈ R p , the convexoptimization problemminimize f(x)xsubject to Cx = d(309)is characterized by the well-known necessary and sufficient optimalitycondition [46,4.2.3]∇f(x ⋆ ) + C T ν = 0 (310)where ν ∈ R p is the eminent Lagrange multiplier. [226] Feasible solution x ⋆is optimal, in other words, if and only if ∇f(x ⋆ ) belongs to R(C T ). Viamembership relation, we now derive this particular condition from the generalfirst-order condition for optimality (308):In this case, the feasible set isC ∆ = {x∈ R n | Cx = d} = {Zξ + x p | ξ ∈ R n−rank C } (311)where Z ∈ R n×n−rank C holds basis N(C) columnar, and x p is any particularsolution to Cx = d . Since x ⋆ ∈ C , we arbitrarily choose x p = x ⋆ whichyields the equivalent optimality condition∇f(x ⋆ ) T Zξ ≥ 0 ∀ξ∈ R n−rank C (312)But this is simply half of a membership relation, and the cone dual toR n−rank C is the origin in R n−rank C . We must therefore haveZ T ∇f(x ⋆ ) = 0 ⇔ ∇f(x ⋆ ) T Zξ ≥ 0 ∀ξ∈ R n−rank C (313)meaning, ∇f(x ⋆ ) must be orthogonal to N(C). This conditionZ T ∇f(x ⋆ ) = 0, x ⋆ ∈ C (314)is necessary and sufficient for optimality of x ⋆ .