v2009.01.01 - Convex Optimization

3.1. CONVEX FUNCTION 217
We learned from Example 3.1.7.0.4 that f(W , x)= ǫx T (W +ǫI) −1 x is
convex simultaneously in both variables over all x ∈ R n when W ∈ S n + is
confined to the entire positive semidefinite cone (including its boundary). It
is now our goal to incorporate f into an optimization problem such that
an optimal solution returned always comprises a projection matrix W . The
set of orthogonal projection matrices is a nonconvex subset of the positive
semidefinite cone. So f cannot be convex on the projection matrices, and
its equivalent (for idempotent W )
f(W , x) = x T( I − (1 + ǫ) −1 W ) x (509)
cannot be convex simultaneously in both variables on either the positive
semidefinite or symmetric projection matrices.
Suppose we allow domf to constitute the entire positive semidefinite
cone but constrain W to a Fantope (82); e.g., for convex set C and 0 < k < n
as in
minimize ǫx T (W + ǫI) −1 x
x∈R n , W ∈S n
subject to 0 ≼ W ≼ I
(510)
trW = k
x ∈ C
Although this is a convex problem, there is no guarantee that optimal W is
a projection matrix because only extreme points of a Fantope are orthogonal
projection matrices UU T .
Let’s try partitioning the problem into two convex parts (one for x and
one for W), substitute equivalence (507), and then iterate solution of convex
problem
minimize x T (I − (1 + ǫ) −1 W)x
x∈R n
(511)
subject to x ∈ C
with convex problem
(a)
minimize x ⋆T (I − (1 + ǫ) −1 W)x ⋆
W ∈S n
subject to 0 ≼ W ≼ I
trW = k
≡
maximize x ⋆T Wx ⋆
W ∈S n
subject to 0 ≼ W ≼ I
trW = k
(512)
until convergence, where x ⋆ represents an optimal solution of (511) from
any particular iteration. The idea is to optimally solve for the partitioned