v2009.01.01 - Convex Optimization

4.4. RANK-CONSTRAINED SEMIDEFINITE PROGRAM 279
are simultaneously diagonalizable with G ⋆ . When W = I , as in the trace
heuristic for example, then −W points directly at the origin (the rank-0
PSD matrix). Inner product of an optimization variable with the direction
matrix W is therefore a generalization of the trace heuristic (7.2.2.1); −W
is instead trained toward the boundary of the positive semidefinite cone.
4.4.1.2 convergence
We study convergence to ascertain conditions under which a direction matrix
will reveal a feasible G matrix of rank n or less in semidefinite program (670).
Denote by W ⋆ a particular optimal direction matrix from semidefinite
program (1581a) such that (671) holds. Then we define global convergence
of the iteration (670) (1581a) to correspond with this vanishing vector
inner-product (671) of optimal solutions.
Because this iterative technique for constraining rank is not a projection
method, it can find a rank-n solution G ⋆ ((671) will be satisfied) only if at
least one exists in the feasible set of program (670).
4.4.1.2.1 Proof. Suppose 〈G ⋆ , W 〉= τ is satisfied for some
nonnegative constant τ after any particular iteration (670) (1581a) of the
two minimization problems. Once a particular value of τ is achieved,
it can never be exceeded by subsequent iterations because existence
of feasible G and W having that vector inner-product τ has been
established simultaneously in each problem. Because the infimum of vector
inner-product of two positive semidefinite matrix variables is zero, the
nonincreasing sequence of iterations is thus bounded below hence convergent
because any bounded monotonic sequence in R is convergent. [222,1.2]
[37,1.1] Local convergence to some τ is thereby established.
When a rank-n feasible solution to (670) exists, it remains to be shown
under what conditions 〈G ⋆ , W ⋆ 〉=0 (671) is achieved by iterative solution of
semidefinite programs (670) and (1581a). 4.24 Then pair (G ⋆ , W ⋆ ) becomes
a fixed point of iteration.
A nonexistent feasible rank-n solution would mean certain failure to
converge by definition (671) but, as proved, convex iteration always converges
locally if not globally. Now, an application:
4.24 The case of affine constraints C is considered in [260].