v2010.10.26 - Convex Optimization

4.4. RANK-CONSTRAINED SEMIDEFINITE PROGRAM 3134.4.1.2.1 Proof. Suppose 〈G ⋆ , W 〉= τ is satisfied for some nonnegativeconstant τ after any particular iteration (736) (1700a) of the twominimization problems. Once a particular value of τ is achieved, it can neverbe exceeded by subsequent iterations because existence of feasible G and Whaving that vector inner-product τ has been established simultaneously ineach problem. Because the infimum of vector inner-product of two positivesemidefinite matrix variables is zero, the nonincreasing sequence of iterationsis thus bounded below hence convergent because any bounded monotonicsequence in R is convergent. [258,1.2] [42,1.1] Local convergence to somenonnegative objective value τ is thereby established.Local convergence, in this context, means convergence to a fixed point ofpossibly infeasible rank. Only local convergence can be established becauseobjective 〈G , W 〉 , when instead regarded simultaneously in two variables(G , W ) , is generally multimodal. (3.8.0.0.3)Local convergence, convergence to τ ≠ 0 and definition of a stall, neverimplies nonexistence of a rank-n feasible solution to (736). A nonexistentrank-n feasible solution would mean certain failure to converge globally bydefinition (737) (convergence to τ ≠ 0) but, as proved, convex iteration alwaysconverges locally if not globally.When a rank-n feasible solution to (736) exists, it remains an openproblem to state conditions under which 〈G ⋆ , W ⋆ 〉=τ =0 (737) is achievedby iterative solution of semidefinite programs (736) and (1700a). ThenrankG ⋆ ≤ n and pair (G ⋆ , W ⋆ ) becomes a globally optimal fixed point ofiteration. There can be no proof of global convergence because of theimplicit high-dimensional multimodal manifold in variables (G , W). Whenstall occurs, direction vector W can be manipulated to steer out; e.g.,reversal of search direction as in Example 4.6.0.0.1, or reinitialization to arandom rank-(N − n) matrix in the same positive semidefinite cone face(2.9.2.3) demanded by the current iterate: given ordered diagonalizationG ⋆ = QΛQ T ∈ S N , then W = U ⋆ ΦU ⋆T where U ⋆ = Q(:, n+1:N)∈ R N×N−nand where eigenvalue vector λ(W) 1:N−n = λ(Φ) has nonnegative uniformlydistributed random entries in (0, 1] by selection of Φ∈ S N−n+ whileλ(W) N−n+1:N = 0. When this direction works, rank with respect to iterationtends to be noisily monotonic. Zero eigenvalues act as memory whilerandomness largely reduces likelihood of stall.