v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
278 CHAPTER 4. SEMIDEFINITE PROGRAMMING N∑ λ(G ⋆ ) i i=n+1 = minimize W ∈ S N 〈G ⋆ , W 〉 subject to 0 ≼ W ≼ I trW = N − n (1581a) whose feasible set is a Fantope (2.3.2.0.1), and where G ⋆ is an optimal solution to problem (670) given some iterate W . The idea is to iterate solution of (670) and (1581a) until convergence as defined in4.4.1.2. 4.22 Optimal matrix W ⋆ is defined as any direction matrix yielding optimal solution G ⋆ of rank n or less to then convex equivalent (670) of feasibility problem (669); id est, any direction matrix for which the last N − n nonincreasingly ordered eigenvalues λ of G ⋆ are zero: (p.599) N∑ λ(G ⋆ ) i = 〈G ⋆ , W ⋆ 〉 = ∆ 0 (671) i=n+1 We emphasize that convex problem (670) is not a relaxation of the rank-constrained feasibility problem (669); at convergence, convex iteration (670) (1581a) makes it instead an equivalent problem. 4.23 4.4.1.1 direction interpretation We make no assumption regarding uniqueness of direction matrix W . The feasible set of direction matrices in (1581a) is the convex hull of outer product of all rank-(N − n) orthonormal matrices; videlicet, conv { UU T | U ∈ R N×N−n , U T U = I } = { A∈ S N | I ≽ A ≽ 0, 〈I , A 〉= N − n } (82) Set {UU T | U ∈ R N×N−n , U T U = I} comprises the extreme points of this Fantope (82). An optimal solution W to (1581a), that is an extreme point, is known in closed form (p.599). By (205), that particular direction (−W) can be regarded as pointing toward positive semidefinite rank-n matrices that 4.22 The proposed iteration is not an alternating projection. (confer Figure 140) 4.23 Terminology equivalent problem meaning, optimal solution to one problem can be derived from optimal solution to another. Terminology same problem means: optimal solution set for one problem is identical to the optimal solution set of another (without transformation).
4.4. RANK-CONSTRAINED SEMIDEFINITE PROGRAM 279 are simultaneously diagonalizable with G ⋆ . When W = I , as in the trace heuristic for example, then −W points directly at the origin (the rank-0 PSD matrix). Inner product of an optimization variable with the direction matrix W is therefore a generalization of the trace heuristic (7.2.2.1); −W is instead trained toward the boundary of the positive semidefinite cone. 4.4.1.2 convergence We study convergence to ascertain conditions under which a direction matrix will reveal a feasible G matrix of rank n or less in semidefinite program (670). Denote by W ⋆ a particular optimal direction matrix from semidefinite program (1581a) such that (671) holds. Then we define global convergence of the iteration (670) (1581a) to correspond with this vanishing vector inner-product (671) of optimal solutions. Because this iterative technique for constraining rank is not a projection method, it can find a rank-n solution G ⋆ ((671) will be satisfied) only if at least one exists in the feasible set of program (670). 4.4.1.2.1 Proof. Suppose 〈G ⋆ , W 〉= τ is satisfied for some nonnegative constant τ after any particular iteration (670) (1581a) of the two minimization problems. Once a particular value of τ is achieved, it can never be exceeded by subsequent iterations because existence of feasible G and W having that vector inner-product τ has been established simultaneously in each problem. Because the infimum of vector inner-product of two positive semidefinite matrix variables is zero, the nonincreasing sequence of iterations is thus bounded below hence convergent because any bounded monotonic sequence in R is convergent. [222,1.2] [37,1.1] Local convergence to some τ is thereby established. When a rank-n feasible solution to (670) exists, it remains to be shown under what conditions 〈G ⋆ , W ⋆ 〉=0 (671) is achieved by iterative solution of semidefinite programs (670) and (1581a). 4.24 Then pair (G ⋆ , W ⋆ ) becomes a fixed point of iteration. A nonexistent feasible rank-n solution would mean certain failure to converge by definition (671) but, as proved, convex iteration always converges locally if not globally. Now, an application: 4.24 The case of affine constraints C is considered in [260].
- Page 227 and 228: 3.1. CONVEX FUNCTION 227 For vector
- Page 229 and 230: 3.1. CONVEX FUNCTION 229 This means
- Page 231 and 232: 3.1. CONVEX FUNCTION 231 f(Y ) −
- Page 233 and 234: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 235 and 236: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 237 and 238: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 239 and 240: 3.3. QUASICONVEX 239 exponential al
- Page 241 and 242: 3.3. QUASICONVEX 241 Unlike convex
- Page 243 and 244: 3.4. SALIENT PROPERTIES 243 6. (af
- Page 245 and 246: Chapter 4 Semidefinite programming
- Page 247 and 248: 4.1. CONIC PROBLEM 247 where K is a
- Page 249 and 250: 4.1. CONIC PROBLEM 249 4.1.1.2 Redu
- Page 251 and 252: 4.1. CONIC PROBLEM 251 In any SDP f
- Page 253 and 254: 4.1. CONIC PROBLEM 253 Proposition
- Page 255 and 256: 4.2. FRAMEWORK 255 sets are closed
- Page 257 and 258: 4.2. FRAMEWORK 257 4.2.1.1.3 Exampl
- Page 259 and 260: 4.2. FRAMEWORK 259 4.2.2 Duals The
- Page 261 and 262: 4.2. FRAMEWORK 261 When equality is
- Page 263 and 264: 4.2. FRAMEWORK 263 The pseudoinvers
- Page 265 and 266: 4.2. FRAMEWORK 265 For the data giv
- Page 267 and 268: 4.2. FRAMEWORK 267 minimizes an aff
- Page 269 and 270: 4.3. RANK REDUCTION 269 whose rank
- Page 271 and 272: 4.3. RANK REDUCTION 271 and where m
- Page 273 and 274: 4.3. RANK REDUCTION 273 4.3.3 Optim
- Page 275 and 276: 4.3. RANK REDUCTION 275 Initialize:
- Page 277: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 281 and 282: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 283 and 284: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 285 and 286: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 287 and 288: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 289 and 290: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 291 and 292: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 293 and 294: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 295 and 296: 4.5. CONSTRAINING CARDINALITY 295 m
- Page 297 and 298: 4.5. CONSTRAINING CARDINALITY 297 m
- Page 299 and 300: 4.5. CONSTRAINING CARDINALITY 299 a
- Page 301 and 302: 4.5. CONSTRAINING CARDINALITY 301 f
- Page 303 and 304: 4.5. CONSTRAINING CARDINALITY 303 n
- Page 305 and 306: 4.5. CONSTRAINING CARDINALITY 305 W
- Page 307 and 308: 4.5. CONSTRAINING CARDINALITY 307 t
- Page 309 and 310: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 311 and 312: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 313 and 314: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 315 and 316: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 317 and 318: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 319 and 320: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 321 and 322: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 323 and 324: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 325 and 326: 4.6. CARDINALITY AND RANK CONSTRAIN
- Page 327 and 328: 4.6. CARDINALITY AND RANK CONSTRAIN
278 CHAPTER 4. SEMIDEFINITE PROGRAMMING<br />
N∑<br />
λ(G ⋆ ) i<br />
i=n+1<br />
= minimize<br />
W ∈ S N 〈G ⋆ , W 〉<br />
subject to 0 ≼ W ≼ I<br />
trW = N − n<br />
(1581a)<br />
whose feasible set is a Fantope (2.3.2.0.1), and where G ⋆ is an optimal<br />
solution to problem (670) given some iterate W . The idea is to iterate<br />
solution of (670) and (1581a) until convergence as defined in4.4.1.2. 4.22<br />
Optimal matrix W ⋆ is defined as any direction matrix yielding optimal<br />
solution G ⋆ of rank n or less to then convex equivalent (670) of feasibility<br />
problem (669); id est, any direction matrix for which the last N − n<br />
nonincreasingly ordered eigenvalues λ of G ⋆ are zero: (p.599)<br />
N∑<br />
λ(G ⋆ ) i = 〈G ⋆ , W ⋆ 〉 = ∆ 0 (671)<br />
i=n+1<br />
We emphasize that convex problem (670) is not a relaxation of the<br />
rank-constrained feasibility problem (669); at convergence, convex iteration<br />
(670) (1581a) makes it instead an equivalent problem. 4.23<br />
4.4.1.1 direction interpretation<br />
We make no assumption regarding uniqueness of direction matrix W . The<br />
feasible set of direction matrices in (1581a) is the convex hull of outer product<br />
of all rank-(N − n) orthonormal matrices; videlicet,<br />
conv { UU T | U ∈ R N×N−n , U T U = I } = { A∈ S N | I ≽ A ≽ 0, 〈I , A 〉= N − n } (82)<br />
Set {UU T | U ∈ R N×N−n , U T U = I} comprises the extreme points of this<br />
Fantope (82). An optimal solution W to (1581a), that is an extreme point,<br />
is known in closed form (p.599). By (205), that particular direction (−W)<br />
can be regarded as pointing toward positive semidefinite rank-n matrices that<br />
4.22 The proposed iteration is not an alternating projection. (confer Figure 140)<br />
4.23 Terminology equivalent problem meaning, optimal solution to one problem can be<br />
derived from optimal solution to another. Terminology same problem means: optimal<br />
solution set for one problem is identical to the optimal solution set of another (without<br />
transformation).