v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
664 APPENDIX C. SOME ANALYTICAL OPTIMAL RESULTSto the two-sided orthogonal Procrustes problemminimize ‖A − R T BR‖ FR= minimize tr ( A T A − 2A T R T BR + B T B )subject to R T = R −1 Rsubject to R T = R −1 (1725)maximizes tr(A T R T BR) over the nonconvex manifold of orthogonal matrices.Optimal product R ⋆T BR ⋆ has the eigenvectors of A but the eigenvalues of B .[164,7.5.1] The optimal value for the objective of minimization is, by (48)‖Q A Λ A Q T A −R ⋆T Q B Λ B Q T BR ⋆ ‖ F = ‖Q A (Λ A −Λ B )Q T A ‖ F = ‖Λ A −Λ B ‖ F (1726)while the corresponding trace maximization has optimal valuesup tr(A T R T BR) = tr(A T R ⋆T BR ⋆ ) = tr(Λ A Λ B ) ≥ tr(A T B) (1727)R T =R −1The lower bound on inner product of eigenvalues is due to Fan (p.603).C.4.0.2MaximizationAny permutation matrix is an orthogonal matrix. Defining a row- andcolumn-swapping permutation matrix (a reflection matrix,B.5.2)⎡ ⎤0 1·Ξ = Ξ T =⎢ ·⎥(1728)⎣ 1 ⎦1 0then an optimal solution R ⋆ to the maximization problem [sic]minimizes tr(A T R T BR) : [200] [239,2.1] [240]maximize ‖A − R T BR‖ FR(1729)subject to R T = R −1R ⋆ = Q B ΞQ T A ∈ R N×N (1730)The optimal value for the objective of maximization is‖Q A Λ A Q T A − R⋆T Q B Λ B Q T B R⋆ ‖ F = ‖Q A Λ A Q T A − Q A ΞT Λ B ΞQ T A ‖ F= ‖Λ A − ΞΛ B Ξ‖ F(1731)while the corresponding trace minimization has optimal valueinf tr(A T R T BR) = tr(A T R ⋆T BR ⋆ ) = tr(Λ A ΞΛ B Ξ) (1732)R T =R −1
C.4. TWO-SIDED ORTHOGONAL PROCRUSTES 665C.4.1Procrustes’ relation to linear programmingAlthough these two-sided Procrustes problems are nonconvex, a connectionwith linear programming [94] was discovered by Anstreicher & Wolkowicz[12,3] [239,2.1] [240]: Given A,B∈ S N , this semidefinite program in Sand Tminimize tr(A T R T BR) = maximize tr(S + T ) (1733)RS , T ∈S Nsubject to R T = R −1 subject to A T ⊗ B − I ⊗ S − T ⊗ I ≽ 0(where ⊗ signifies Kronecker product (D.1.2.1)) has optimal objectivevalue (1732). These two problems in (1733) are strong duals (2.13.1.0.3).Given ordered diagonalizations (1723), make the observation:infR tr(AT R T BR) = inf tr(Λ ˆRT A Λ B ˆR) (1734)ˆRbecause ˆRQ B TRQ A on the set of orthogonal matrices (which includes thepermutation matrices) is a bijection. This means, basically, diagonal matricesof eigenvalues Λ A and Λ B may be substituted for A and B , so only the maindiagonals of S and T come into play;maximizeS,T ∈S N 1 T δ(S + T )subject to δ(Λ A ⊗ (ΞΛ B Ξ) − I ⊗ S − T ⊗ I) ≽ 0(1735)a linear program in δ(S) and δ(T) having the same optimal objective valueas the semidefinite program (1733).We relate their results to Procrustes problem (1725) by manipulatingsigns (1678) and permuting eigenvalues:maximize tr(A T R T BR) = minimize 1 T δ(S + T )RS , T ∈S Nsubject to R T = R −1 subject to δ(I ⊗ S + T ⊗ I − Λ A ⊗ Λ B ) ≽ 0= minimize tr(S + T )(1736)S , T ∈S Nsubject to I ⊗ S + T ⊗ I − A T ⊗ B ≽ 0This formulation has optimal objective value identical to that in (1727).
- Page 613 and 614: A.4. SCHUR COMPLEMENT 613A.4.0.0.3
- Page 615 and 616: A.4. SCHUR COMPLEMENT 615From Corol
- Page 617 and 618: A.5. EIGENVALUE DECOMPOSITION 617wh
- Page 619 and 620: A.5. EIGENVALUE DECOMPOSITION 619A.
- Page 621 and 622: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 623 and 624: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 625 and 626: A.7. ZEROS 625A.6.5SVD of symmetric
- Page 627 and 628: A.7. ZEROS 627(Transpose.)Likewise,
- Page 629 and 630: A.7. ZEROS 629For X,A∈ S M +[34,2
- Page 631 and 632: A.7. ZEROS 631A.7.5.0.1 Proposition
- Page 633 and 634: Appendix BSimple matricesMathematic
- Page 635 and 636: B.1. RANK-ONE MATRIX (DYAD) 635R(v)
- Page 637 and 638: B.1. RANK-ONE MATRIX (DYAD) 637B.1.
- Page 639 and 640: B.2. DOUBLET 639R([u v ])R(Π)= R([
- Page 641 and 642: B.3. ELEMENTARY MATRIX 641has N −
- Page 643 and 644: B.4. AUXILIARY V -MATRICES 643is an
- Page 645 and 646: B.4. AUXILIARY V -MATRICES 64514. [
- Page 647 and 648: B.5. ORTHOGONAL MATRIX 647Given X
- Page 649 and 650: B.5. ORTHOGONAL MATRIX 649Figure 15
- Page 651 and 652: B.5. ORTHOGONAL MATRIX 651which is
- Page 653 and 654: Appendix CSome analytical optimal r
- Page 655 and 656: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 657 and 658: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 659 and 660: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 661 and 662: C.3. ORTHOGONAL PROCRUSTES PROBLEM
- Page 663: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 667 and 668: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 669 and 670: Appendix DMatrix calculusFrom too m
- Page 671 and 672: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 673 and 674: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 675 and 676: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 677 and 678: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 679 and 680: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 687 and 688: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 689 and 690: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 713 and 714: E.3. SYMMETRIC IDEMPOTENT MATRICES
C.4. TWO-SIDED ORTHOGONAL PROCRUSTES 665C.4.1Procrustes’ relation to linear programmingAlthough these two-sided Procrustes problems are nonconvex, a connectionwith linear programming [94] was discovered by Anstreicher & Wolkowicz[12,3] [239,2.1] [240]: Given A,B∈ S N , this semidefinite program in Sand Tminimize tr(A T R T BR) = maximize tr(S + T ) (1733)RS , T ∈S Nsubject to R T = R −1 subject to A T ⊗ B − I ⊗ S − T ⊗ I ≽ 0(where ⊗ signifies Kronecker product (D.1.2.1)) has optimal objectivevalue (1732). These two problems in (1733) are strong duals (2.13.1.0.3).Given ordered diagonalizations (1723), make the observation:infR tr(AT R T BR) = inf tr(Λ ˆRT A Λ B ˆR) (1734)ˆRbecause ˆRQ B TRQ A on the set of orthogonal matrices (which includes thepermutation matrices) is a bijection. This means, basically, diagonal matricesof eigenvalues Λ A and Λ B may be substituted for A and B , so only the maindiagonals of S and T come into play;maximizeS,T ∈S N 1 T δ(S + T )subject to δ(Λ A ⊗ (ΞΛ B Ξ) − I ⊗ S − T ⊗ I) ≽ 0(1735)a linear program in δ(S) and δ(T) having the same optimal objective valueas the semidefinite program (1733).We relate their results to Procrustes problem (1725) by manipulatingsigns (1678) and permuting eigenvalues:maximize tr(A T R T BR) = minimize 1 T δ(S + T )RS , T ∈S Nsubject to R T = R −1 subject to δ(I ⊗ S + T ⊗ I − Λ A ⊗ Λ B ) ≽ 0= minimize tr(S + T )(1736)S , T ∈S Nsubject to I ⊗ S + T ⊗ I − A T ⊗ B ≽ 0This formulation has optimal objective value identical to that in (1727).