v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
minimizeR662 APPENDIX C. SOME ANALYTICAL OPTIMAL RESULTSOptimal solution R ⋆ can reveal rotation/reflection (5.5.2,B.5) of onelist in the columns of matrix A with respect to another list in B . Solution isunique if rankBV N = n . [113,2.4.1] In the case that A is a vector andpermutation of B , solution R ⋆ is not necessarily a permutation matrix(4.6.0.0.3) although the optimal objective will be zero. More generally, theoptimal value for objective of minimization istr ( A T A + B T B − 2AB T R ⋆) = tr(A T A) + tr(B T B) − 2 tr(UΣU T )= ‖A‖ 2 F + ‖B‖2 F − 2δ(Σ)T 1while the optimal value for corresponding trace maximization is(1715)sup tr(A T R T B) = tr(A T R ⋆T B) = δ(Σ) T 1 ≥ tr(A T B) (1716)R T =R −1The same optimal solution R ⋆ solvesC.3.0.1Procrustes relaxationmaximize ‖A + R T B‖ FR(1717)subject to R T = R −1By replacing its feasible set with the convex hull of orthogonal matrices(Example 2.3.2.0.5), we relax Procrustes problem (1712) to a convex problem‖A − R T B‖ 2 Fsubject to R T = R −1= tr(AT A + B T B) − 2 maximizeRsubject towhose adjusted objective must always equal Procrustes’. C.4C.3.1Effect of translationtr(A T R T B)[ ] I RR T ≽ 0 (1718)IConsider the impact on problem (1712) of dc offset in known listsA,B∈ R n×N . Rotation of B there is with respect to the origin, so betterresults may be obtained if offset is first accounted. Because the geometriccenters of the lists AV and BV are the origin, instead we solveminimize ‖AV − R T BV ‖ FR(1719)subject to R T = R −1C.4 (because orthogonal matrices are the extreme points of this hull) and whose optimalnumerical solution (SDPT3 [348]) [167] is reliably observed to be orthogonal for n≤N .
C.4. TWO-SIDED ORTHOGONAL PROCRUSTES 663where V ∈ S N is the geometric centering matrix (B.4.1). Now we define thefull singular value decompositionand an optimal rotation matrixAV B T UΣQ T ∈ R n×n (1720)R ⋆ = QU T ∈ R n×n (1713)The desired result is an optimally rotated offset listR ⋆T BV + A(I − V ) ≈ A (1721)which most closely matches the list in A . Equality is attained when the listsare precisely related by a rotation/reflection and an offset. When R ⋆T B=Aor B1=A1=0, this result (1721) reduces to R ⋆T B ≈ A .C.3.1.1Translation of extended listSuppose an optimal rotation matrix R ⋆ ∈ R n×n were derived as beforefrom matrix B ∈ R n×N , but B is part of a larger list in the columns of[C B ]∈ R n×M+N where C ∈ R n×M . In that event, we wish to applythe rotation/reflection and translation to the larger list. The expressionsupplanting the approximation in (1721) makes 1 T of compatible dimension;R ⋆T [C −B11 T 1 NBV ] + A11 T 1 N(1722)id est, C −B11 T 1 N ∈ Rn×M and A11 T 1 N ∈ Rn×M+N .C.4 Two-sided orthogonal ProcrustesC.4.0.1MinimizationGiven symmetric A,B∈ S N , each having diagonalization (A.5.1)A Q A Λ A Q T A , B Q B Λ B Q T B (1723)where eigenvalues are arranged in their respective diagonal matrix Λ innonincreasing order, then an optimal solution [130]R ⋆ = Q B Q T A ∈ R N×N (1724)
- Page 611 and 612: A.4. SCHUR COMPLEMENT 611A.4 Schur
- Page 613 and 614: A.4. SCHUR COMPLEMENT 613A.4.0.0.3
- Page 615 and 616: A.4. SCHUR COMPLEMENT 615From Corol
- Page 617 and 618: A.5. EIGENVALUE DECOMPOSITION 617wh
- Page 619 and 620: A.5. EIGENVALUE DECOMPOSITION 619A.
- Page 621 and 622: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 623 and 624: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 625 and 626: A.7. ZEROS 625A.6.5SVD of symmetric
- Page 627 and 628: A.7. ZEROS 627(Transpose.)Likewise,
- Page 629 and 630: A.7. ZEROS 629For X,A∈ S M +[34,2
- Page 631 and 632: A.7. ZEROS 631A.7.5.0.1 Proposition
- Page 633 and 634: Appendix BSimple matricesMathematic
- Page 635 and 636: B.1. RANK-ONE MATRIX (DYAD) 635R(v)
- Page 637 and 638: B.1. RANK-ONE MATRIX (DYAD) 637B.1.
- Page 639 and 640: B.2. DOUBLET 639R([u v ])R(Π)= R([
- Page 641 and 642: B.3. ELEMENTARY MATRIX 641has N −
- Page 643 and 644: B.4. AUXILIARY V -MATRICES 643is an
- Page 645 and 646: B.4. AUXILIARY V -MATRICES 64514. [
- Page 647 and 648: B.5. ORTHOGONAL MATRIX 647Given X
- Page 649 and 650: B.5. ORTHOGONAL MATRIX 649Figure 15
- Page 651 and 652: B.5. ORTHOGONAL MATRIX 651which is
- Page 653 and 654: Appendix CSome analytical optimal r
- Page 655 and 656: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 657 and 658: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 659 and 660: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 661: C.3. ORTHOGONAL PROCRUSTES PROBLEM
- Page 665 and 666: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 667 and 668: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 669 and 670: Appendix DMatrix calculusFrom too m
- Page 671 and 672: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 673 and 674: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 675 and 676: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 677 and 678: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 679 and 680: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 687 and 688: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 689 and 690: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
C.4. TWO-SIDED ORTHOGONAL PROCRUSTES 663where V ∈ S N is the geometric centering matrix (B.4.1). Now we define thefull singular value decompositionand an optimal rotation matrixAV B T UΣQ T ∈ R n×n (1720)R ⋆ = QU T ∈ R n×n (1713)The desired result is an optimally rotated offset listR ⋆T BV + A(I − V ) ≈ A (1721)which most closely matches the list in A . Equality is attained when the listsare precisely related by a rotation/reflection and an offset. When R ⋆T B=Aor B1=A1=0, this result (1721) reduces to R ⋆T B ≈ A .C.3.1.1Translation of extended listSuppose an optimal rotation matrix R ⋆ ∈ R n×n were derived as beforefrom matrix B ∈ R n×N , but B is part of a larger list in the columns of[C B ]∈ R n×M+N where C ∈ R n×M . In that event, we wish to applythe rotation/reflection and translation to the larger list. The expressionsupplanting the approximation in (1721) makes 1 T of compatible dimension;R ⋆T [C −B11 T 1 NBV ] + A11 T 1 N(1722)id est, C −B11 T 1 N ∈ Rn×M and A11 T 1 N ∈ Rn×M+N .C.4 Two-sided orthogonal ProcrustesC.4.0.1MinimizationGiven symmetric A,B∈ S N , each having diagonalization (A.5.1)A Q A Λ A Q T A , B Q B Λ B Q T B (1723)where eigenvalues are arranged in their respective diagonal matrix Λ innonincreasing order, then an optimal solution [130]R ⋆ = Q B Q T A ∈ R N×N (1724)