v2009.01.01 - Convex Optimization

666 APPENDIX E. PROJECTION
E.6.4.1.1 Example. Eigenvalues λ as coefficients of orthogonal projection.
Let C represent any convex subset of subspace S M , and let C 1 be any element
of C . Then C 1 can be expressed as the orthogonal expansion
C 1 =
M∑
i=1
M∑
〈E ij , C 1 〉 E ij ∈ C ⊂ S M (1856)
j=1
j ≥ i
where E ij ∈ S M is a member of the standard orthonormal basis for S M
(52). This expansion is a sum of one-dimensional orthogonal projections
of C 1 ; each projection on the range of a vectorized standard basis matrix.
Vector inner-product 〈E ij , C 1 〉 is the coefficient of projection of svec C 1 on
R(svec E ij ).
When C 1 is any member of a convex set C whose dimension is L ,
Carathéodory’s theorem [96] [266] [173] [36] [37] guarantees that no more
than L +1 affinely independent members from C are required to faithfully
represent C 1 by their linear combination. E.11
Dimension of S M is L=M(M+1)/2 in isometrically isomorphic
R M(M+1)/2 . Yet because any symmetric matrix can be diagonalized, (A.5.2)
C 1 ∈ S M is a linear combination of its M eigenmatrices q i q T i (A.5.1) weighted
by its eigenvalues λ i ;
C 1 = QΛQ T =
M∑
λ i q i qi T (1857)
i=1
where Λ ∈ S M is a diagonal matrix having δ(Λ) i =λ i , and Q=[q 1 · · · q M ]
is an orthogonal matrix in R M×M containing corresponding eigenvectors.
To derive eigen decomposition (1857) from expansion (1856), M standard
basis matrices E ij are rotated (B.5) into alignment with the M eigenmatrices
q i qi
T of C 1 by applying a similarity transformation; [287,5.6]
{QE ij Q T } =
{
qi q T i , i = j = 1... M
}
( )
√1
qi 2
qj T + q j qi
T , 1 ≤ i < j ≤ M
(1858)
E.11 Carathéodory’s theorem guarantees existence of a biorthogonal expansion for any
element in aff C when C is any pointed closed convex cone.