v2009.01.01 - Convex Optimization

564 APPENDIX A. LINEAR ALGEBRA
eigenvalues must be real and whose eigenvectors can be chosen to make a
real orthonormal set; [289,6.4] [287, p.315] id est, for X ∈ S m
s T1
X = SΛS T = [ s 1 · · · s m ] Λ⎣
.
⎡
s T m
⎤
⎦ =
m∑
λ i s i s T i (1448)
where δ 2 (Λ) = Λ∈ S m (A.1) and S −1 = S T ∈ R m×m (orthogonal matrix,
B.5) because of symmetry: SΛS −1 = S −T ΛS T .
Because the arrangement of eigenvectors and their corresponding
eigenvalues is arbitrary, we almost always arrange eigenvalues in
nonincreasing order as is the convention for singular value decomposition.
Then to diagonalize a symmetric matrix that is already a diagonal matrix,
orthogonal matrix S becomes a permutation matrix.
A.5.2.1
i=1
Positive semidefinite matrix square root
When X ∈ S m + , its unique positive semidefinite matrix square root is defined
√
X ∆ = S √ ΛS T ∈ S m + (1449)
where the square root of nonnegative diagonal matrix √ Λ is taken entrywise
and positive. Then X = √ X √ X .
A.6 Singular value decomposition, SVD
A.6.1
Compact SVD
[134,2.5.4] For any A∈ R m×n
q T1
A = UΣQ T = [u 1 · · · u η ] Σ⎣
.
⎡
⎤
∑
⎦ = η σ i u i qi
T
qη
T i=1
(1450)
U ∈ R m×η , Σ ∈ R η×η , Q ∈ R n×η
where U and Q are always skinny-or-square each having orthonormal
columns, and where
η ∆ = min{m , n} (1451)