v2010.10.26 - Convex Optimization

A.6. SINGULAR VALUE DECOMPOSITION, SVD 621where U and Q are always skinny-or-square, each having orthonormalcolumns, and whereη min{m , n} (1564)Square matrix Σ is diagonal (A.1.1)δ 2 (Σ) = Σ ∈ R η×η (1565)holding the singular values {σ i ∈ R} of A which are always arranged innonincreasing order by convention and are related to eigenvalues λ by A.14⎧√ ⎨ λ(AT A)σ(A) i = σ(A T i = √ (√ )λ(AA T ) i = λ AT A)= λ(√AAT> 0, 1 ≤ i ≤ ρ) i =ii⎩0, ρ < i ≤ η(1566)of which the last η −ρ are 0 , A.15 whereρ rankA = rank Σ (1567)A point sometimes lost: Any real matrix may be decomposed in terms ofits real singular values σ(A)∈ R η and real matrices U and Q as in (1563),where [159,2.5.3]R{u i |σ i ≠0} = R(A)R{u i |σ i =0} ⊆ N(A T )R{q i |σ i ≠0} = R(A T (1568))R{q i |σ i =0} ⊆ N(A)A.6.2Subcompact SVDSome authors allow only nonzero singular values. In that case the compactdecomposition can be made smaller; it can be redimensioned in terms ofrank ρ because, for any A∈ R m×nρ = rankA = rank Σ = max {i∈{1... η} | σ i ≠ 0} ≤ η (1569)There are η singular values. For any flavor SVD, rank is equivalent tothe number of nonzero singular values on the main diagonal of Σ.A.14 When matrix A is normal,√σ(A) = |λ(A)|.)[393,8.1]A.15 For η = n , σ(A) = λ(A T A) = λ(√AT A .√)For η = m , σ(A) = λ(AA T ) = λ(√AAT.