v2007.09.13 - Convex Optimization

A.6. SINGULAR VALUE DECOMPOSITION, SVD 505A.6 Singular value decomposition, SVDA.6.1Compact SVD[109,2.5.4] For any A∈ R m×nq T1A = UΣQ T = [ u 1 · · · u η ] Σ⎣.⎡⎤∑⎦ = η σ i u i qiTqηT i=1(1345)U ∈ R m×η , Σ ∈ R η×η , Q ∈ R n×ηwhere U and Q are always skinny-or-square each having orthonormalcolumns, and whereη ∆ = min{m , n} (1346)Square matrix Σ is diagonal (A.1.1)δ 2 (Σ) = Σ ∈ R η×η (1347)holding the singular values σ i of A which are always arranged innonincreasing order by convention and are related to eigenvalues by A.14⎧√ ⎨ λ(AT A) i = √ (√ )λ(AA T ) i = λ AT A)= λ(√AAT> 0, i = 1... ρσ(A) i =ii⎩0, i = ρ+1... η(1348)of which the last η −ρ are 0 , A.15 whereρ ∆ = rankA = rank Σ (1349)A point sometimes lost: Any real matrix may be decomposed in terms ofits real singular values σ(A) ∈ R η and real matrices U and Q as in (1345),where [109,2.5.3]R{u i |σ i ≠0} = R(A)R{u i |σ i =0} ⊆ N(A T )R{q i |σ i ≠0} = R(A T (1350))R{q i |σ i =0} ⊆ N(A)A.14 When A is normal,√σ(A) = |λ(A)|. [298,8.1])A.15 For η = n , σ(A) = λ(A T A) = λ(√AT A where λ denotes eigenvalues.√)For η = m , σ(A) = λ(AA T ) = λ(√AAT.