v2007.09.13 - Convex Optimization

2.2. VECTORIZED-MATRIX INNER PRODUCT 452.2 Vectorized-matrix inner productEuclidean space R n comes equipped with a linear vector inner-product〈y,z〉 ∆ = y T z (26)We prefer those angle brackets to connote a geometric rather than algebraicperspective. Two vectors are orthogonal (perpendicular) to one another ifand only if their inner product vanishes;A vector inner-product defines a normy ⊥ z ⇔ 〈y,z〉 = 0 (27)‖y‖ 2 ∆ = √ y T y , ‖y‖ 2 = 0 ⇔ y = 0 (28)When orthogonal vectors each have unit norm, then they are orthonormal.For linear operation A on a vector, represented by a real matrix, the adjointoperation A T is transposition and defined for matrix A by [165,3.10]〈y,A T z〉 ∆ = 〈Ay,z〉 (29)The vector inner-product for matrices is calculated just as it is for vectors;by first transforming a matrix in R p×k to a vector in R pk by concatenatingits columns in the natural order. For lack of a better term, we shall callthat linear bijective (one-to-one and onto [165, App.A1.2]) transformationvectorization. For example, the vectorization of Y = [y 1 y 2 · · · y k ] ∈ R p×k[115] [243] is⎡ ⎤y 1vec Y =∆ y⎢ 2⎥⎣ . ⎦ ∈ Rpk (30)y kThen the vectorized-matrix inner-product is trace of matrix inner-product;for Z ∈ R p×k , [46,2.6.1] [147,0.3.1] [287,8] [273,2.2]where (A.1.1)〈Y , Z〉 ∆ = tr(Y T Z) = vec(Y ) T vec Z (31)tr(Y T Z) = tr(ZY T ) = tr(YZ T ) = tr(Z T Y ) = 1 T (Y ◦ Z)1 (32)