v2009.01.01 - Convex Optimization

46 CHAPTER 2. CONVEX GEOMETRY
and where ◦ denotes the Hadamard product 2.10 of matrices [134,1.1.4]. The
adjoint operation A T on a matrix can therefore be defined in like manner:
〈Y , A T Z〉 ∆ = 〈AY , Z〉 (33)
Take any element C 1 from a matrix-valued set in R p×k , for example, and
consider any particular dimensionally compatible real vectors v and w .
Then vector inner-product of C 1 with vw T is
〈vw T , C 1 〉 = 〈v , C 1 w〉 = v T C 1 w = tr(wv T C 1 ) = 1 T( (vw T )◦ C 1
)
1 (34)
Further, linear bijective vectorization is distributive with respect to
Hadamard product; id est,
vec(Y ◦ Z) = vec(Y ) ◦ vec(Z) (35)
2.2.0.0.1 Example. Application of the image theorem.
Suppose the set C ⊆ R p×k is convex. Then for any particular vectors v ∈R p
and w ∈R k , the set of vector inner-products
Y ∆ = v T Cw = 〈vw T , C〉 ⊆ R (36)
is convex. This result is a consequence of the image theorem. Yet it is easy
to show directly that convex combination of elements from Y remains an
element of Y . 2.11
More generally, vw T in (36) may be replaced with any particular matrix
Z ∈ R p×k while convexity of the set 〈Z , C〉⊆ R persists. Further, by
replacing v and w with any particular respective matrices U and W of
dimension compatible with all elements of convex set C , then set U T CW
is convex by the image theorem because it is a linear mapping of C .
2.10 The Hadamard product is a simple entrywise product of corresponding entries from
two matrices of like size; id est, not necessarily square. A commutative operation, the
Hadamard product can be extracted from within a Kronecker product. [176, p.475]
2.11 To verify that, take any two elements C 1 and C 2 from the convex matrix-valued set C ,
and then form the vector inner-products (36) that are two elements of Y by definition.
Now make a convex combination of those inner products; videlicet, for 0≤µ≤1
µ 〈vw T , C 1 〉 + (1 − µ) 〈vw T , C 2 〉 = 〈vw T , µ C 1 + (1 − µ)C 2 〉
The two sides are equivalent by linearity of inner product. The right-hand side remains
a vector inner-product of vw T with an element µ C 1 + (1 − µ)C 2 from the convex set C ;
hence, it belongs to Y . Since that holds true for any two elements from Y , then it must
be a convex set.