v2007.09.13 - Convex Optimization
v2007.09.13 - Convex Optimization v2007.09.13 - Convex Optimization
46 CHAPTER 2. CONVEX GEOMETRYand where ◦ denotes the Hadamard product 2.10 of matrices [149] [109,1.1.4].The adjoint operation A T on a matrix can therefore be defined in like manner:〈Y , A T Z〉 ∆ = 〈AY , Z〉 (33)For example, take any element C 1 from a matrix-valued set in R p×k ,and consider any particular dimensionally compatible real vectors v andw . Then vector inner-product of C 1 with vw T is〈vw T , C 1 〉 = v T C 1 w = tr(wv T C 1 ) = 1 T( (vw T )◦ C 1)1 (34)2.2.0.0.1 Example. Application of the image theorem.Suppose the set C ⊆ R p×k is convex. Then for any particular vectors v ∈R pand w ∈R k , the set of vector inner-productsY ∆ = v T Cw = 〈vw T , C〉 ⊆ R (35)is convex. This result is a consequence of the image theorem. Yet it is easyto show directly that convex combination of elements from Y remains anelement of Y . 2.11More generally, vw T in (35) may be replaced with any particular matrixZ ∈ R p×k while convexity of the set 〈Z , C〉⊆ R persists. Further, byreplacing v and w with any particular respective matrices U and W ofdimension compatible with all elements of convex set C , then set U T CWis convex by the image theorem because it is a linear mapping of C .2.10 The Hadamard product is a simple entrywise product of corresponding entries fromtwo matrices of like size; id est, not necessarily square.2.11 To verify that, take any two elements C 1 and C 2 from the convex matrix-valued setC , and then form the vector inner-products (35) that are two elements of Y by definition.Now make a convex combination of those inner products; videlicet, for 0≤µ≤1µ 〈vw T , C 1 〉 + (1 − µ) 〈vw T , C 2 〉 = 〈vw T , µ C 1 + (1 − µ)C 2 〉The two sides are equivalent by linearity of inner product. The right-hand side remainsa vector inner-product of vw T with an element µ C 1 + (1 − µ)C 2 from the convex set C ;hence it belongs to Y . Since that holds true for any two elements from Y , then it mustbe a convex set.
2.2. VECTORIZED-MATRIX INNER PRODUCT 472.2.1 Frobenius’2.2.1.0.1 Definition. Isomorphic.An isomorphism of a vector space is a transformation equivalent to a linearbijective mapping. The image and inverse image under the transformationoperator are then called isomorphic vector spaces.△Isomorphic vector spaces are characterized by preservation of adjacency;id est, if v and w are points connected by a line segment in one vector space,then their images will also be connected by a line segment. Two Euclideanbodies may be considered isomorphic of there exists an isomorphism of theircorresponding ambient spaces. [274,I.1]When Z =Y ∈ R p×k in (31), Frobenius’ norm is resultant from vectorinner-product; (confer (1462))‖Y ‖ 2 F = ‖ vec Y ‖2 2 = 〈Y , Y 〉 = tr(Y T Y )= ∑ i,jY 2ij = ∑ iλ(Y T Y ) i = ∑ iσ(Y ) 2 i(36)where λ(Y T Y ) i is the i th eigenvalue of Y T Y , and σ(Y ) i the i th singularvalue of Y . Were Y a normal matrix (A.5.2), then σ(Y )= |λ(Y )|[298,8.1] thus‖Y ‖ 2 F = ∑ iλ(Y ) 2 i = ‖λ(Y )‖ 2 2 (37)The converse (37) ⇒ normal matrix Y also holds. [149,2.5.4]Because the metrics are equivalent‖ vec X −vec Y ‖ 2 = ‖X −Y ‖ F (38)and because vectorization (30) is a linear bijective map, then vector spaceR p×k is isometrically isomorphic with vector space R pk in the Euclidean senseand vec is an isometric isomorphism on R p×k . 2.12 Because of this Euclideanstructure, all the known results from convex analysis in Euclidean space R ncarry over directly to the space of real matrices R p×k .2.12 Given matrix A, its range R(A) (2.5) is isometrically isomorphic with its vectorizedrange vec R(A) but not with R(vec A).
- Page 1 and 2: DATTORROCONVEXOPTIMIZATION&EUCLIDEA
- Page 3 and 4: Convex Optimization&Euclidean Dista
- Page 5 and 6: for Jennie Columba♦Antonio♦♦&
- Page 7 and 8: PreludeThe constant demands of my d
- Page 9 and 10: Convex Optimization&Euclidean Dista
- Page 11 and 12: CONVEX OPTIMIZATION & EUCLIDEAN DIS
- Page 13 and 14: List of Figures1 Overview 191 Orion
- Page 15 and 16: LIST OF FIGURES 1559 Quadratic func
- Page 17 and 18: LIST OF FIGURES 17E Projection 5791
- Page 19 and 20: Chapter 1OverviewConvex Optimizatio
- Page 21 and 22: ˇx 4ˇx 3ˇx 2Figure 2: Applicatio
- Page 23 and 24: 23Figure 4: This coarsely discretiz
- Page 25 and 26: ases (biorthogonal expansion). We e
- Page 27 and 28: 27Figure 7: These bees construct a
- Page 29 and 30: its membership to the EDM cone. The
- Page 31 and 32: 31appendicesProvided so as to be mo
- Page 33 and 34: Chapter 2Convex geometryConvexity h
- Page 35 and 36: 2.1. CONVEX SET 35Figure 9: A slab
- Page 37 and 38: 2.1. CONVEX SET 372.1.6 empty set v
- Page 39 and 40: 2.1. CONVEX SET 392.1.7.1 Line inte
- Page 41 and 42: 2.1. CONVEX SET 41(a)R 2(b)R 3(c)(d
- Page 43 and 44: 2.1. CONVEX SET 43This theorem in c
- Page 45: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 49 and 50: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 51 and 52: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 53 and 54: 2.3. HULLS 53Figure 12: Convex hull
- Page 55 and 56: 2.3. HULLS 55Aaffine hull (drawn tr
- Page 57 and 58: 2.3. HULLS 57The union of relative
- Page 59 and 60: 2.4. HALFSPACE, HYPERPLANE 59of dim
- Page 61 and 62: 2.4. HALFSPACE, HYPERPLANE 61H +ay
- Page 63 and 64: 2.4. HALFSPACE, HYPERPLANE 63Inters
- Page 65 and 66: 2.4. HALFSPACE, HYPERPLANE 65Conver
- Page 67 and 68: 2.4. HALFSPACE, HYPERPLANE 67A 1A 2
- Page 69 and 70: 2.4. HALFSPACE, HYPERPLANE 69tradit
- Page 71 and 72: 2.4. HALFSPACE, HYPERPLANE 71There
- Page 73 and 74: 2.5. SUBSPACE REPRESENTATIONS 732.5
- Page 75 and 76: 2.5. SUBSPACE REPRESENTATIONS 752.5
- Page 77 and 78: 2.6. EXTREME, EXPOSED 77In other wo
- Page 79 and 80: 2.6. EXTREME, EXPOSED 792.6.1 Expos
- Page 81 and 82: 2.7. CONES 812.6.1.3.1 Definition.
- Page 83 and 84: 2.7. CONES 830Figure 24: Boundary o
- Page 85 and 86: 2.7. CONES 852.7.2 Convex coneWe ca
- Page 87 and 88: 2.7. CONES 87Thus the simplest and
- Page 89 and 90: 2.7. CONES 89nomenclature generaliz
- Page 91 and 92: 2.8. CONE BOUNDARY 91Proper cone {0
- Page 93 and 94: 2.8. CONE BOUNDARY 93the same extre
2.2. VECTORIZED-MATRIX INNER PRODUCT 472.2.1 Frobenius’2.2.1.0.1 Definition. Isomorphic.An isomorphism of a vector space is a transformation equivalent to a linearbijective mapping. The image and inverse image under the transformationoperator are then called isomorphic vector spaces.△Isomorphic vector spaces are characterized by preservation of adjacency;id est, if v and w are points connected by a line segment in one vector space,then their images will also be connected by a line segment. Two Euclideanbodies may be considered isomorphic of there exists an isomorphism of theircorresponding ambient spaces. [274,I.1]When Z =Y ∈ R p×k in (31), Frobenius’ norm is resultant from vectorinner-product; (confer (1462))‖Y ‖ 2 F = ‖ vec Y ‖2 2 = 〈Y , Y 〉 = tr(Y T Y )= ∑ i,jY 2ij = ∑ iλ(Y T Y ) i = ∑ iσ(Y ) 2 i(36)where λ(Y T Y ) i is the i th eigenvalue of Y T Y , and σ(Y ) i the i th singularvalue of Y . Were Y a normal matrix (A.5.2), then σ(Y )= |λ(Y )|[298,8.1] thus‖Y ‖ 2 F = ∑ iλ(Y ) 2 i = ‖λ(Y )‖ 2 2 (37)The converse (37) ⇒ normal matrix Y also holds. [149,2.5.4]Because the metrics are equivalent‖ vec X −vec Y ‖ 2 = ‖X −Y ‖ F (38)and because vectorization (30) is a linear bijective map, then vector spaceR p×k is isometrically isomorphic with vector space R pk in the Euclidean senseand vec is an isometric isomorphism on R p×k . 2.12 Because of this Euclideanstructure, all the known results from convex analysis in Euclidean space R ncarry over directly to the space of real matrices R p×k .2.12 Given matrix A, its range R(A) (2.5) is isometrically isomorphic with its vectorizedrange vec R(A) but not with R(vec A).
Change language