Chapter 3 Geometry of convex functions - Meboo Publishing ...
Chapter 3 Geometry of convex functions - Meboo Publishing ... Chapter 3 Geometry of convex functions - Meboo Publishing ...
254 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS 3.8.0.0.1 Definition. Convex matrix-valued function: 1) Matrix-definition. A function g(X) : R p×k →S M is convex in X iff domg is a convex set and, for each and every Y,Z ∈domg and all 0≤µ≤1 [210,2.3.7] g(µY + (1 − µ)Z) ≼ µg(Y ) + (1 − µ)g(Z) (597) S M + Reversing sense of the inequality flips this definition to concavity. Strict convexity is defined less a stroke of the pen in (597) similarly to (478). 2) Scalar-definition. It follows that g(X) : R p×k →S M is convex in X iff w T g(X)w : R p×k →R is convex in X for each and every ‖w‖= 1; shown by substituting the defining inequality (597). By dual generalized inequalities we have the equivalent but more broad criterion, (2.13.5) g convex w.r.t S M + ⇔ 〈W , g〉 convex for each and every W ≽ S M∗ + 0 (598) Strict convexity on both sides requires caveat W ≠ 0. Because the set of all extreme directions for the selfdual positive semidefinite cone (2.9.2.4) comprises a minimal set of generators for that cone, discretization (2.13.4.2.1) allows replacement of matrix W with symmetric dyad ww T as proposed. △ 3.8.0.0.2 Example. Taxicab distance matrix. Consider an n-dimensional vector space R n with metric induced by the 1-norm. Then distance between points x 1 and x 2 is the norm of their difference: ‖x 1 −x 2 ‖ 1 . Given a list of points arranged columnar in a matrix X = [x 1 · · · x N ] ∈ R n×N (76) then we could define a taxicab distance matrix D 1 (X) (I ⊗ 1 T n) | vec(X)1 T − 1 ⊗X | ∈ S N h ∩ R N×N + ⎡ ⎤ 0 ‖x 1 − x 2 ‖ 1 ‖x 1 − x 3 ‖ 1 · · · ‖x 1 − x N ‖ 1 ‖x 1 − x 2 ‖ 1 0 ‖x 2 − x 3 ‖ 1 · · · ‖x 2 − x N ‖ 1 = ‖x ⎢ 1 − x 3 ‖ 1 ‖x 2 − x 3 ‖ 1 0 ‖x 3 − x N ‖ 1 ⎥ ⎣ . . ... . ⎦ ‖x 1 − x N ‖ 1 ‖x 2 − x N ‖ 1 ‖x 3 − x N ‖ 1 · · · 0 (599)
3.8. MATRIX-VALUED CONVEX FUNCTION 255 where 1 n is a vector of ones having dim1 n = n and where ⊗ represents Kronecker product. This matrix-valued function is convex with respect to the nonnegative orthant since, for each and every Y,Z ∈ R n×N and all 0≤µ≤1 D 1 (µY + (1 − µ)Z) ≼ R N×N + µD 1 (Y ) + (1 − µ)D 1 (Z) (600) 3.8.0.0.3 Exercise. 1-norm distance matrix. The 1-norm is called taxicab distance because to go from one point to another in a city by car, road distance is a sum of grid lengths. Prove (600). 3.8.1 first-order convexity condition, matrix function From the scalar-definition (3.8.0.0.1) of a convex matrix-valued function, for differentiable function g and for each and every real vector w of unit norm ‖w‖= 1, we have w T g(Y )w ≥ w T →Y −X T g(X)w + w dg(X) w (601) that follows immediately from the first-order condition (585) for convexity of a real function because →Y −X T w dg(X) w = 〈 ∇ X w T g(X)w , Y − X 〉 (602) →Y −X where dg(X) is the directional derivative (D.1.4) of function g at X in direction Y −X . By discretized dual generalized inequalities, (2.13.5) g(Y ) − g(X) − →Y −X dg(X) ≽ S M + 0 ⇔ 〈 g(Y ) − g(X) − For each and every X,Y ∈ domg (confer (592)) →Y −X 〉 dg(X) , ww T ≥ 0 ∀ww T (≽ 0) (603) S M∗ + g(Y ) ≽ g(X) + →Y −X dg(X) (604) S M + must therefore be necessary and sufficient for convexity of a matrix-valued function of matrix variable on open convex domain.
- Page 1 and 2: Chapter 3 Geometry of convex functi
- Page 3 and 4: 3.1. CONVEX FUNCTION 213 f 1 (x) f
- Page 5 and 6: 3.1. CONVEX FUNCTION 215 Rf (b) f(X
- Page 7 and 8: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 9 and 10: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 11 and 12: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 13 and 14: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 15 and 16: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 17 and 18: 3.4. INVERTED FUNCTIONS AND ROOTS 2
- Page 19 and 20: 3.5. AFFINE FUNCTION 229 3.4.1.3 po
- Page 21 and 22: 3.5. AFFINE FUNCTION 231 3.5.0.0.2
- Page 23 and 24: 3.6. EPIGRAPH, SUBLEVEL SET 233 q(x
- Page 25 and 26: 3.6. EPIGRAPH, SUBLEVEL SET 235 To
- Page 27 and 28: 3.6. EPIGRAPH, SUBLEVEL SET 237 con
- Page 29 and 30: 3.6. EPIGRAPH, SUBLEVEL SET 239 3.6
- Page 31 and 32: 3.7. GRADIENT 241 2 1.5 1 0.5 Y 2 0
- Page 33 and 34: 3.7. GRADIENT 243 From (1749) andD.
- Page 35 and 36: 3.7. GRADIENT 245 This equivalence
- Page 37 and 38: 3.7. GRADIENT 247 3.7.1.0.3 Example
- Page 39 and 40: 3.7. GRADIENT 249 f(Y ) [ ∇f(X)
- Page 41 and 42: 3.7. GRADIENT 251 meaning, the grad
- Page 43: 3.8. MATRIX-VALUED CONVEX FUNCTION
- Page 47 and 48: 3.8. MATRIX-VALUED CONVEX FUNCTION
- Page 49 and 50: 3.9. QUASICONVEX 259 3.8.3.0.6 Exam
- Page 51 and 52: 3.9. QUASICONVEX 261 3.9.0.0.2 Defi
- Page 53: 3.10. SALIENT PROPERTIES 263 7. - N
3.8. MATRIX-VALUED CONVEX FUNCTION 255<br />
where 1 n is a vector <strong>of</strong> ones having dim1 n = n and where ⊗ represents<br />
Kronecker product. This matrix-valued function is <strong>convex</strong> with respect to the<br />
nonnegative orthant since, for each and every Y,Z ∈ R n×N and all 0≤µ≤1<br />
D 1 (µY + (1 − µ)Z)<br />
≼<br />
R N×N<br />
+<br />
µD 1 (Y ) + (1 − µ)D 1 (Z) (600)<br />
<br />
3.8.0.0.3 Exercise. 1-norm distance matrix.<br />
The 1-norm is called taxicab distance because to go from one point to another<br />
in a city by car, road distance is a sum <strong>of</strong> grid lengths. Prove (600). <br />
3.8.1 first-order <strong>convex</strong>ity condition, matrix function<br />
From the scalar-definition (3.8.0.0.1) <strong>of</strong> a <strong>convex</strong> matrix-valued function,<br />
for differentiable function g and for each and every real vector w <strong>of</strong> unit<br />
norm ‖w‖= 1, we have<br />
w T g(Y )w ≥ w T →Y −X<br />
T<br />
g(X)w + w dg(X) w (601)<br />
that follows immediately from the first-order condition (585) for <strong>convex</strong>ity <strong>of</strong><br />
a real function because<br />
→Y −X<br />
T<br />
w dg(X) w = 〈 ∇ X w T g(X)w , Y − X 〉 (602)<br />
→Y −X<br />
where dg(X) is the directional derivative (D.1.4) <strong>of</strong> function g at X in<br />
direction Y −X . By discretized dual generalized inequalities, (2.13.5)<br />
g(Y ) − g(X) −<br />
→Y −X<br />
dg(X) ≽<br />
S M +<br />
0 ⇔<br />
〈<br />
g(Y ) − g(X) −<br />
For each and every X,Y ∈ domg (confer (592))<br />
→Y −X<br />
〉<br />
dg(X) , ww T ≥ 0 ∀ww T (≽ 0)<br />
(603)<br />
S M∗<br />
+<br />
g(Y ) ≽<br />
g(X) +<br />
→Y −X<br />
dg(X) (604)<br />
S M +<br />
must therefore be necessary and sufficient for <strong>convex</strong>ity <strong>of</strong> a matrix-valued<br />
function <strong>of</strong> matrix variable on open <strong>convex</strong> domain.