v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
228 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS 3.1.8.1.4 Exercise. Product and ratio of convex functions. [53, exer.3.32] In general the product or ratio of two convex functions is not convex. [198] However, there are some results that apply to functions on R [real domain]. Prove the following. 3.14 (a) If f and g are convex, both nondecreasing (or nonincreasing), and positive functions on an interval, then fg is convex. (b) If f , g are concave, positive, with one nondecreasing and the other nonincreasing, then fg is concave. (c) If f is convex, nondecreasing, and positive, and g is concave, nonincreasing, and positive, then f/g is convex. 3.1.9 first-order convexity condition, real function Discretization of w ≽0 in (444) invites refocus to the real-valued function: 3.1.9.0.1 Theorem. Necessary and sufficient convexity condition. [53,3.1.3] [110,I.5.2] [342,1.2.3] [37,1.2] [286,4.2] [265,3] For real differentiable function f(X) : R p×k →R with matrix argument on open convex domain, the condition (conferD.1.7) f(Y ) ≥ f(X) + 〈∇f(X) , Y − X〉 for each and every X,Y ∈ domf (544) is necessary and sufficient for convexity of f . When f(X) : R p →R is a real differentiable convex function with vector argument on open convex domain, there is simplification of the first-order condition (544); for each and every X,Y ∈ domf f(Y ) ≥ f(X) + ∇f(X) T (Y − X) (545) From this we can find ∂H − a unique [324,5.5.4] nonvertical [173,B.1.2] hyperplane [ ](2.4), expressed in terms of function gradient, supporting epif X at : videlicet, defining f(Y /∈ domf) = ∆ ∞ [53,3.1.7] f(X) [ ] Y ∈ epif ⇔ t ≥ f(Y ) ⇒ [ ∇f(X) T −1 ]([ ] [ ]) Y X − ≤ 0 t t f(X) (546) 3.14 Hint: Prove the result in (a) by verifying Jensen’s inequality. ⋄
3.1. CONVEX FUNCTION 229 This means, for each and every point X in the domain of a real convex [ function f(X) ] , there exists a hyperplane ∂H − [ in R p × ] R having normal ∇f(X) X supporting the function epigraph at ∈ ∂H −1 f(X) − {[ ] [ ] Y R p [ ∂H − = ∈ ∇f(X) T −1 ]([ ] [ ]) } Y X − = 0 t R t f(X) (547) One such supporting hyperplane (confer Figure 25(a)) is illustrated in Figure 67 for a convex quadratic. From (545) we deduce, for each and every X,Y ∈ domf ∇f(X) T (Y − X) ≥ 0 ⇒ f(Y ) ≥ f(X) (548) meaning, the gradient at X identifies a supporting hyperplane there in R p {Y ∈ R p | ∇f(X) T (Y − X) = 0} (549) to the convex sublevel sets of convex function f (confer (496)) L f(X) f ∆ = {Y ∈ domf | f(Y ) ≤ f(X)} ⊆ R p (550) illustrated for an arbitrary real convex function in Figure 68. 3.1.10 first-order convexity condition, vector function Now consider the first-order necessary and sufficient condition for convexity of a vector-valued function: Differentiable function f(X) : R p×k →R M is convex if and only if domf is open, convex, and for each and every X,Y ∈ domf f(Y ) ≽ R M + f(X) + →Y −X df(X) = f(X) + d dt∣ f(X+ t (Y − X)) (551) t=0 →Y −X where df(X) is the directional derivative 3.15 [190] [288] of f at X in direction Y −X . This, of course, follows from the real-valued function case: by dual generalized inequalities (2.13.2.0.1), 3.15 We extend the traditional definition of directional derivative inD.1.4 so that direction may be indicated by a vector or a matrix, thereby broadening the scope of the Taylor series (D.1.7). The right-hand side of the inequality (551) is the first-order Taylor series expansion of f about X .
- Page 177 and 178: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 179 and 180: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 181 and 182: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 183 and 184: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 185 and 186: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 187 and 188: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 189 and 190: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 191 and 192: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 193 and 194: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 195 and 196: Chapter 3 Geometry of convex functi
- Page 197 and 198: 3.1. CONVEX FUNCTION 197 f 1 (x) f
- Page 199 and 200: 3.1. CONVEX FUNCTION 199 3.1.3 norm
- Page 201 and 202: 3.1. CONVEX FUNCTION 201 A B 1 Figu
- Page 203 and 204: 3.1. CONVEX FUNCTION 203 k/m 1 0.9
- Page 205 and 206: k∑ i=1 3.1. CONVEX FUNCTION 205 S
- Page 207 and 208: 3.1. CONVEX FUNCTION 207 rather x >
- Page 209 and 210: 3.1. CONVEX FUNCTION 209 rather ] x
- Page 211 and 212: 3.1. CONVEX FUNCTION 211 3.1.6.0.2
- Page 213 and 214: 3.1. CONVEX FUNCTION 213 q(x) f(x)
- Page 215 and 216: 3.1. CONVEX FUNCTION 215 3.1.7.0.2
- Page 217 and 218: 3.1. CONVEX FUNCTION 217 We learned
- Page 219 and 220: 3.1. CONVEX FUNCTION 219 Since opti
- Page 221 and 222: 3.1. CONVEX FUNCTION 221 2 1.5 1 0.
- Page 223 and 224: 3.1. CONVEX FUNCTION 223 Setting th
- Page 225 and 226: 3.1. CONVEX FUNCTION 225 Similarly,
- Page 227: 3.1. CONVEX FUNCTION 227 For vector
- Page 231 and 232: 3.1. CONVEX FUNCTION 231 f(Y ) −
- Page 233 and 234: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 235 and 236: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 237 and 238: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 239 and 240: 3.3. QUASICONVEX 239 exponential al
- Page 241 and 242: 3.3. QUASICONVEX 241 Unlike convex
- Page 243 and 244: 3.4. SALIENT PROPERTIES 243 6. (af
- Page 245 and 246: Chapter 4 Semidefinite programming
- Page 247 and 248: 4.1. CONIC PROBLEM 247 where K is a
- Page 249 and 250: 4.1. CONIC PROBLEM 249 4.1.1.2 Redu
- Page 251 and 252: 4.1. CONIC PROBLEM 251 In any SDP f
- Page 253 and 254: 4.1. CONIC PROBLEM 253 Proposition
- Page 255 and 256: 4.2. FRAMEWORK 255 sets are closed
- Page 257 and 258: 4.2. FRAMEWORK 257 4.2.1.1.3 Exampl
- Page 259 and 260: 4.2. FRAMEWORK 259 4.2.2 Duals The
- Page 261 and 262: 4.2. FRAMEWORK 261 When equality is
- Page 263 and 264: 4.2. FRAMEWORK 263 The pseudoinvers
- Page 265 and 266: 4.2. FRAMEWORK 265 For the data giv
- Page 267 and 268: 4.2. FRAMEWORK 267 minimizes an aff
- Page 269 and 270: 4.3. RANK REDUCTION 269 whose rank
- Page 271 and 272: 4.3. RANK REDUCTION 271 and where m
- Page 273 and 274: 4.3. RANK REDUCTION 273 4.3.3 Optim
- Page 275 and 276: 4.3. RANK REDUCTION 275 Initialize:
- Page 277 and 278: 4.4. RANK-CONSTRAINED SEMIDEFINITE
228 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS<br />
3.1.8.1.4 Exercise. Product and ratio of convex functions. [53, exer.3.32]<br />
In general the product or ratio of two convex functions is not convex. [198]<br />
However, there are some results that apply to functions on R [real domain].<br />
Prove the following. 3.14<br />
(a) If f and g are convex, both nondecreasing (or nonincreasing), and<br />
positive functions on an interval, then fg is convex.<br />
(b) If f , g are concave, positive, with one nondecreasing and the other<br />
nonincreasing, then fg is concave.<br />
(c) If f is convex, nondecreasing, and positive, and g is concave,<br />
nonincreasing, and positive, then f/g is convex.<br />
<br />
3.1.9 first-order convexity condition, real function<br />
Discretization of w ≽0 in (444) invites refocus to the real-valued function:<br />
3.1.9.0.1 Theorem. Necessary and sufficient convexity condition.<br />
[53,3.1.3] [110,I.5.2] [342,1.2.3] [37,1.2] [286,4.2] [265,3] For real<br />
differentiable function f(X) : R p×k →R with matrix argument on open<br />
convex domain, the condition (conferD.1.7)<br />
f(Y ) ≥ f(X) + 〈∇f(X) , Y − X〉 for each and every X,Y ∈ domf (544)<br />
is necessary and sufficient for convexity of f .<br />
When f(X) : R p →R is a real differentiable convex function with vector<br />
argument on open convex domain, there is simplification of the first-order<br />
condition (544); for each and every X,Y ∈ domf<br />
f(Y ) ≥ f(X) + ∇f(X) T (Y − X) (545)<br />
From this we can find ∂H − a unique [324,5.5.4] nonvertical [173,B.1.2]<br />
hyperplane [ ](2.4), expressed in terms of function gradient, supporting epif<br />
X<br />
at : videlicet, defining f(Y /∈ domf) = ∆ ∞ [53,3.1.7]<br />
f(X)<br />
[ ] Y<br />
∈ epif ⇔ t ≥ f(Y ) ⇒ [ ∇f(X) T −1 ]([ ] [ ])<br />
Y X<br />
−<br />
≤ 0<br />
t<br />
t f(X)<br />
(546)<br />
3.14 Hint: Prove the result in (a) by verifying Jensen’s inequality.<br />
⋄