v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
232 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS 3.1.11.0.1 Exercise. Real fractional function. (confer3.1.4,3.1.7.0.4) Prove that real function f(x,y) = x/y is not convex on the first quadrant. Also exhibit this in a plot of the function. (f is quasilinear (p.242) on {y > 0} , in fact, and nonmonotonic even on the first quadrant.) 3.1.11.0.2 Exercise. Stress function. (confer (1213)) Define |x −y| ∆ = √ (x −y) 2 and X = [x 1 · · · x N ] ∈ R 1×N (68) Given symmetric nonnegative data [h ij ] ∈ S N ∩ R N×N + , consider function f(vec X) = N−1 ∑ i=1 j=i+1 N∑ (|x i − x j | − h ij ) 2 ∈ R (555) Take the gradient and Hessian of f . Then explain why f is not a convex function; id est, why doesn’t second-order condition (554) apply to the constant positive semidefinite Hessian matrix you found. For N = 6 and h ij data from (1284), apply the line theorem from3.2.3.0.1 to plot f along some arbitrary lines through its domain. 3.1.11.1 second-order ⇒ first-order condition For a twice-differentiable real function f i (X) : R p →R having open domain, a consequence of the mean value theorem from calculus allows compression of its complete Taylor series expansion about X ∈ dom f i (D.1.7) to three terms: On some open interval of ‖Y ‖ 2 , so that each and every line segment [X,Y ] belongs to domf i , there exists an α∈[0, 1] such that [342,1.2.3] [37,1.1.4] f i (Y ) = f i (X) + ∇f i (X) T (Y −X) + 1 2 (Y −X)T ∇ 2 f i (αX + (1 − α)Y )(Y −X) (556) The first-order condition for convexity (545) follows directly from this and the second-order condition (554).
3.2. MATRIX-VALUED CONVEX FUNCTION 233 3.2 Matrix-valued convex function We need different tools for matrix argument: We are primarily interested in continuous matrix-valued functions g(X). We choose symmetric g(X)∈ S M because matrix-valued functions are most often compared (557) with respect to the positive semidefinite cone S M + in the ambient space of symmetric matrices. 3.16 3.2.0.0.1 Definition. Convex matrix-valued function: 1) Matrix-definition. A function g(X) : R p×k →S M is convex in X iff domg is a convex set and, for each and every Y,Z ∈domg and all 0≤µ≤1 [186,2.3.7] g(µY + (1 − µ)Z) ≼ µg(Y ) + (1 − µ)g(Z) (557) S M + Reversing sense of the inequality flips this definition to concavity. Strict convexity is defined less a stroke of the pen in (557) similarly to (446). 2) Scalar-definition. It follows that g(X) : R p×k →S M is convex in X iff w T g(X)w : R p×k →R is convex in X for each and every ‖w‖= 1; shown by substituting the defining inequality (557). By dual generalized inequalities we have the equivalent but more broad criterion, (2.13.5) g convex ⇔ 〈W , g〉 convex ∀W ≽ S M + 0 (558) Strict convexity on both sides requires caveat W ≠ 0. Because the set of all extreme directions for the positive semidefinite cone (2.9.2.4) comprises a minimal set of generators for that cone, discretization (2.13.4.2.1) allows replacement of matrix W with symmetric dyad ww T as proposed. △ 3.16 Function symmetry is not a necessary requirement for convexity; indeed, for A∈R m×p and B ∈R m×k , g(X) = AX + B is a convex (affine) function in X on domain R p×k with respect to the nonnegative orthant R m×k + . Symmetric convex functions share the same benefits as symmetric matrices. Horn & Johnson [176,7.7] liken symmetric matrices to real numbers, and (symmetric) positive definite matrices to positive real numbers.
- Page 181 and 182: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 183 and 184: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 185 and 186: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 187 and 188: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 189 and 190: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 191 and 192: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 193 and 194: 2.13. DUAL CONE & GENERALIZED INEQU
- Page 195 and 196: Chapter 3 Geometry of convex functi
- Page 197 and 198: 3.1. CONVEX FUNCTION 197 f 1 (x) f
- Page 199 and 200: 3.1. CONVEX FUNCTION 199 3.1.3 norm
- Page 201 and 202: 3.1. CONVEX FUNCTION 201 A B 1 Figu
- Page 203 and 204: 3.1. CONVEX FUNCTION 203 k/m 1 0.9
- Page 205 and 206: k∑ i=1 3.1. CONVEX FUNCTION 205 S
- Page 207 and 208: 3.1. CONVEX FUNCTION 207 rather x >
- Page 209 and 210: 3.1. CONVEX FUNCTION 209 rather ] x
- Page 211 and 212: 3.1. CONVEX FUNCTION 211 3.1.6.0.2
- Page 213 and 214: 3.1. CONVEX FUNCTION 213 q(x) f(x)
- Page 215 and 216: 3.1. CONVEX FUNCTION 215 3.1.7.0.2
- Page 217 and 218: 3.1. CONVEX FUNCTION 217 We learned
- Page 219 and 220: 3.1. CONVEX FUNCTION 219 Since opti
- Page 221 and 222: 3.1. CONVEX FUNCTION 221 2 1.5 1 0.
- Page 223 and 224: 3.1. CONVEX FUNCTION 223 Setting th
- Page 225 and 226: 3.1. CONVEX FUNCTION 225 Similarly,
- Page 227 and 228: 3.1. CONVEX FUNCTION 227 For vector
- Page 229 and 230: 3.1. CONVEX FUNCTION 229 This means
- Page 231: 3.1. CONVEX FUNCTION 231 f(Y ) −
- Page 235 and 236: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 237 and 238: 3.2. MATRIX-VALUED CONVEX FUNCTION
- Page 239 and 240: 3.3. QUASICONVEX 239 exponential al
- Page 241 and 242: 3.3. QUASICONVEX 241 Unlike convex
- Page 243 and 244: 3.4. SALIENT PROPERTIES 243 6. (af
- Page 245 and 246: Chapter 4 Semidefinite programming
- Page 247 and 248: 4.1. CONIC PROBLEM 247 where K is a
- Page 249 and 250: 4.1. CONIC PROBLEM 249 4.1.1.2 Redu
- Page 251 and 252: 4.1. CONIC PROBLEM 251 In any SDP f
- Page 253 and 254: 4.1. CONIC PROBLEM 253 Proposition
- Page 255 and 256: 4.2. FRAMEWORK 255 sets are closed
- Page 257 and 258: 4.2. FRAMEWORK 257 4.2.1.1.3 Exampl
- Page 259 and 260: 4.2. FRAMEWORK 259 4.2.2 Duals The
- Page 261 and 262: 4.2. FRAMEWORK 261 When equality is
- Page 263 and 264: 4.2. FRAMEWORK 263 The pseudoinvers
- Page 265 and 266: 4.2. FRAMEWORK 265 For the data giv
- Page 267 and 268: 4.2. FRAMEWORK 267 minimizes an aff
- Page 269 and 270: 4.3. RANK REDUCTION 269 whose rank
- Page 271 and 272: 4.3. RANK REDUCTION 271 and where m
- Page 273 and 274: 4.3. RANK REDUCTION 273 4.3.3 Optim
- Page 275 and 276: 4.3. RANK REDUCTION 275 Initialize:
- Page 277 and 278: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 279 and 280: 4.4. RANK-CONSTRAINED SEMIDEFINITE
- Page 281 and 282: 4.4. RANK-CONSTRAINED SEMIDEFINITE
232 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS<br />
3.1.11.0.1 Exercise. Real fractional function. (confer3.1.4,3.1.7.0.4)<br />
Prove that real function f(x,y) = x/y is not convex on the first quadrant.<br />
Also exhibit this in a plot of the function. (f is quasilinear (p.242) on<br />
{y > 0} , in fact, and nonmonotonic even on the first quadrant.) <br />
3.1.11.0.2 Exercise. Stress function. (confer (1213))<br />
Define |x −y| ∆ = √ (x −y) 2 and<br />
X = [x 1 · · · x N ] ∈ R 1×N (68)<br />
Given symmetric nonnegative data [h ij ] ∈ S N ∩ R N×N<br />
+ , consider function<br />
f(vec X) =<br />
N−1<br />
∑<br />
i=1 j=i+1<br />
N∑<br />
(|x i − x j | − h ij ) 2 ∈ R (555)<br />
Take the gradient and Hessian of f . Then explain why f is not a convex<br />
function; id est, why doesn’t second-order condition (554) apply to the<br />
constant positive semidefinite Hessian matrix you found. For N = 6 and<br />
h ij data from (1284), apply the line theorem from3.2.3.0.1 to plot f along<br />
some arbitrary lines through its domain.<br />
<br />
3.1.11.1 second-order ⇒ first-order condition<br />
For a twice-differentiable real function f i (X) : R p →R having open domain,<br />
a consequence of the mean value theorem from calculus allows compression<br />
of its complete Taylor series expansion about X ∈ dom f i (D.1.7) to three<br />
terms: On some open interval of ‖Y ‖ 2 , so that each and every line segment<br />
[X,Y ] belongs to domf i , there exists an α∈[0, 1] such that [342,1.2.3]<br />
[37,1.1.4]<br />
f i (Y ) = f i (X) + ∇f i (X) T (Y −X) + 1 2 (Y −X)T ∇ 2 f i (αX + (1 − α)Y )(Y −X)<br />
(556)<br />
The first-order condition for convexity (545) follows directly from this and<br />
the second-order condition (554).