v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
There is a great race under way to determine which important problemscan be posed in a convex setting. Yet, that skill acquired by understandingthe geometry and application of Convex Optimization will remain more anart for some time to come; the reason being, there is generally no uniquetransformation of a given problem to its convex equivalent. This means,two researchers pondering the same problem are likely to formulate a convexequivalent differently; hence, one solution is likely different from the otherfor the same problem, and any convex combination of those two solutionsremains optimal. Any presumption of only one right or correct solutionbecomes nebulous. Study of equivalence & sameness, uniqueness, and dualitytherefore pervade study of Optimization.Tremendous benefit accrues when an optimization problem can betransformed to its convex equivalent, primarily because any locally optimalsolution is then guaranteed globally optimal. Solving a nonlinear system,for example, by instead solving an equivalent convex optimization problemis therefore highly preferable. 0.1 Yet it can be difficult for the engineer toapply theory without an understanding of Analysis.These pages comprise my journal over an eight year period bridginggaps between engineer and mathematician; they constitute a translation,unification, and cohering of about three hundred papers, books, and reportsfrom several different fields of mathematics and engineering. Beacons ofhistorical accomplishment are cited throughout. Much of what is written herewill not be found elsewhere. Care to detail, clarity, accuracy, consistency,and typography accompanies removal of ambiguity and verbosity out ofrespect for the reader. Consequently there is much cross-referencing andbackground material provided in the text, footnotes, and appendices so asto be self-contained and to provide understanding of fundamental concepts.−Jon DattorroStanford, California20090.1 That is what motivates a convex optimization known as geometric programming,invented in 1960s, [60] [79] which has driven great advances in the electronic circuit designindustry. [34,4.7] [248] [383] [386] [101] [180] [189] [190] [191] [192] [193] [264] [265] [309]8
Convex Optimization&Euclidean Distance Geometry1 Overview 212 Convex geometry 352.1 Convex set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.2 Vectorized-matrix inner product . . . . . . . . . . . . . . . . . 502.3 Hulls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.4 Halfspace, Hyperplane . . . . . . . . . . . . . . . . . . . . . . 722.5 Subspace representations . . . . . . . . . . . . . . . . . . . . . 852.6 Extreme, Exposed . . . . . . . . . . . . . . . . . . . . . . . . . 922.7 Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 962.8 Cone boundary . . . . . . . . . . . . . . . . . . . . . . . . . . 1062.9 Positive semidefinite (PSD) cone . . . . . . . . . . . . . . . . . 1132.10 Conic independence (c.i.) . . . . . . . . . . . . . . . . . . . . . 1402.11 When extreme means exposed . . . . . . . . . . . . . . . . . . 1462.12 Convex polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . 1472.13 Dual cone & generalized inequality . . . . . . . . . . . . . . . 1533 Geometry of convex functions 2193.1 Convex function . . . . . . . . . . . . . . . . . . . . . . . . . . 2203.2 Practical norm functions, absolute value . . . . . . . . . . . . 2253.3 Inverted functions and roots . . . . . . . . . . . . . . . . . . . 2353.4 Affine function . . . . . . . . . . . . . . . . . . . . . . . . . . 2373.5 Epigraph, Sublevel set . . . . . . . . . . . . . . . . . . . . . . 2413.6 Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2509
- Page 1 and 2: DATTORROCONVEXOPTIMIZATION&EUCLIDEA
- Page 3 and 4: Convex Optimization&Euclidean Dista
- Page 5 and 6: for Jennie Columba♦Antonio♦♦&
- Page 7: PreludeThe constant demands of my d
- Page 11 and 12: CONVEX OPTIMIZATION & EUCLIDEAN DIS
- Page 13 and 14: List of Figures1 Overview 211 Orion
- Page 15 and 16: LIST OF FIGURES 1562 Shrouded polyh
- Page 17: LIST OF FIGURES 17130 Elliptope E 3
- Page 21 and 22: Chapter 1OverviewConvex Optimizatio
- Page 23 and 24: ˇx 4ˇx 3ˇx 2Figure 2: Applicatio
- Page 25 and 26: 25Figure 4: This coarsely discretiz
- Page 27 and 28: (biorthogonal expansion) is examine
- Page 29 and 30: 29cardinality Boolean solution to a
- Page 31 and 32: 31Figure 8: Robotic vehicles in con
- Page 33 and 34: an elaborate exposition offering in
- Page 35 and 36: Chapter 2Convex geometryConvexity h
- Page 37 and 38: 2.1. CONVEX SET 372.1.2 linear inde
- Page 39 and 40: 2.1. CONVEX SET 392.1.6 empty set v
- Page 41 and 42: 2.1. CONVEX SET 41(a)R(b)R 2(c)R 3F
- Page 43 and 44: 2.1. CONVEX SET 43where Q∈ R 3×3
- Page 45 and 46: 2.1. CONVEX SET 45Now let’s move
- Page 47 and 48: 2.1. CONVEX SET 47By additive inver
- Page 49 and 50: 2.1. CONVEX SET 49R nR mR(A T )x px
- Page 51 and 52: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 53 and 54: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 55 and 56: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 57 and 58: 2.2. VECTORIZED-MATRIX INNER PRODUC
<strong>Convex</strong> <strong>Optimization</strong>&Euclidean Distance Geometry1 Overview 212 <strong>Convex</strong> geometry 352.1 <strong>Convex</strong> set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.2 Vectorized-matrix inner product . . . . . . . . . . . . . . . . . 502.3 Hulls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.4 Halfspace, Hyperplane . . . . . . . . . . . . . . . . . . . . . . 722.5 Subspace representations . . . . . . . . . . . . . . . . . . . . . 852.6 Extreme, Exposed . . . . . . . . . . . . . . . . . . . . . . . . . 922.7 Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 962.8 Cone boundary . . . . . . . . . . . . . . . . . . . . . . . . . . 1062.9 Positive semidefinite (PSD) cone . . . . . . . . . . . . . . . . . 1132.10 Conic independence (c.i.) . . . . . . . . . . . . . . . . . . . . . 1402.11 When extreme means exposed . . . . . . . . . . . . . . . . . . 1462.12 <strong>Convex</strong> polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . 1472.13 Dual cone & generalized inequality . . . . . . . . . . . . . . . 1533 Geometry of convex functions 2193.1 <strong>Convex</strong> function . . . . . . . . . . . . . . . . . . . . . . . . . . 2203.2 Practical norm functions, absolute value . . . . . . . . . . . . 2253.3 Inverted functions and roots . . . . . . . . . . . . . . . . . . . 2353.4 Affine function . . . . . . . . . . . . . . . . . . . . . . . . . . 2373.5 Epigraph, Sublevel set . . . . . . . . . . . . . . . . . . . . . . 2413.6 Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2509