v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
80 CHAPTER 2. CONVEX GEOMETRY For n>k A ∩ R k = {x∈ R n | Ax=b} ∩ R k = m⋂ { } x∈ R k | a i (1:k) T x=b i i=1 (137) The result in2.4.2.2 is extensible; id est, any affine subset A also has a vertex-description: 2.5.1.2 ...as span of nullspace basis Alternatively, we may compute a basis for nullspace of matrix A in (135) and then equivalently express the affine subset A as its range plus an offset: Define Z ∆ = basis N(A)∈ R n×n−m (138) so AZ = 0. Then we have the vertex-description in Z , A = {x∈ R n | Ax = b} = { Zξ + x p | ξ ∈ R n−m} ⊆ R n (139) the offset span of n − m column vectors, where x p is any particular solution to Ax = b . For example, A describes a subspace whenever x p = 0. 2.5.1.2.1 Example. Intersecting planes in 4-space. Two planes can intersect at a point in four-dimensional Euclidean vector space. It is easy to visualize intersection of two planes in three dimensions; a line can be formed. In four dimensions it is harder to visualize. So let’s resort to the tools acquired. Suppose an intersection of two hyperplanes in four dimensions is specified by a fat full-rank matrix A 1 ∈ R 2×4 (m = 2, n = 4) as in (136): A 1 ∆ = { ∣ [ ] } ∣∣∣ x∈ R 4 a11 a 12 a 13 a 14 x = b a 21 a 22 a 23 a 1 24 (140) The nullspace of A 1 is two dimensional (from Z in (139)), so A 1 represents a plane in four dimensions. Similarly define a second plane in terms of A 2 ∈ R 2×4 : { ∣ [ ] } ∣∣∣ A ∆ 2 = x∈ R 4 a31 a 32 a 33 a 34 x = b a 41 a 42 a 43 a 2 (141) 44
2.5. SUBSPACE REPRESENTATIONS 81 If the two planes are independent (meaning any line in one is linearly independent [ ] of any line from the other), they will intersect at a point because A1 then is invertible; A 2 { ∣ [ ] [ ]} ∣∣∣ A 1 ∩ A 2 = x∈ R 4 A1 b1 x = (142) A 2 b 2 2.5.1.2.2 Exercise. Linear program. Minimize a hyperplane over affine set A in the nonnegative orthant minimize c T x x subject to Ax = b x ≽ 0 (143) where A = {x | Ax = b}. Two cases of interest are drawn in Figure 26. Graphically illustrate and explain optimal solutions indicated in the caption. Why is α ⋆ negative in both cases? Is there solution on the vertical axis? 2.5.2 Intersection of subspaces The intersection of nullspaces associated with two matrices A∈ R m×n and B ∈ R k×n can be expressed most simply as ([ ]) [ ] A ∆ A N(A) ∩ N(B) = N = {x∈ R n | x = 0} (144) B B the nullspace of their rowwise concatenation. Suppose the columns of a matrix Z constitute a basis for N(A) while the columns of a matrix W constitute a basis for N(BZ). Then [134,12.4.2] N(A) ∩ N(B) = R(ZW) (145) If each basis is orthonormal, then the columns of ZW constitute an orthonormal basis for the intersection. In the particular circumstance A and B are each positive semidefinite [20,6], or in the circumstance A and B are two linearly independent dyads (B.1.1), then N(A) ∩ N(B) = N(A + B), ⎧ ⎨ ⎩ A,B ∈ S M + or A + B = u 1 v T 1 + u 2 v T 2 (l.i.) (146)
- Page 29 and 30: that establish its membership to th
- Page 31 and 32: 31 appendices Provided so as to be
- Page 33 and 34: Chapter 2 Convex geometry Convexity
- Page 35 and 36: 2.1. CONVEX SET 35 2.1.2 linear ind
- Page 37 and 38: 2.1. CONVEX SET 37 2.1.6 empty set
- Page 39 and 40: 2.1. CONVEX SET 39 2.1.7.1 Line int
- Page 41 and 42: 2.1. CONVEX SET 41 (a) R 2 (b) R 3
- Page 43 and 44: 2.1. CONVEX SET 43 This theorem in
- Page 45 and 46: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 47 and 48: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 49 and 50: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 51 and 52: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 53 and 54: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 55 and 56: 2.3. HULLS 55 Figure 16: Convex hul
- Page 57 and 58: 2.3. HULLS 57 The affine hull of tw
- Page 59 and 60: 2.3. HULLS 59 2.3.2 Convex hull The
- Page 61 and 62: 2.3. HULLS 61 In case k = N , the F
- Page 63 and 64: 2.3. HULLS 63 2.3.2.0.3 Exercise. C
- Page 65 and 66: 2.3. HULLS 65 Figure 20: A simplici
- Page 67 and 68: 2.4. HALFSPACE, HYPERPLANE 67 H + a
- Page 69 and 70: 2.4. HALFSPACE, HYPERPLANE 69 1 1
- Page 71 and 72: 2.4. HALFSPACE, HYPERPLANE 71 Recal
- Page 73 and 74: 2.4. HALFSPACE, HYPERPLANE 73 C H
- Page 75 and 76: 2.4. HALFSPACE, HYPERPLANE 75 2.4.2
- Page 77 and 78: 2.4. HALFSPACE, HYPERPLANE 77 (conf
- Page 79: 2.5. SUBSPACE REPRESENTATIONS 79 Ra
- Page 83 and 84: 2.6. EXTREME, EXPOSED 83 2.6 Extrem
- Page 85 and 86: 2.6. EXTREME, EXPOSED 85 A B C D Fi
- Page 87 and 88: 2.7. CONES 87 2.6.1.3.1 Definition.
- Page 89 and 90: 2.7. CONES 89 0 Figure 30: Boundary
- Page 91 and 92: 2.7. CONES 91 2.7.2 Convex cone We
- Page 93 and 94: 2.7. CONES 93 Then a pointed closed
- Page 95 and 96: 2.7. CONES 95 A pointed closed conv
- Page 97 and 98: 2.8. CONE BOUNDARY 97 So the ray th
- Page 99 and 100: 2.8. CONE BOUNDARY 99 2.8.1.1 extre
- Page 101 and 102: 2.8. CONE BOUNDARY 101 2.8.2 Expose
- Page 103 and 104: 2.8. CONE BOUNDARY 103 From Theorem
- Page 105 and 106: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 107 and 108: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 109 and 110: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 111 and 112: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 113 and 114: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 115 and 116: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 117 and 118: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 119 and 120: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 121 and 122: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 123 and 124: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 125 and 126: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 127 and 128: 2.10. CONIC INDEPENDENCE (C.I.) 127
- Page 129 and 130: 2.10. CONIC INDEPENDENCE (C.I.) 129
80 CHAPTER 2. CONVEX GEOMETRY<br />
For n>k<br />
A ∩ R k = {x∈ R n | Ax=b} ∩ R k =<br />
m⋂ { }<br />
x∈ R k | a i (1:k) T x=b i<br />
i=1<br />
(137)<br />
The result in2.4.2.2 is extensible; id est, any affine subset A also has a<br />
vertex-description:<br />
2.5.1.2 ...as span of nullspace basis<br />
Alternatively, we may compute a basis for nullspace of matrix A in (135)<br />
and then equivalently express the affine subset A as its range plus an offset:<br />
Define<br />
Z ∆ = basis N(A)∈ R n×n−m (138)<br />
so AZ = 0. Then we have the vertex-description in Z ,<br />
A = {x∈ R n | Ax = b} = { Zξ + x p | ξ ∈ R n−m} ⊆ R n (139)<br />
the offset span of n − m column vectors, where x p is any particular solution<br />
to Ax = b . For example, A describes a subspace whenever x p = 0.<br />
2.5.1.2.1 Example. Intersecting planes in 4-space.<br />
Two planes can intersect at a point in four-dimensional Euclidean vector<br />
space. It is easy to visualize intersection of two planes in three dimensions;<br />
a line can be formed. In four dimensions it is harder to visualize. So let’s<br />
resort to the tools acquired.<br />
Suppose an intersection of two hyperplanes in four dimensions is specified<br />
by a fat full-rank matrix A 1 ∈ R 2×4 (m = 2, n = 4) as in (136):<br />
A 1 ∆ =<br />
{ ∣ [ ] }<br />
∣∣∣<br />
x∈ R 4 a11 a 12 a 13 a 14<br />
x = b<br />
a 21 a 22 a 23 a 1<br />
24<br />
(140)<br />
The nullspace of A 1 is two dimensional (from Z in (139)), so A 1 represents<br />
a plane in four dimensions. Similarly define a second plane in terms of<br />
A 2 ∈ R 2×4 :<br />
{ ∣ [ ] }<br />
∣∣∣<br />
A ∆ 2 = x∈ R 4 a31 a 32 a 33 a 34<br />
x = b<br />
a 41 a 42 a 43 a 2 (141)<br />
44