Lecture 18 Subgradients

More documents

Recommendations

Info

Lecture 18 Subdifferential and Directional Derivatives Definition The directional derivative f ′ (x; d) of f at x along direction d is the following limiting value f ′ f(x + αd) − f(x) (x; d) = lim . α→0 α • When f is convex, the ratio f(x+αd)−f(x) is nondecreasing function of α α > 0, and as α decreases to zero, the ratio converges to some value or decreases to −∞. (HW) Theorem 4 Let x ∈ int(dom f). Then, the directional derivative f ′ (x; d) is finite for all d ∈ R n . In particular, we have f ′ (x; d) = max s∈∂f(x) sT d. Convex Optimization 11
Lecture 18 Proof When x ∈ int(dom f), the subdifferential ∂f(x) is nonempty and compact. Using the subgradient defining relation, we can see that f ′ (x; d) ≥ s T d for all s ∈ ∂f(x). Therefore, f ′ (x; d) ≥ max s∈∂f(x) sT d. To show that actually equality holds, we rely on Separating Hyperplane Theorem. Define C 1 = {(z, w) | z ∈ dom f, f(z) < w}, C 2 = {(y, v) | y = x + αd, v = f(x) + αf ′ (x; d), α ≥ 0}. These sets are nonempty, convex, and disjoint (HW). By the Separating Convex Optimization 12
Page 1 and 2: Lecture 18 Subgradients November 3,
Page 3 and 4: Convex-Constrained Non-differentiab
Page 5 and 6: Lecture 18 Subgradients and Epigrap
Page 7 and 8: Lecture 18 Partial Proof: If ˆx
Page 9 and 10: Lecture 18 Boundedness of the Subdi
Page 11: Lecture 18 Continuity of the Subdif
Page 15 and 16: Lecture 18 Letting z = x, w ↓ f(z
Page 17 and 18: Lecture 18 Examples • The functio
Page 19: Lecture 18 Optimality Conditions: C

Lecture 18 Subgradients

Create successful ePaper yourself

Delete template?

Save as template?