Lecture 18 Subgradients

More documents

Recommendations

Info

Definition of Subgradient and Subdifferential Def. A vector s ∈ R n is a subgradient of f at ˆx ∈ dom f when Lecture 18 f(x) ≥ f(ˆx) + s T (x − ˆx) for all x ∈ dom f Def. A subdifferential of f at ˆx ∈ dom f is the set of all subgradients s of f at ˆx ∈ dom f • The subdifferential of f at ˆx is denoted by ∂f(ˆx) • When f is differentible at ˆx, we have ∂f(ˆx) = {∇f(ˆx)} (the subdifferential is a singleton) • Examples f(x) = |x|, ∂f(0) = f(x) = ⎧ ⎨ ⎩ ⎧ ⎨ ⎩ sign(x) for x ≠ 0 [−1, 1] for x = 0 x 2 + 2|x| − 3 for |x| > 1 0 for |x| ≤ 1 Convex Optimization 3
Lecture 18 Subgradients and Epigraph • Let s be a subgradient of f at ˆx: f(x) ≥ f(ˆx) + s T (x − ˆx) • The subgradient inequality is equivalent to −s T ˆx + f(ˆx) ≤ −s T x + f(x) for all x ∈ dom f for all x ∈ dom f • Let f(x) > −∞ for all x ∈ R n . Then epi f = {(x, w) | f(x) ≤ w, x ∈ R n } Thus, −s T ˆx + f(ˆx) ≤ −s T x + w for all (x, w) ∈ epi f, equivalent to [ ] T [ ] [ ] T [ ] −s ˆx −s x ≤ for all (x, w) ∈ epi f 1 f(ˆx) 1 w Therefore, the hyperplane H = { (x, γ) ∈ R n+1 | (−s, 1) T (x, γ) = (−s, 1) T (ˆx, f(ˆx)) } supports epi f at the vector (ˆx, f(ˆx)) Convex Optimization 4
Page 1 and 2: Lecture 18 Subgradients November 3,
Page 3: Convex-Constrained Non-differentiab
Page 7 and 8: Lecture 18 Partial Proof: If ˆx
Page 9 and 10: Lecture 18 Boundedness of the Subdi
Page 11 and 12: Lecture 18 Continuity of the Subdif
Page 13 and 14: Lecture 18 Proof When x ∈ int(dom
Page 15 and 16: Lecture 18 Letting z = x, w ↓ f(z
Page 17 and 18: Lecture 18 Examples • The functio
Page 19: Lecture 18 Optimality Conditions: C

Lecture 18 Subgradients

Create successful ePaper yourself

Delete template?

Save as template?