Lecture 18 Subgradients

Lecture 18
Hyperplane Theorem, there exists a nonzero vector (a, β) ∈ R n+1 such
that
a T (x + αd) + β(f(x) + αf ′ (x; d)) ≤ a T z + βw, (1)
for all α ≥ 0, z ∈ dom f, and f(z) < w. We must have β ≥ 0 - why? We
cannot have β = 0 -why?
Thus, β > 0 and we can divide by β the relation in (1), and obtain with
ã = a/β,
ã T (x + αd) + f(x) + αf ′ (x; d) ≤ ã T z + w, (2)
for all α ≥ 0, z ∈ dom f, and f(z) < w. Choosing α = 0 and letting
w ↓ f(z), we see
ã T x + f(x) ≤ ã T z + f(z),
implying that f(x) − ã T (z − x) ≤ f(z) for all z ∈ dom f. Therefore
−ã ∈ ∂f(x).
Convex Optimization 13