(i) {α - Convex Optimization

Class of problems in CS
Recall y = Hx + ε, ε iid and Var (ε) =σε 2 < +∞,
x = Φα, α (nearly-)sparse.
Typical (equivalent) minimization problems:
(P eq ) : min Ψ (α) s.t. y = HΦα
(P σ ) : min Ψ (α) s.t. ‖y − HΦα‖ l2
≤ σ
(P τ ) : min 1 2 ‖y − HΦα‖2 l 2
s.t. Ψ (α) ≤ τ
(P λ ) : min 1 2 ‖y − HΦα‖2 l 2
+λΨ(α)
(P eq ) is (P σ ) when no noise.
Ψ (α) = ∑ γ∈Γ ψ (α γ).
ψ a sparsity-promoting penalty: non-negative, continuous, even-symmetric, and
non-decreasing on R + , not necessarily smooth at point zero to produce sparse
solutions.
e.g. Ψ(α) =‖α‖ l1
.
Stanford seminar 08-7