v2009.01.01 - Convex Optimization

224 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS
When A is fat full-rank, then AA T is invertible, X ⋆ = A T (AA T ) −1 is the
pseudoinverse A † , and AA † =I . Otherwise, we can make AA T invertible
by adding a positively scaled identity, for any A∈ R m×n
X = A T (AA T + t I) −1 (534)
Invertibility is guaranteed for any finite positive value of t by (1354). Then
matrix X becomes the pseudoinverse X → A † = ∆ X ⋆ in the limit t → 0 + .
Minimizing instead ‖AX − I‖ 2 F yields the second flavor in (1745).
3.1.8.0.3 Example. Hyperplane, line, described by affine function.
Consider the real affine function of vector variable, (confer Figure 63)
f(x) : R p → R = a T x + b (535)
whose domain is R p and whose gradient ∇f(x)=a is a constant vector
(independent of x). This function describes the real line R (its range), and
it describes a nonvertical [173,B.1.2] hyperplane ∂H in the space R p × R
for any particular vector a (confer2.4.2);
{[
∂H =
having nonzero normal
x
a T x + b
η =
[ a
−1
] }
| x∈ R p ⊂ R p ×R (536)
]
∈ R p ×R (537)
This equivalence to a hyperplane holds only for real functions. [ 3.13
] The
R
p
epigraph of real affine function f(x) is therefore a halfspace in , so we
R
have:
The real affine function is to convex functions
as
the hyperplane is to convex sets.
3.13 To prove that, consider a vector-valued affine function
f(x) : R p →R M = Ax + b
having gradient ∇f(x)=A T ∈ R p×M : The affine set
{[ ] }
x
| x∈ R p ⊂ R p ×R M
Ax + b