v2007.09.13 - Convex Optimization

206 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS3.1.8.0.2 Example. Pseudoinverse.The pseudoinverse matrix is the unique solution to an unconstrained convexoptimization problem [109,5.5.4]: given A∈ R m×nwherewhose gradient (D.2.3)vanishes whenminimizeX∈R n×m ‖XA − I‖2 F (496)‖XA − I‖ 2 F = tr ( A T X T XA − XA − A T X T + I ) (497)∇ X ‖XA − I‖ 2 F = 2 ( XAA T − A T) = 0 (498)XAA T = A T (499)When A is fat full-rank, then AA T is invertible, X ⋆ = A T (AA T ) −1 is thepseudoinverse A † , and AA † =I . Otherwise, we can make AA T invertibleby adding a positively scaled identity, for any A∈ R m×nX = A T (AA T + t I) −1 (500)Invertibility is guaranteed for any finite positive value of t by (1251). Thenmatrix X becomes the pseudoinverse X → A † = ∆ X ⋆ in the limit t → 0 + .Minimizing instead ‖AX − I‖ 2 F yields the second flavor in (1633). 3.1.8.0.3 Example. Hyperplane, line, described by affine function.Consider the real affine function of vector variable,f(x) : R p → R = a T x + b (501)whose domain is R p and whose gradient ∇f(x)=a is a constant vector(independent of x). This function describes the real line R (its range), andit describes a nonvertical [147,B.1.2] hyperplane ∂H in the space R p × Rfor any particular vector a (confer2.4.2);{[∂H =having nonzero normalxa T x + bη =[ a−1] }| x∈ R p ⊂ R p ×R (502)]∈ R p ×R (503)