v2007.09.13 - Convex Optimization
v2007.09.13 - Convex Optimization v2007.09.13 - Convex Optimization
566 APPENDIX D. MATRIX CALCULUSIn the case g(X) : R K →R has vector argument, they further simplify:→Ydg(X) = ∇g(X) T Y (1603)→Ydg 2 (X) = Y T ∇ 2 g(X)Y (1604)and so on.→Ydg 3 (X) = ∇ X(Y T ∇ 2 g(X)Y ) TY (1605)D.1.7Taylor seriesSeries expansions of the differentiable matrix-valued function g(X) , ofmatrix argument, were given earlier in (1577) and (1598). Assuming g(X)has continuous first-, second-, and third-order gradients over the open setdomg , then for X ∈ dom g and any Y ∈ R K×L the complete Taylor serieson some open interval of µ∈R is expressedg(X+µY ) = g(X) + µ dg(X) →Y+ 1 →Yµ2dg 2 (X) + 1 →Yµ3dg 3 (X) + o(µ 4 ) (1606)2! 3!or on some open interval of ‖Y ‖g(Y ) = g(X) +→Y −X→Y −X→Y −Xdg(X) + 1 dg 2 (X) + 1 dg 3 (X) + o(‖Y ‖ 4 ) (1607)2! 3!which are third-order expansions about X . The mean value theorem fromcalculus is what insures finite order of the series. [160] [30,1.1] [29, App.A.5][147,0.4]D.1.7.0.1 Exercise. log det. (confer [46, p.644])Find the first two terms of the Taylor series expansion (1607) for log detX .
D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 567D.1.8Correspondence of gradient to derivativeFrom the foregoing expressions for directional derivative, we derive arelationship between the gradient with respect to matrix X and the derivativewith respect to real variable t :D.1.8.1first-orderRemoving from (1578) the evaluation at t = 0 , D.4 we find an expression forthe directional derivative of g(X) in direction Y evaluated anywhere alonga line {X+ t Y | t ∈ R} intersecting domg→Ydg(X+ t Y ) = d g(X+ t Y ) (1608)dtIn the general case g(X) : R K×L →R M×N , from (1571) and (1574) we findtr ( ∇ X g mn (X+ t Y ) T Y ) = d dt g mn(X+ t Y ) (1609)which is valid at t = 0, of course, when X ∈ domg . In the important caseof a real function g(X) : R K×L →R , from (1600) we have simplytr ( ∇ X g(X+ t Y ) T Y ) = d g(X+ t Y ) (1610)dtWhen, additionally, g(X) : R K →R has vector argument,∇ X g(X+ t Y ) T Y = d g(X+ t Y ) (1611)dtD.4 Justified by replacing X with X+ tY in (1571)-(1573); beginning,dg mn (X+ tY )| dX→Y= ∑ k, l∂g mn (X+ tY )Y kl∂X kl
- Page 516 and 517: 516 APPENDIX A. LINEAR ALGEBRA
- Page 518 and 519: 518 APPENDIX B. SIMPLE MATRICESB.1
- Page 520 and 521: 520 APPENDIX B. SIMPLE MATRICESProo
- Page 522 and 523: 522 APPENDIX B. SIMPLE MATRICESB.1.
- Page 524 and 525: 524 APPENDIX B. SIMPLE MATRICESN(u
- Page 526 and 527: 526 APPENDIX B. SIMPLE MATRICESDue
- Page 528 and 529: 528 APPENDIX B. SIMPLE MATRICESB.4.
- Page 530 and 531: 530 APPENDIX B. SIMPLE MATRICEShas
- Page 532 and 533: 532 APPENDIX B. SIMPLE MATRICESFigu
- Page 534 and 535: 534 APPENDIX B. SIMPLE MATRICESB.5.
- Page 536 and 537: 536 APPENDIX C. SOME ANALYTICAL OPT
- Page 538 and 539: 538 APPENDIX C. SOME ANALYTICAL OPT
- Page 540 and 541: 540 APPENDIX C. SOME ANALYTICAL OPT
- Page 542 and 543: 542 APPENDIX C. SOME ANALYTICAL OPT
- Page 544 and 545: 544 APPENDIX C. SOME ANALYTICAL OPT
- Page 546 and 547: 546 APPENDIX C. SOME ANALYTICAL OPT
- Page 548 and 549: 548 APPENDIX C. SOME ANALYTICAL OPT
- Page 550 and 551: 550 APPENDIX D. MATRIX CALCULUSThe
- Page 552 and 553: 552 APPENDIX D. MATRIX CALCULUSGrad
- Page 554 and 555: 554 APPENDIX D. MATRIX CALCULUSBeca
- Page 556 and 557: 556 APPENDIX D. MATRIX CALCULUSwhic
- Page 558 and 559: 558 APPENDIX D. MATRIX CALCULUS⎡
- Page 560 and 561: 560 APPENDIX D. MATRIX CALCULUS→Y
- Page 562 and 563: 562 APPENDIX D. MATRIX CALCULUSD.1.
- Page 564 and 565: 564 APPENDIX D. MATRIX CALCULUSwhic
- Page 568 and 569: 568 APPENDIX D. MATRIX CALCULUSD.1.
- Page 570 and 571: 570 APPENDIX D. MATRIX CALCULUSD.2
- Page 572 and 573: 572 APPENDIX D. MATRIX CALCULUSalge
- Page 574 and 575: 574 APPENDIX D. MATRIX CALCULUStrac
- Page 576 and 577: 576 APPENDIX D. MATRIX CALCULUSD.2.
- Page 578 and 579: 578 APPENDIX D. MATRIX CALCULUS
- Page 580 and 581: 580 APPENDIX E. PROJECTIONThe follo
- Page 582 and 583: 582 APPENDIX E. PROJECTIONFor matri
- Page 584 and 585: 584 APPENDIX E. PROJECTION(⇐) To
- Page 586 and 587: 586 APPENDIX E. PROJECTIONNonorthog
- Page 588 and 589: 588 APPENDIX E. PROJECTIONE.2.0.0.1
- Page 590 and 591: 590 APPENDIX E. PROJECTIONE.3.2Orth
- Page 592 and 593: 592 APPENDIX E. PROJECTIONE.3.5Unif
- Page 594 and 595: 594 APPENDIX E. PROJECTIONE.4 Algeb
- Page 596 and 597: 596 APPENDIX E. PROJECTIONa ∗ 2K
- Page 598 and 599: 598 APPENDIX E. PROJECTIONwhere Y =
- Page 600 and 601: 600 APPENDIX E. PROJECTION(B.4.2).
- Page 602 and 603: 602 APPENDIX E. PROJECTIONis a nono
- Page 604 and 605: 604 APPENDIX E. PROJECTIONE.6.4.1Or
- Page 606 and 607: 606 APPENDIX E. PROJECTIONq i q T i
- Page 608 and 609: 608 APPENDIX E. PROJECTIONThe test
- Page 610 and 611: 610 APPENDIX E. PROJECTIONPerpendic
- Page 612 and 613: 612 APPENDIX E. PROJECTIONE.8 Range
- Page 614 and 615: 614 APPENDIX E. PROJECTIONAs for su
D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 567D.1.8Correspondence of gradient to derivativeFrom the foregoing expressions for directional derivative, we derive arelationship between the gradient with respect to matrix X and the derivativewith respect to real variable t :D.1.8.1first-orderRemoving from (1578) the evaluation at t = 0 , D.4 we find an expression forthe directional derivative of g(X) in direction Y evaluated anywhere alonga line {X+ t Y | t ∈ R} intersecting domg→Ydg(X+ t Y ) = d g(X+ t Y ) (1608)dtIn the general case g(X) : R K×L →R M×N , from (1571) and (1574) we findtr ( ∇ X g mn (X+ t Y ) T Y ) = d dt g mn(X+ t Y ) (1609)which is valid at t = 0, of course, when X ∈ domg . In the important caseof a real function g(X) : R K×L →R , from (1600) we have simplytr ( ∇ X g(X+ t Y ) T Y ) = d g(X+ t Y ) (1610)dtWhen, additionally, g(X) : R K →R has vector argument,∇ X g(X+ t Y ) T Y = d g(X+ t Y ) (1611)dtD.4 Justified by replacing X with X+ tY in (1571)-(1573); beginning,dg mn (X+ tY )| dX→Y= ∑ k, l∂g mn (X+ tY )Y kl∂X kl
Change language