Lyapunov Based Analysis and Controller Synthesis for Polynomial ...

Lyapunov Based Analysis and Controller Synthesis for PolynomialSystems using Sum-of-Squares OptimizationbyZachary William Jarvis-WloszekB.S.E. (Princeton University) 1999M.S. (University of California, Berkeley) 2001A dissertation submitted in partial satisfaction of therequirements for the degree ofDoctor of PhilosophyinEngineering-Mechanical Engineeringin theGRADUATE DIVISIONof theUNIVERSITY OF CALIFORNIA, BERKELEYCommittee in charge:Professor Andrew K. Packard, ChairProfessor J. Karl HedrickProfessor Laurent El GhaouiFall 2003

The dissertation of Zachary William Jarvis-Wloszek is approved:ChairDateDateDateUniversity of California, BerkeleyFall 2003

Lyapunov Based Analysis and Controller Synthesis for PolynomialSystems using Sum-of-Squares OptimizationCopyright Fall 2003byZachary William Jarvis-Wloszek

1AbstractLyapunov Based Analysis and Controller Synthesis for Polynomial Systems usingSum-of-Squares OptimizationbyZachary William Jarvis-WloszekDoctor of Philosophy in Engineering-Mechanical EngineeringUniversity of California, BerkeleyProfessor Andrew K. Packard, ChairThis thesis considers a Lyapunov based approach to analysis and controller synthesisfor systems whose dynamics are described by polynomials. We restrict the candidateLyapunov functions as well as the controllers to be polynomials, so that the conditions inthe Lyapunov theorem involve only polynomials. The Positivstellensatz delineates the exactmanner to ascertain (ie. “certify”) if the theorem’s conditions hold. For computationalreasons we further restrict the choice of certificates to those, which, with fixed Lyapunovfunctions and controllers, can be checked using sum-of-squares optimization.Followingthese steps, we pose convex or coordinatewise convex (convex in one variable when theothers are held fixed) iterative algorithms to search for Lyapunov functions and controllers.We provide a basic review of polynomials, the Positivstellensatz and the sum-ofsquaresoptimization results, which gives the necessary background to follow the subsequent

2developments that lead to our proposed algorithms. First, we consider global stability byconstructing convex algorithms to search for Lyapunov functions that demonstrate semiglobalexponential stability. We then extend these algorithms in a coordinatewise convexform for both state and output feedback controller design.Additionally, we include aconvex procedure to quantify a system’s performance by bounding the induced norm fromdisturbances to outputs. Examples are included for illustration.Since we do not always desire global results, we provide two algorithmic approachesto prove local stability. These approaches are coordinatewise convex and estimate the sizeof the system’s region of attraction by finding the largest level set of a Lyapunov function onwhich the stability theorem’s conditions hold. An example provides a graphical comparisonof the two approaches. As with the global case, we then extend these algorithms to allowfor state and output feedback controller design.Also, we derive bounds for the largestpeak disturbance under which an invariant set remains invariant, and bounds for the localinduced gain from disturbances to outputs on the set.Additionally, we extend the two local asymptotic stability algorithms to discretetime polynomial systems.Unfortunately, the structure of the local asymptotic stabilityLyapunov theorem in discrete time does not allow for controller design using our iterativeapproach.Professor Andrew K. PackardDissertation Committee Chair

To Sarah for her support and encouragementiii

ivContents1 Introduction 11.1 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Summary of Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Polynomial Background 82.1 Sum-of-Squares Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.1 Properties of the Set of SOS Polynomials . . . . . . . . . . . . . . . 102.1.2 Computational Aspects of SOS Polynomials . . . . . . . . . . . . . . 112.2 The Positivstellensatz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.2.2 Theorems Related to the Positivstellensatz . . . . . . . . . . . . . . 232.3 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 Global Analysis and Controller Synthesis 273.1 Stability Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.1.2 Stability Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2 Convex Stability Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.2.1 Stability Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.3 Disturbance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.4 State Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . . 393.4.1 Iterative State Feedback Design Algorithms . . . . . . . . . . . . . . 403.4.2 State Feedback Design Example: Non-Holonomic System . . . . . . 453.4.3 State Feedback Design Example: Nonlinear Spring-Mass System . . 463.5 Output Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . 493.5.1 Iterative Output Feedback Design Algorithms . . . . . . . . . . . . . 513.5.2 Output Feedback Design Example . . . . . . . . . . . . . . . . . . . 553.6 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

v4 Local Stability and Controller Synthesis 604.1 Local Stability Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.2 Convex Stability Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.2.1 Expanding D Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 634.2.2 Expanding Interior Algorithm . . . . . . . . . . . . . . . . . . . . . . 704.2.3 Estimating the Region of Attraction Example . . . . . . . . . . . . . 744.3 Disturbance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.3.1 Reachable Set Bounds under Unit Energy Disturbances . . . . . . . 784.3.2 Set Invariance under Peak Bounded Disturbances . . . . . . . . . . . 824.3.3 Induced L 2 → L 2 Gain . . . . . . . . . . . . . . . . . . . . . . . . . . 844.4 State Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . . 864.4.1 Expanding D Algorithm for State Feedback Design . . . . . . . . . . 864.4.2 Expanding Interior Algorithm for State Feedback Design . . . . . . 914.4.3 State Feedback Design Example . . . . . . . . . . . . . . . . . . . . 954.5 Output Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . 1004.5.1 Expanding D Algorithm for Output Feedback Design . . . . . . . . 1014.5.2 Expanding Interior Algorithm for Output Feedback Design . . . . . 1074.5.3 Output Feedback Design Example . . . . . . . . . . . . . . . . . . . 1114.6 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1165 Discrete Time Containment & Stability 1175.1 Set Invariance for Discrete Time Systems . . . . . . . . . . . . . . . . . . . 1185.1.1 Set Invariance Example . . . . . . . . . . . . . . . . . . . . . . . . . 1185.1.2 Set Invariance under Disturbances . . . . . . . . . . . . . . . . . . . 1195.1.3 Set Invariance under Disturbances Example . . . . . . . . . . . . . . 1205.2 Discrete Time Stability Background . . . . . . . . . . . . . . . . . . . . . . 1215.3 Convex Stability Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1235.3.1 Expanding D Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 1235.3.2 Expanding Interior Algorithm . . . . . . . . . . . . . . . . . . . . . . 1265.3.3 Estimating the Region of Attraction Example . . . . . . . . . . . . . 1295.4 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1306 Conclusions and Recommendations 132Bibliography 135A Semidefinite Programming 140

viAcknowledgementsI would like to thank everyone at Berkeley who has made my time here interesting andexciting. Things would not have been the same without all my friends and fellow graduatestudents, especially Ryan White, Derek Caveney and the BCCI lab.I would also like to thank Pete Seiler for writing the polynomial software classthat I used in almost every line of my code; he has always been a great source of helpfulquestions, comments, and one line counter examples.Additionally, I would like to thank my committee for taking a genuine interest inmy project. Particular thanks go to Andy Packard for giving me the space to go off andtake my own apporach to exploring these topics.

1Chapter 1IntroductionIn this thesis, we consider the problems of stability analysis and controller synthesisfor nonlinear systems with polynomial dynamics. Our approach uses the primary tool ofnonlinear control, Lyapunov functions. The standard method of analysis consists of seeingif one can pick a candidate Lyapunov function such that when it is combined with thesystem, they meet the Lyapunov theorem requirements, which are detailed as needed atthe beginnings of Chapters 3-5. The synthesis approach is similar in that it searches fora candidate Lyapunov function as well as a controller that will make the system fit theassumptions of a Lyapunov theorem. Any Lyapunov function that admits such a controlleris called a Control Lyapunov Function (CLF).Given a CLF, there are many ways to design a controller to meet the Lyapunovassumptions and many of the important design advances are detailed in [17]. Most of thesetechniques are based on backstepping, which is a control design procedure that uses thecontrol input to sequentially counteract the system’s linear and nonlinear dynamics. This

2approach is often quite successful, but it does rely on the system’s dynamics having specialstructure. Additional transformations can be introduced to widen the types of structure forwhich the backstepping concept will work, however, some structure is still required.We approach the candidate Lyapunov function and controller search from a differentangle: optimization. Recent advances in convex optimization, [21], when paired withthe important Positivstellensatz from real algebraic geometry, [3], allow us to use convexoptimization to search for polynomials that meet certain conditions. If we restrict the Lyapunovfunctions and controllers to be polynomials this allows us to formulate the conditionsof the Lyapunov theorems as the constraints of a convex optimization problem.The ideas behind the optimization approach to Lyapunov analysis and synthesisare not new, since they are at the heart of the Linear Matrix Inequality (LMI) approachto linear systems, which is well illustrated in [5]. We use the new optimization results tocreate algorithms that are either convex or stepwise convex to design Lyapunov functionsas well as state and output feedback controllers. These algorithms extend the early resultsof [20] and [13].1.1 Thesis OverviewThis thesis is centered on the questions of proving stability and designing stabilizingcontrollers for polynomial systems.Our approach is based on classic Lyapunovfunction results that can be turned into computationally tractable optimization problemsusing recent theoretical results. In this section we give an outline of this thesis and topicsencountered along the way.

3Chapter 2 provides background material to introduce the important polynomialproperties and concepts that are exploited for analysis and synthesis in the later chapters.First, we introduce the basic polynomial definitions that allow us to define sum-of-squarespolynomials, which allows us to introduce the very important Theorem 2, from [21]. Thistheorem makes the essential link between existence of certain polynomials and convex optimization,and when coupled with Theorem 4, the Positivstellensatz, provides our motivationfor working with polynomial systems. After introducing these theoretical results we applythem to a series of examples and briefly present a few interesting extensions.Chapter 3 considers the problems of proving global stability for polynomial systems.After a review of pertinent elements of Lyapunov stability theory, we pose lemmasfor global asymptotic stability and semi-global exponential stability of polynomial systems.Upon investigation of these, we find that we can construct a pair of algorithms to solvethe semi-global exponential stability problem whose feasibility properties are superior tothose of the global asymptotic stability problem. We then extend these stability algorithmsto consider both state and output feedback controller design problems and provide a testsystem to illustrate their utilization. Additionally, we provide a convex method to bound asystem’s induced L 2 → L 2 gain from disturbance to output.Chapter 4 expands on the global stability results of the previous chapter to studylocal stability. We provide two complementary algorithms to search for Lyapunov functionsto demonstrate local asymptotic stability and approximate the system’s region of attraction.After demonstrating the aspects of the two algorithmic approaches, we look at techniquesto do local disturbance analysis, including finding the maximum peak disturbance such that

4region remains invariant and a local induced L 2 → L 2 gain bound. We then extend thestability algorithms, as in the global case, to both state and output feedback controller design.We apply the synthesis algorithms to find simple controllers that stabilize increasinglydifficult versions of the test system from Chapter 3.Chapter 5 looks at applying the polynomial techniques developed in Chapter 2 tothe discrete time polynomial systems. The discrete time formulation allows us to pose simpleinvariant set problems in a straight forward manner without using Lyapunov functions. Wethen consider the discrete time Lyapunov function framework and find applicable versionsof the local stability algorithms from Chapter 4. Unfortunately, due to the nature of thediscrete time Lyapunov theorem, we can not extend the local stability algorithms to do anyform of controller synthesis.Chapter 6 presents conclusions and gives recommendations for future work.1.2 Thesis ContributionsThis thesis makes several contributions in the areas of stability analysis and controllersynthesis for polynomial systems. These contributions are discussed below.1. Nonlinear Stability Analysis: We provide convex algorithms to construct polynomialLyapunov functions for polynomial dynamic systems that prove either globalor local stability. The global algorithms presented in Chapter 3 construct Lyapunovfunctions that make the system semi-globally exponentially stable, while the two approachesto local stability in both Chapter 4 and 5 yield stepwise convex algorithmsto prove that the system is locally asymptotically stable.As well as constructing

5Lyapunov functions, these algorithms provide estimates of the region of attraction.The local algorithms are applied to both continuous time and discrete time polynomialsystems, and importantly, if the system’s linearization is stable, then the localalgorithms will always be feasible.2. Nonlinear Disturbance Analysis: We investigate the effects of external disturbanceson polynomial systems by constructing a number of convex Lyapunov basedalgorithms to bounds their effects. In Chapter 3 we provide a convex bounding procedurefor the global induced L 2 → L 2 gain from disturbances to system outputs. InChapter 4, we extend this bound so that we can apply it on local invariant regions ofstate space to quantify local system performance. Additionally, we provide bounds forthe largest peak value that a disturbance can have so that a given set remains invariant,as well as, bounds for a system’s reachable set under unit energy disturbances,which previously appeared in [13]. We construct a non-Lyapunov based disturbancepeak bound for set invariance of discrete time systems in Chapter 5.3. Nonlinear Controller Synthesis: We construct stepwise convex algorithms to designstabilizing polynomial controllers for polynomial systems for both the global andthe local cases. In Chapter 3, the global synthesis algorithms find both state and outputfeedback controllers to make the closed loop system semi-globally exponentiallystable. As in the local stability case, we use two different approaches to design bothstate and output feedback locally stabilizing controllers in Chapter 4, as well as estimatetheir domain of attraction. One of the algorithms for state feedback controllerpreviously appeared in [13]. Additionally, the local algorithms are feasible as long the

6system’s linearization is controllable, however, the local approaches are not applicableto discrete time controller design.1.3 Summary of ExamplesExamples of the algorithms and techniques constructed in this thesis are providedfor the following topics in the sections indicated. Additionally, all the software necessary torun the examples listed below is provided at http://jagger.berkeley.edu/~zachary.1. Stability Analysis• Semi-global Exponential Stability: §3.2.1• Local Asymptotic Stability: §4.2.3• Discrete Time Set Invariance §5.1.1• Discrete Time Local Asymptotic Stability: §5.3.32. Disturbance Analysis• Global Induced L 2 → L 2 Gain Bound: §3.4.3 and §3.5.2.• Local Induced L 2 → L 2 Gain Bound: §4.4.3• Bounding Maximum Disturbance Peak for Set Invariance: §4.4.3• Discrete Time Set Invariance under Disturbances: §5.1.33. State Feedback Controller Design• Semi-global Exponential Stabilization: §3.4.2 and §3.4.3

7• Local Asymptotic Stabilization: §4.4.34. Output Feedback Controller Design• Semi-global Exponential Stabilization: §3.5.2• Local Asymptotic Stabilization: §4.5.3

8Chapter 2Polynomial BackgroundThe properties of real polynomials and especially one very important subset ofthem form the basis for the results of later chapters; this chapter provides the necessarybackground for the later results as well as pointers to references for more in-depth treatmentsof the well developed fields that these topics touch upon.First, let R denote the real numbers and Z + denote the set of nonnegative integers,{0, 1, . . .}. Using this notation we can make the formal definitions that will be used in almostevery result.Definition 1 (Monomial) Every α ∈ Z n + defines a function m α : R n → R, called amonomial. Given a specific α ∈ Z n +, the monomial m α maps x ∈ R n into m α (x) = x α :=x α 11 xα 22 · · · xαn n . The degree of a monomial is defined as deg m α := ∑ ni=1 α i.Definition 2 (Polynomial) A polynomial p is defined as a linear combination of a finiteset of monomials {m αj } k j=1 . Given a set of scalar reals, {c j} k j=1 ∈ R, a polynomial p is

9defined as:k∑p := c j m αjj=1or in terms of its action on x ∈ R nk∑k∑p(x) = c j m αj (x) = c j x α jj=1j=1Using the definition of degree for a monomial, the degree of p is defined as deg p :=max j (deg m αj ).The set of polynomials with real coefficients and common independent variables,say, x 1 , . . . , x n , is often denoted as R[x 1 , . . . , x n ] to emphasize that these polynomials forma ring. To eliminate reference to a particular set of independent variables, we will denotethe set of all polynomials in n variables with real coefficients as R n , with the assumptionthat if p ∈ R n and f ∈ R n then p and f are functions of the same independent variables.Additionally, define a subset of R n , R n,d := {p ∈ R n | deg p ≤ d}; this is just theset of all polynomials in n variables that have maximum degree d. If all the monomials ofpolynomial p are of the same degree, say d, then p is called homogeneous and it obeys therelation p(λx) = λ d p(x) for any scalar λ.Another subset of R n is the set of positive semidefinite (PSD) polynomials, whichare nonnegative on all of R n . This set is defined as P n := {p ∈ R n |p(x) ≥ 0, ∀x ∈ R n }.Also define P n,d := P n ∩ R n,d .Following standard notation for the real numbers, we will define any of these setsraised to a integer power, m, to denote an m-vector whose elements are drawn from theindicated set; as an example, R m ndenotes an m-vector of polynomials in n variables.

102.1 Sum-of-Squares PolynomialsA very important subset of the polynomials are the Sum-of-Squares (SOS) polynomials.Let Σ n be the set of all SOS polynomials in n variables, which is defined as{ ∣ ∣∣∣∣ M∑Σ n := s ∈ R n ∃M < ∞, ∃{p i } M i=1 ⊂ R n such that s =The SOS polynomials take their name from the fact that they can be represented as sumsof squares of other polynomials. Additionally, define Σ n,d = Σ n ∩ R n,d .i=1p 2 i}2.1.1 Properties of the Set of SOS PolynomialsSince every s ∈ Σ n is a sum of squared polynomials, it is clear that s(x) ≥ 0,∀x ∈ R n , which implies that Σ n ⊆ P n . An interesting question is whether the set of SOSpolynomials is equal to or strictly contained in the set of positive semidefinite polynomials.Hilbert showed that, when restricted to homogeneous polynomials, there are onlythree cases of n, d such that Σ n,d = P n,d . These results can be translated to generalpolynomials, see §3.2 in [21], to prove that Σ n,d = P n,d only for• Polynomials in one variable, n = 1.• Quadratic polynomials, d = 2.• Quartics in two variables, n = 2, d = 4.Thus, in general Σ n,d ⊂ P n,d . Hilbert’s method to construct a polynomial inP n \ Σ n is very complicated and he did not use it to demonstrate any examples. In [29],Reznick gives an overview of the technique used, its relation to Hilbert’s 17th problem, as

11well as, a series of examples derived by more modern methods. One of the first examplesexhibited dates from 1965 and is the Motzkin polynomial, M(x, y, z) given below, from [19].M(x, y, z) = x 4 y 2 + x 2 y 4 + z 6 − 3x 2 y 2 z 2This polynomial can be shown to be positive semidefinite using the arithmetic-geometricinequality a+b+c3≥ (abc) 1 3 with (a, b, c) = (x 4 y 2 , x 2 y 4 , z 6 ). By methods to be describedlater, §2.1.2, it can be shown to not be SOS.2.1.2 Computational Aspects of SOS PolynomialsWorking with polynomials in P n can be difficult since there is no full parameterizationof the set, nor, in general, are there efficient tests to check if a given polynomial isin the set. However, given the number of variables, n, and degree of polynomials, d, wecan form a full parameterization of Σ n,d , which directly leads to an efficient semidefiniteprogramming test to check if a polynomial is SOS (see Appendix A for a brief overview ofsemidefinite programming).A full parameterization of fixed degree SOS polynomialsFirst we note that SOS polynomials must always be of even degree, so we willconsider the parameterization of the set Σ n,2d for some n, d ∈ Z + . The following lemmaprovides the starting point for the parameterization.Lemma 1 If s ∈ Σ n,2d , then there exist p i ∈ R n,d , i = 1, . . . , M, for some finite M suchthats =M∑i=1p 2 i

12This lemma is a restricted version of Theorem 1 in [28], which gives tighter restrictions onthe p i ’s when s is known.Using Lemma 1, we can pose a full parameterization, often referred to as the “Grammatrix” approach, [7]. First, define z n,d to be the vector of all monomials in n variablesof degree less than or equal to d ordered in the following manner. Given α, β ∈ Z n +, x αprecedes x β if deg x α < deg β or if deg x α = deg β and the first entry of α−β that is strictlynegative is preceded by a strictly positive entry. As an example, with n = 2, d = 2⎡ ⎤1x 1x 2z 2,2 (x) :=x 2 1x 1 x 2⎢ ⎥⎣ ⎦For a general pair of n and d, z n,d (x) will be a ( )n+dd -vector.asx 2 2With the definition of z n,d (x) it is possible to characterize a polynomial, p ∈ R n,2d ,p(x) = z ∗ n,d (x)Qz n,d(x)where Q is the “Gram” matrix. The idea of representing polynomials as quadratic formsof vectors of monomials predates [7] by many years. The earliest quadratic representation,dating from 1968, [4], started a framework which was used to find homogeneous polynomialLyapunov functions in [36]. The following result, which first appeared as Proposition 2.3 in[7], generalizes the earlier works and establishes when a polynomial is SOS.

13Theorem 1 Fix p ∈ R n,2d .p ∈ Σ n,2d if and only if there exists a Q ≽ 0 such thatp(x) = z n,d (x) ∗ Qz n,d (x).Proof:⇒ If p ∈ Σ n,2d then via Lemma 1 we know that there exist p i ∈ R n,d , i = 1, . . . , M suchthat p = ∑ Mi=1 p2 i . Writing each of these polynomials as p i(x) = q ∗ i z n,d(x) with q i a realvector of appropriate dimension, we havep(x) ==M∑i=1( ) 2qi ∗ z n,d (x)M∑zn,d ∗ (x)q iqi ∗ z n,d (x)i=1( M)= zn,d ∗ (x) ∑q i qi∗ z n,d (x)i=1= z ∗ n,d (x)Qz n,d(x)From its construction it is clear that Q ≽ 0.⇐ Since Q ≽ 0, we can factor Q = ∑ ri=1 q i q∗ i where r is the rank of Q. Thenreversing the argument abovep(x) =( r∑zn,d ∗ (x) q i qi∗i=1z n,d (x)r∑= zn,d ∗ (x)q iqi ∗ z n,d (x)=(a)=i=1r∑i=1( ) 2qi ∗ z n,d (x)r∑p 2 i (x)i=1where (a) comes from defining p i (x) := q ∗ i z n,d(x).✷)

14Corollary 1 The number of terms for an SOS decomposition can be chosen to be the numberof elements in z n,d or fewer.This theorem gives necessary and sufficient conditions for a polynomial to beSOS, however for p ∈ Σ n,2d there are, in general, many symmetric Q such that p(x) =z ∗ n,d (x)Qz n,d(x) and some are not positive semidefinite as the following example shows.Example 1 Take p ∈ R 2,4 to be such that p(x) = x 4 1 + x2 1 x2 2 + x4 2 . Both Q 1 and Q 2 beloware such that z2,2 ∗ (x)Q iz 2,2 (x) = p(x).⎡ ⎤∗⎡⎤⎡⎤⎡⎤∗⎡10 0 0 0 0 0110 0 0 0 0 01x 10 0 0 0 0 0x 1x 10 0 0 0 0 0x 1x 20 0 0 0 0 0x 2x 20 0 0 0 0 0x 2== p(x)x 2 10 0 0 1 0 0x 2 1x 2 10 0 0 1 0 1x 2 1x 1 x 20 0 0 0 1 0x 1 x 2x 1 x 20 0 0 0 −1 0x 1 x 2⎢ ⎥ ⎢⎥⎢⎥ ⎢ ⎥ ⎢⎥⎢⎥⎣ ⎦ ⎣⎦⎣⎦ ⎣ ⎦ ⎣⎦⎣⎦x 2 2 0 0 0 0 0 1 x 2 2 x 2 2 0 0 0 1 0 1 x 2 2} {{ }} {{ }Q 1Q 2Note that Q 1 ≽ 0 while Q 2 ⋡ 0. Q 1 ’s positive semidefiniteness shows that p ∈ Σ 2,4 , whileQ 2 ⋡ 0 shows nothing.⎤⎡⎤An LMI Test for SOSTheorem 1 gives the complete parameterization of the SOS polynomials for a givennumber of variables and fixed degree, however it does not give a method to check if a givenpolynomial is SOS.In an effort to understand the set of matrices Q that make zn,d ∗ (x)Qz n,d(x) = p(x)for some p ∈ R n,2d , pick the standard basis for symmetric matrices, {E i }, of the appropriate

15size, ( ) (n+dd × n+d)d . Working out z∗n,d(x) ( ∑ q i E i ) z n,d (x) and equating coefficients withp(x) shows that set of matrices that make the equality hold are an affine subspace of thesymmetric matrices as was shown in [22].Given p ∈ R n,2d , let Q 0 be any symmetric matrix such thatz ∗ n,d (x)Q 0z n,d (x) = p(x)and let {Q i } nqi=1be the set of symmetric matrices such thatz ∗ n,d (x)Q iz n,d (x) = 0With this setup, we can define the affine subspace of symmetric matrices related to p as{n q}Q p := {Q|zn,d ∗ (x)Qz ∑ ∣ ∣∣λin,d = p(x)} = Q 0 + λ i Q i ∈ R, i = 1, . . . , n qThe following example illustrates the general procedure for finding the set of Q i ’s that definethe subspace.i=1Example 2 For p ∈ R 2,4 find some symmetric Q 0 such that z2,2 ∗ (x)Q 0z 2,2 (x) = p(x).Picking the standard basis for symmetric matrices, we write z2,2 ∗ (x)Qz 2,2(x) = 0 as⎡ ⎤1x 1x 2x 2 1x 1 x 2⎢ ⎥⎣ ⎦x 2 2equating terms we have⎤ ⎡ ⎤q 1 q 2 q 3 q 4 q 5 q 61q 2 q 7 q 8 q 9 q 10 q 11x 1q 3 q 8 q 12 q 13 q 14 q 15x 2= 0q 4 q 9 q 13 q 16 q 17 q 18x 2 1q 5 q 10 q 14 q 17 q 19 q 20x 1 x 2⎢⎥ ⎢ ⎥⎣⎦ ⎣ ⎦q 6 q 11 q 15 q 18 q 20 q 21∗ ⎡x 2 2

16Monomial Coefficient1 q 1x 1 2q 2x 2 2q 3x 2 1 q 7 + 2q 4x 1 x 2 2q 8 + 2q 5x 2 2 q 12 + 2q 6x 3 1 2q 9x 2 1 x 2 2q 10 + 2q 13x 1 x 2 2 2q 11 + 2q 14x 3 2 2q 15x 4 1 q 16x 3 1 x 2 2q 17x 2 1 x2 2 q 19 + 2q 18x 1 x 3 2 2q 20x 4 2 q 21and since each coefficient must be identically zero, we can find the subspace of matrices suchthat z ∗ 2,2 (x)Qz 2,2(x) = 0 to be{Q|z2,2(x)Qz ∗ 2,2 (x) = 0} = {λ 1 (2E 7 − E 4 ) +λ} {{ } 2 (E 8 − E 5 ) +λ} {{ } 3 (2E 12 − E 6 )} {{ }Q 1 Q 2+λ 4 (E 10 − E 13 ) +λ} {{ } 5 (E 11 − E 14 )} {{ }Q 4 Q 5+λ 6 (2E 19 − E 18 ) |λ} {{ } 1 , . . . , λ 6 ∈ R}Q 66∑= { λ i Q i |λ i ∈ R}i=1which shows how to find Q p for a polynomial with fixed number of variables and degree.Q 3In [22], Powers and Wörmann got as far as finding the affine subspace, whichallowed them to give the equivalence p ∈ Σ n,2d iff ∃Q ∈ Q p such that Q ≽ 0. However, theydid not recognize that checking if there existed λ i ’s to make Q 0 + ∑ λ i Q i ≽ 0 was convexand just an LMI feasibility problem, instead they proposed a less efficient search methodusing quantifier elimination. In [21], Parrilo realized that the existence of a Q ≽ 0, can besolved as an LMI and gave the following theorem.

17Theorem 2 ([21], Theorem 3.3) Given p ∈ R n,2d , find the relevant affine subspace Q p ={Q 0 + ∑ i λ iQ i |λ i ∈ R}. p ∈ Σ n,2d iff the following LMI is feasible∃λ is.t. Q 0 + ∑ λ i Q i ≽ 0Proof:From Theorem 1, we know that p ∈ Σ n,2d iff there exists a Q ≽ 0 such that z2n,d ∗ (x)Qz n,d(x) =p(x), so we only need search over Q p , which is exactly the LMI given.✷Parrilo also introduced the following important extension that can be proved in asimilar manner to Theorem 2.Theorem 3 ([21], §3.2) Given a finite set {p i } m i=0 ∈ R n, the existence of {a i } m i=1 ∈ Rsuch thatis an LMI feasibility problem.m∑p 0 + a i p i ∈ Σ ni=1This theorem is very useful since it allows to answer questions like the following example.Example 3 Given p 0 , p 1 ∈ R n , does there exist k ∈ R n such thatp 0 + kp 1 ∈ Σ n (2.1)To answer this question, write k as a linear combination of its monomials {m j }, k =∑ sj=1 a jm j . Rewrite (2.1) using this decompositions∑p 0 + kp 1 = p 0 + a j (m j p 1 )j=1

18which, since (m j p 1 ) ∈ R n , can be checked by Theorem 3.A software package, SOSTOOLS [23, 24], was written to aid in solving the LMIsthat result from Theorem 3. This package sets up the LMIs from the polynomial problems,does some smart preprocessing to reduce problem size and uses Sturm’s SeDuMi semidefiniteprogramming solver, [33], to solve the LMIs.Additional computational gains can be had by exploiting polynomial symmetries,[9], and using the Newton polytope algorithm presented in [7] to reduce the number ofmonomials in the Gram matrix formulation, which makes the resulting LMIs smaller withfewer free parameters. These computational improvements are set to appear in the nextrelease of SOSTOOLS.2.2 The PositivstellensatzHaving introduced SOS polynomials it is now possible to make the algebraic definitionsthat are necessary to present one of the seminal theorems of real algebraic geometry,which generalizes many known results including the S-Procedure, as shown in §2.2.1.Definition 3 Given {g 1 , . . . , g t } ∈ R n , the Multiplicative Monoid generated by g j ’s isthe set of all finite products of g j ’s, including the empty product, defined to be 1.It isdenoted as M(g 1 , . . . , g t ). For completeness define M(φ) := 1.An example: M(g 1 , g 2 ) = {g k 11 gk 22 | k 1, k 2 ∈ Z + }

19Definition 4 Given {f 1 , . . . , f s } ∈ R n , the Cone generated by f i ’s isP(f 1 , . . . , f s ) :={s 0 + ∑ }s i b i | s i ∈ Σ n , b i ∈ M(f 1 , . . . , f s )For completeness note that P(φ) := Σ n .Remembering that if f ∈ R n , s ∈ Σ n , then sf 2 ∈ Σ n , allows us to write any cone as a sumof 2 s terms. Note that this reduction in the number of free SOS polynomials need not bebeneficial.An example: P(f 1 , f 2 ) = {s 0 + s 1 f 1 + s 2 f 2 + s 3 f 1 f 2 | s 0 , . . . , s 3 ∈ Σ n }Definition 5 Given {h 1 , . . . , h u } ∈ R n , the Ideal generated by h k ’s is{∑ }I(h 1 , . . . , h u ) := hk p k | p k ∈ R nFor completeness note that I(φ) := 0.With these definitions we can state the following theorem which is a version of theoriginal theorem in [31] restricted to R n .Theorem 4 (Positivstellensatz [3, Theorem 4.2.2] ) Given sets of polynomials {f 1 , . . . , f s },{g 1 , . . . , g t }, and {h 1 , . . . , h u } in R n , the following are equivalent:1. The set ⎧⎪ ⎨⎪ ⎩x ∈ R n ∣ ∣∣∣∣∣∣∣∣∣∣∣f 1 (x) ≥ 0, . . . , f s (x) ≥ 0,g 1 (x) ≠ 0, . . . , g t (x) ≠ 0,h 1 (x) = 0, . . . , h u (x) = 0⎫⎪⎬⎪⎭is empty,

202. There exist polynomials f ∈ P(f 1 , . . . , f s ), g ∈ M(g 1 , . . . , g t ), h ∈ I(h 1 , . . . , h u ) suchthatf + g 2 + h = 02.2.1 ExamplesTo reinforce the usefulness of the Positivstellensatz (P-satz), consider the range ofthe following examples that become convex and thus tractable when the P-satz is combinedwith the results of Theorem 3.Positivstellensatz CertificatesThe LMI based tests for SOS polynomials from Theorem 3 can be used to provethat the set emptiness condition from the P-satz holds, by finding specific f, g, and h suchthat f +g 2 +h = 0. These f, g, and h are known as P-satz certificates since they certify thatthe equality holds. The following theorem states precisely how semidefinite programmingcan be used to search for certificates.Theorem 5 (Theorem 4.8, [21]) Given polynomials {f 1 , . . . , f s }, {g 1 , . . . , g t }, and {h 1 , . . . , h u }in R n , if the set{x ∈ R n |f i (x) ≥ 0, g j (x) ≠ 0, h k (x) = 0, i = 1, . . . , s, j = 1, . . . , t, k = 1, . . . , u}is empty then the search for bounded degree Positivstellensatz refutations can be done usingsemidefinite programming. If the degree bound is chosen large enough the semidefiniteprograms will be feasible and give the refutation certificates.

21The proof of this theorem involves writing out all of the terms of P(f 1 , . . . , f s ),M(g 1 , . . . , g t ), and I(h 1 , . . . , h u ) to form the equality constraint, f + g 2 + h = 0. For afixed degree d, set the term g ∈ M(g 1 , . . . , g t ) such that it is of degree greater than or equalto d/2, then pick each of the free polynomials in f and h such that they have degree atleast d. Now run the LMI with the equality constraint as well as the SOS constraints onthe free polynomials in f. If you search over all g for each d, then you eventually find thePositivstellensatz certificates.An LMI test for P nUsing the P-satz, we can now test to see if a polynomial p ∈ R n is in P n . If p ∈ P n ,then ∀x ∈ R n , p(x) ≥ 0. Equivalently {x ∈ R n |p(x) < 0} is empty, or in the P-satz format{x ∈ R n | − p(x) ≥ 0, p(x) ≠ 0} is emptyThis condition holds iff ∃f ∈ P(−p) and g ∈ M(p) such that f +g 2 = 0. Using the definitionof the cone and the monoid, p ∈ P n iff ∃s 0 , s 1 ∈ Σ n and k ∈ Z + such thats 0 − ps 1 + p 2k = 0If we fix k and the degree of s 1 to be d, we can rewrite the conditions above as p ∈ P n iff∃s 1 such thats 1 ∈ Σ n,dps 1 − p 2k ∈ Σ n, ˆd(2.2)with ˆd = max(2k deg p, d + deg p). For fixed k and d we know that, via Theorem 3, checkingthe conditions in (2.2) is just an LMI, so for fixed k and d we have an LMI sufficient

22condition for a polynomial to be PSD.The S-ProcedureWhat does the familiar S-procedure look like in the Positivstellensatz formalism?Given symmetric n × n matrices {A i } m i=0, the S-procedure states: if there exist nonnegativescalars {λ i } m i=1 such that A 0 − ∑ mi=1 λ iA i ≽ 0, thenm⋂{x ∈ R n |x ∗ A i x ≥ 0} ⊂ {x ∈ R n |x ∗ A 0 x ≥ 0}i=1Rephrased as a set emptiness question, we would like to know ifW := {x ∈ R n |x ∗ A 1 x ≥ 0, . . . , x ∗ A m x ≥ 0, −x ∗ A 0 x ≥ 0, x ∗ A 0 x ≠ 0}is empty?If the λ i exist, define Q := A 0 − ∑ mi=1 λ iA i . By assumption Q ≽ 0 and thusx ∗ Qx ∈ Σ n . Define g(x) := x ∗ A 0 x ∈ M(x ∗ A 0 x) as well asf(x) := (x ∗ Qx)(−x ∗ A 0 x) +m∑λ i (−x ∗ A 0 x)(x ∗ A i x)By their non-negativity each λ i ∈ Σ n and because x ∗ Qx ∈ Σ n we know that the functionf(x) is in the cone P (x ∗ A 1 x, . . . , x ∗ A m x, −x ∗ A 0 x). An easy rearrangement gives f +g 2 = 0,which illustrates that f and g are Positivstellensatz certificates that prove that W is empty.i=1A generalized S-ProcedureThe S-procedure given above can be generalized to deal with non-quadratic functionsand non-scalar weights in the following way. Given {p i } m i=0 ∈ R n, if there exist

23{s i } m i=1 ∈ Σ n such thatm∑p 0 − s i p i = qi=1with q ∈ Σ n , which for fixed degree s i ’s can be checked with Theorem 3, thenm⋂{x ∈ R n |p i (x) ≥ 0} ⊂ {x ∈ R n |p 0 (x) ≥ 0}i=1The related set emptiness question asks ifW := {x ∈ R n |p 1 (x) ≥ 0, . . . , p m (x) ≥ 0, −p 0 (x) ≥ 0, p 0 (x) ≠ 0}is empty. Similar to the standard S-procedure approach, define g := p 0 ∈ M(p 0 ) as well asn∑f := −qp 0 − s i p 0 p iSince q as well as the s i ’s are SOS, f ∈ P(p 1 , . . . , p m , −p 0 ). Verifying f + g 2 = 0,f + g 2 = −qp 0 −(= − p 0 −= 0i=1n∑s i p 0 p i + p 2 0i=1)m∑s i p i p 0 −i=1n∑s i p 0 p i + p 2 0i=1illustrating that f and g provide certificates that the set W is empty.2.2.2 Theorems Related to the PositivstellensatzThe multidimensional moment problem, which considers when a sequence of numbersare the moments of some nonnegative Borel measure on R n , has a long relation withSOS polynomials, [1]. Many interesting results about SOS polynomials have been generatedfrom this approach and two theorems that are especially interesting to polynomial

24optimization are presented below.The setup is as follows; let {f i } m i=1 ∈ R n and defineK := {x|f 1 (x) ≥ 0, · · · , f m (x) ≥ 0}.Theorem 6 (Corollary 3, [30]) If K is compact and p ∈ R n is such that p(x) > 0 forall x ∈ K, then p ∈ P(f 1 , · · · , f m ).This tells us that if a polynomial is positive on K then it is in the cone generated by the polynomialsthat describe K. With one additional assumption this result can be strengthenedfurtherTheorem 7 (Lemma 4.1, [25]) Let K be compact. p ∈ R n such that p(x) > 0 for allx ∈ K belongs to the set{s 0 + f 1 s 1 + · · · + f m s m |s 0 , · · · , s m ∈ Σ n }if and only if there is a polynomial g in the set with the property that g −1 [0, ∞) is compactin R n .These theorems can be used to define certificate searches with fewer terms thanthe P-satz would require, however, these smaller searches can require polynomials of muchhigher degree. In [32], Stengle provides a simple example that requires unbounded degreepolynomial certificates using Theorem 7, but has degree four certificates if the P-satz isused, as shown in [6].

25Applications to Polynomial OptimizationConsider the following optimization problemminx∈R nf 0(x)s.t. f 1 (x) ≥ 0.f m (x) ≥ 0 (2.3)with {f i } m i=0 ∈ R n and no assumed convexity. Let the optimum value be f ⋆ > −∞ withf 0 (x ⋆ ) = f ⋆ .Lasserre, [18], noticed that if the feasible region is compact we can always satisfythe requirements of Theorem 7 by adding an additional constraint on norm of x, f m+1 (x) :=a − ‖x‖ 2 ≥ 0, since fm+1 −1 [0, ∞) is clearly compact. Additionally, if γ is a lower bound onf ⋆ , then f 0 (x) > γ at any feasible point, which implies that f 0 (x) − γ > 0 for {x|f 1 (x) ≥0, · · · , f m+1 (x) ≥ 0}. Using the theorem, we can rewrite the optimization (2.3) asmax γs.t. f 0 − γ = s 0 + f 1 s 1 + · · · + f m+1 s m+1s 0 , · · ·, s m+1 ∈ Σ n (2.4)which Theorem 3 shows to be an LMI, as long as the degree of the SOS polynomials isfixed, however, the degree for which the equality constraint in (2.4) holds is unknown.By increasing the maximum degree of the s i ’s, this approach allows for a series of convexrelaxations to the nonconvex problem (2.3), which are shown to monotonically converge tof ⋆ in [18]. A software package to carry out this algorithm is described in [12].

262.3 Chapter SummaryIn this chapter we provided the polynomial background for all of the results inthe following chapters. Most importantly, we illustrated the convex optimization approachto checking if a given polynomial is a SOS polynomial. Additionally, we introduced thePositivstellensatz and illustrated some of its applications, which we will expand on in thefollowing chapters.

27Chapter 3Global Analysis and ControllerSynthesisIf we consider the systemẋ(t) = f(x(t)) (3.1)for x(t) ∈ R n with f ∈ R n n as well as f(0) = 0, we can pose many global system theoreticquestions about its behavior as searches for SOS polynomials. However, first we need a fewdefinitions that will allow us to make the Lyapunov based stability arguments that will beat the heart of the polynomial searches.Define the flow of the system (3.1) starting from a point x 0 ∈ R n and evolvingforward for t time units to be φ t (x 0 ). Additionally for a differentiable scalar function V ,defined on the same state space as (3.1), define its derivative with respect to time, ˙V , asthe dot product between its gradient, ∇V , and f,˙V (x) := ∇V (x) ∗ f(x)

283.1 Stability BackgroundUsing a polynomial as the Lyapunov function, V , we will be able to prove globalasymptotic stability as well as semi-global exponential stability of the system (3.1) bychecking semialgebraic conditions on polynomials. However, we will first explicitly defineall of the terminology and prove the conditions that we will later exploit to design Lyapunovfunctions.3.1.1 DefinitionsDefinition 6 (Stability) The system (3.1) is stable about x = 0 if for every ɛ > 0 thereexists δ ɛ > 0 such that if ‖x 0 ‖ < δ ɛ , then ‖φ t (x 0 )‖ < ɛ for all t ≥ 0.Definition 7 (Asymptotic Stability) The system (3.1) is asymptotically stable aboutx = 0 if it is stable about x = 0 and, additionally, there exists h > 0 such that if ‖x 0 ‖ < h,then limt→∞‖φ t (x 0 )‖ = 0. Furthermore, if ∀x 0 ∈ R n , limt→∞‖φ t (x 0 )‖ = 0 then the system isglobally asymptotically stable.Definition 8 (Exponential Stability) The system (3.1) is exponentially stable about x =0 if there exist m, c, h > 0 such that‖φ t (x 0 )‖ ≤ me −ct ‖x 0 ‖for all ‖x 0 ‖ < h and t ≥ 0; c is also referred to as a convergence rate for the system.Furthermore, if for every h > 0 there exist m and c that depend on h and validatethe inequality, then the system is semi-globally exponentially stable. If one fixed pair of mand c validate the inequality for all h > 0, then the system is globally exponentially stable.

29Note that both semi-global and global exponential stability imply global asymptotic stability.Additionally, we need to define the following classes of functions.Definition 9 (K ∞ Functions) A function σ : R → R is called a K ∞ function if it iscontinuous, strictly increasing, and has the properties σ(0) = 0 and σ(ξ) → ∞ as ξ → ∞.Definition 10 (Positive Definite Functions) A function ρ : R n → R is called positivedefinite if it is continuous, has the property ρ(0) = 0, and there exists some K ∞ function σsuch thatσ(‖x‖) ≤ ρ(x)for all x ∈ R n .3.1.2 Stability TheoremsWith the definitions above we can state and prove the standard Lyapunov theoremfor global asymptotic stability as well as an extension for semi-global exponential stability.Theorem 8 (Lyapunov) The system (3.1) is globally asymptotically stable about its equilibriumpoint if there exists a positive definite function V : R n → R + such that − ˙V is alsopositive definite.Proof:The proof below follows along the lines of the Lyapunov theorem proof in [16, §3.1].By definition there exist K ∞ functions α, β such that α(‖x‖) ≤ V (x), ∀x ∈ R n andβ(‖x‖) ≤ − ˙V (x), ∀x ∈ R n . We will first prove stability followed by asymptotic convergenceto the fixed point.

30Given any ɛ > 0 pick δ ɛ such thatsup V (x) < α(ɛ)‖x‖ 0, we need to find a T > 0 such that ‖φ t (x 0 )‖ < ɛ for all t ≥ T . If x 0 = 0 then thereis nothing to prove, so we can assume that x 0 ≠ 0. Assume that there exists an x 0 ≠ 0 andɛ > 0 such that ‖φ t (x 0 )‖ ≥ ɛ for all t > 0. By integration we know that∫ tV (φ t (x 0 )) = V (x 0 ) +0˙V (φ τ (x 0 )) dτUsing the positive definiteness of V as well as the fact that ‖φ t (x 0 )‖ ≥ ɛ we can bound theexpression above from below,∫ tα(ɛ) ≤ α(‖φ t (x 0 )‖) ≤ V (φ t (x 0 )) = V (x 0 ) +0˙V (φ τ (x 0 )) dτ

31Additionally using the positive definiteness of − ˙V and the fact that ‖φ t (x 0 )‖ ≥ ɛ we canbound V (φ t (x 0 )) from above,∫ tV (φ t (x 0 )) = V (x 0 ) +0End-to-end we now have˙V (φ τ (x 0 )) dτ ≤ V (x 0 ) −∫ tα(ɛ) ≤ V (x 0 ) − tβ(ɛ)0β(‖φ τ (x 0 )‖) dτ ≤ V (x 0 ) − tβ(ɛ)for all t > 0. However, if t ≥ V (x 0 )/β(ɛ) this implies that α(ɛ) ≤ 0, which contradicts ɛ > 0.✷Building from Theorem 8, we can now prove the following theorem for semi-globalexponential stability.Theorem 9 If there exists a function V , such that V (x) ≥ α‖x‖ d d ∀x ∈ Rn , where α > 0and d is an integer greater than one, as well as a γ > 0 such that˙V (x) ≤ −γV (x)for all x ∈ R n , then the system (3.1) is semi-globally exponentially stable about its fixedpoint, x = 0, with a convergence rate of γ/d.Proof:By definition α‖x‖ d d ≤ V (x), ∀x ∈ Rn .Clearly the function α(·) d is in class K ∞ , andγα‖x‖ d d ≤ γV (x) ≤ − ˙V (x), for all x ∈ R n , which shows that − ˙V is positive definite andtherefore the system (3.1) is globally asymptotically stable about x = 0.For x ≠ 0, V (x) > 0 which allows us to write one of the assumptions as˙V (x)V (x) ≤ −γ

32ordlog(V (x)) ≤ −γdtwhich can be integrated over [0, t] starting from x 0 to givelog(V (φ t (x 0 ))) ≤ log(V (x 0 )) − γtwhich gives the exponential boundV (φ t (x 0 )) ≤ V (x 0 )e −γtthat proves that V (φ t (x 0 )) decays exponentially with rate γ. Since α‖φ t (x 0 )‖ d d ≤ V (φ t(x 0 ))for all t > 0, we can make the following bound( ) V‖φ t (x 0 )‖ d d ≤ (x0 )α‖x 0 ‖ d e −γt ‖x 0 ‖ d ddor‖φ t (x 0 )‖ d ≤ me −ct ‖x 0 ‖ dwith m =(V (x0 )α‖x 0 ‖ d d) 1dand c = γ/d. Since m depends on x 0 and the inequality holds forall x 0 ∈ R n the system is semi-globally exponentially stable. Interestingly the convergencerate can be chosen to be γ/d for all x 0 .✷Remark 1 If the system (3.1) has equilibrium points away from x = 0, then the systemcan not be semi-globally exponentially stable nor can it be globally asymptotically stable.

333.2 Convex Stability TestsWith the previous section’s Lyapunov background, we can now look to designingLyapunov functions for given dynamic systems. Our approach is to design Lyapunovfunctions using the assumptions of Theorems 8 and 9 as constraints of an optimizationthat searches over a class of candidate Lyapunov functions. This optimization approach isextensively used and is usually designed so that the resulting optimization is convex. [5]presents a survey of convex optimization results relating to analysis for linear systems withquadratic Lyapunov functions. These ideas are extended for smooth nonlinear systems withlinearly parameterized non-quadratic Lyapunov functions in [15]. Additionally a convex optimizationapproach using the concept of a dual to the Lyapunov function is provided in[27].Our approach is to consider polynomial systems and restrict the set of candidateLyapunov functions to be polynomials. By doing so, we can formulate the following convexstability tests.Lemma 2 Given the system (3.1) and fixed positive definite functions l 1 , l 2∈ R n , thesystem is globally asymptotically stable if there exists V ∈ R n with V (0) = 0 such thatV − l 1∈ Σ n−(∇V ∗ f + l 2 ) ∈ Σ n .Proof:From Theorem 3, it is clear that the conditions that V − l 1 and −(∇V ∗ f + l 2 ) are SOSpolynomials can be checked as LMIs.If a V is found that meets these conditions, thepositive definiteness of l 1 and l 2 insure that both V and − ˙V are positive definite, which

34meets the assumptions of Theorem 8 making the system globally asymptotically stable.✷Note that if the dynamics were chosen to be linear, f(x) = Ax and the Lyapunov functionto be quadratic V (x) = x ∗ P x then the lemma’s SOS conditions can be simplified to theLMI conditions P ≻ 0 and A ∗ P + P A ≺ 0.A set of sufficient conditions which are very similar to Lemma 2 appear in [20],where they are used to prove non-asymptotic stability for systems like (3.1) with additionalstate and control equality and inequality constraints.Looking to Theorem 9, we can formulate a similar set of polynomial conditions,however due to a term that is bilinear in γ and V , we can not check the assumptions witha single LMI.Lemma 3 Given the system (3.1) and the fixed positive definite function l(x) = ‖x‖ d d withd an integer greater than one, the system is semi-globally exponentially stable if there existsγ > 0 and V ∈ R n with V (0) = 0 such thatV − l−(γV + ∇V ∗ f)∈ Σ n∈ Σ nwhich when V has fixed degree can be checked by performing a linesearch on γ and solvingthe resulting LMI at each point. Additionally, γ/d is a rate of convergence for the system.Proof:The proof follows along the same lines as the proof for Lemma 2 by establishing that theassumptions for Theorem 9 are met.In general the value for d in this and other related lemmas will be picked to be

35the highest even degree in V .✷Again, if the dynamics are linear and the Lyapunov function is quadratic the SOSconditions of Lemma 3 collapse into simple LMIs, which are remain bilinear in γ, P ≻ 0and A ∗ P + P A ≼ −γP .3.2.1 Stability ExamplesConsider the following system⎡ ⎤ ⎡ ⎤x˙1⎢ ⎥⎣ ⎦ = ⎢⎥⎣ ⎦x˙2 x 1 − x 3 2} {{ }f(x)(3.2)where it is clear that f(0) = 0. The linearization about the origin has eigenvalues of ±j, soit is not even possible to verify that the nonlinear system (3.2) is stable.If we look to construct a Lyapunov function to demonstrate stability, a simplequadratic Lyapunov function V (x) = ‖x‖ 2 2will work. This V is clearly positive definite,and if we compute its time derivative we find˙V = ∇V ∗ f= 2x 1 (−x 2 − x 3 1) + 2x 2 (x 1 − x 3 2)= −2(x 4 1 + x 4 2)which clearly makes − ˙V positive definite. The definiteness of V and − ˙V satisfy the assumptionsof Theorem 8, so it is clear that the system (3.2) is globally asymptotically stable.

36d γ max γ/d2 0 04 .0052 .00136 .0431 .00728 .1094 .013710 .1472 .0147Table 3.1: Results of applying Lemma 3 to system (3.2)However, it is unclear if the system is semi-globally exponentially stable, so we will useLemma 3 to set up a linesearch on a sum of squares optimization problem to construct aLyapunov function to show that it is.If we follow the approach given in Lemma 3 for d = {2, 4, 6, 8, 10} we can constructthe table of maximal γ values given in Table 3.1. The table shows that for d = 4 we candemonstrate semi-global exponential stability, since γ max > 0, and that the state decayswith rate γ/d = .0013. As the degree of the Lyapunov function is increased, the maximumfeasible value for γ increases, with γ/d increasing as well. For d > 10, the numerical errorsin solving the resulting LMIs cause the linesearch to become erratic and return lower valuesfor γ max .3.3 Disturbance AnalysisIn the previous section, we considered two algorithmic approaches to constructLyapunov functions that demonstrate specific types of stability. Now, we will consider the

37effect of an external disturbance, w(t) on the following systemẋ = f(x) + g w (x)wy = h(x)(3.3)with x(t) ∈ R n , w(t) ∈ R nw , y(t) ∈ R p , f ∈ R n n, f(0) = 0, g w ∈ R n×nwn , and h ∈ R p n withh(0) = 0. We will follow the Lyapunov approach used in [5, §6.3.2] and presented in thefollowing lemma to find the induced L 2 → L 2 gain from w(t) to y(t).Lemma 4 For a system whose dynamics are given by (3.3) with initial condition x 0 = 0,if there exists a positive definite function V : R n → R + such that˙V (x, w) + h(x) ∗ h(x) − αw ∗ w ≤ 0 (3.4)for all x ∈ R n and w ∈ R nw , then the induced L 2 → L 2 gain from w(t) to y(t) is less thanor equal to √ α.Proof:If we integrate the inequality (3.4) from 0 to T ≥ 0 we have∫ T ()V (φ T (x 0 )) − V (x 0 ) + h(x(t)) ∗ h(x(t)) − αw(t) ∗ w(t) dt ≤ 00and since V (x 0 ) = 0 and V (φ T (x 0 )) ≥ 0 we getwhich completes the proof.✷‖y(t)‖ 2‖w(t)‖ 2≤ √ αNote that if we assume w(t) := 0 then the inequality (3.4) shows that − ˙V is positivesemi-definite and thus that the system is stable.

38We can now use the SOS relaxations from the stability lemmas to pose the followinglemma that provides our best estimate of the induced gain from w(t) to y(t).Lemma 5 The best estimate of the induced L 2 → L 2 gain from w(t) to y(t) for the system(3.3) is √ α where α comes from the solution of the following optimization. Fix the degreeof the Lyapunov function to be d V . Let l ∈ R n be a fixed positive definite polynomial andV ∈ R n,dV with V (0) = 0minVαs.t.V − l ∈ Σ n()− ∇V (x) ∗ (f(x) + g w (x)w) + h(x) ∗ h(x) − αw ∗ w ∈Σ n+nwProof:The proof follows from the proof of Lemma 4.✷Using this lemma we can apply SOS programming to quantify a system’s abilityto reject disturbances.However, this approach can run into feasibility problems, sinceit requires that − ˙V (x, w) have terms to counteract the −h(x) ∗ h(x) in the second SOSconstraint.Examples of the use of this lemma are provided to analyze the disturbancerejection capabilities of the controllers that are designed as examples in §3.4.3 and §3.5.2.

393.4 State Feedback Controller DesignWe will now move from analysis to controller synthesis using Lyapunov techniques.The Lyapunov based approach to controller synthesis has produced many notable resultsand design procedures, and much of their history is given in [17]. Most of these resultscenter on designing a controller from a provided control Lyapunov function, while in ourapproach we use optimization to find both the controller and the Lyapunov function. Arelated approach using convex optimization to design a controller with a dual to the standardLyapunov theorem is shown in [26].As shown in section 3.2, sufficient conditions for the assumptions of Theorems 8and 9 can be checked as LMIs or linesearches on LMIs. The existence of efficient tests forstability analysis brings us to apply similar techniques for controller synthesis. Considernow the systemẋ = f(x) + g(x)u (3.5)for x ∈ R n with f ∈ R n n, f(0) = 0 and u ∈ R m with g ∈ R n×mn . If we allow u to be generatedby a state feedback controller K ∈ R m nwith K(0) = 0, we get the following closed loopsystemẋ = f(x) + g(x)K(x) (3.6)where K is still unknown.Now we can look for conditions on K such that we can find a Lyapunov equationthat meets the assumptions for Theorems 8 and 9. The analog for Lemmas 2 and 3 for thesystem (3.6) are as follows.

40Lemma 6 Given fixed positive definite functions l 1 , l 2 ∈ R n , if there exists V ∈ R n withV (0) = 0 and K ∈ R m nwith K(0) = 0 such thatV − l 1−(∇V ∗ (f + gK) + l 2 )∈ Σ n∈ Σ nthen the system (3.5) is globally asymptotically stabilized by the control law u = K(x).Lemma 7 Given the fixed positive definite function l(x) = ‖x‖ d dwith d an integer greaterthan one, if there exists γ > 0, K ∈ R m nwith K(0) = 0 and V ∈ R n with V (0) = 0 suchthatV − l−(γV + ∇V ∗ (f + gK))∈ Σ n∈ Σ nthen the system (3.5) is semi-globally exponentially stabilized by the control law u = K(x).Additionally, γ/d is a rate of convergence for the closed loop system.The proofs for these lemmas follow exactly along the lines of the proofs of Lemmas2 and 3. However, since both lemmas have conditions that are bilinear in the monomials ofV and K, the SOS conditions of Lemma 6 nor those of 7 will have to be checked iteratively.3.4.1 Iterative State Feedback Design AlgorithmsSince, in general, Lemmas 6 and 7 are not amenable to the semidefinite programmingbased approach of Theorem 3, we will need to employ an iterative approach thatsolves the lemmas’ SOS conditions in V and K by holding one of these polynomials fixedwhile adjusting the other. Even though each step in the iteration will be convex, the overallproblem will remain non-convex. In some cases iterative design procedures of this type canbe shown to converge to answers away from the global optimum as the example in [8] shows.

41However, there is one special case when the SOS conditions in Lemmas 6 and 7can be checked directly with semidefinite programming. This case is when the dynamicsand controller are linear and the Lyapunov function is quadratic. In this case we can use anonlinear change of coordinates from [2] which is often referred to as the “feedback trick,”to yield LMI conditions for Lemma 6 and a linesearch on LMI conditions for Lemma 7.Consider the general case for the SOS conditions of Lemma 6; the second SOScondition is bilinear in V and K as noted above, which indicates that if either is fixed itis an LMI feasibility problem to find the other. The problem with this approach is that ifV is fixed and the resulting convex search for K fails, then there is no way to redesign Vfrom the search on K. The same pitfall occurs if K is held fixed and the search is for V .This approach gives a single shot at finding a controller for a given Lyapunov function orLyapunov function for a fixed controller, but it can not be extended to search for both.The conditions for Lemma 7 have better feasibility properties that allow us topropose a pair of iterative design procedures to establish semi-global exponential stability.The procedures either require either a candidate controller or Lyapunov function to begantheir iterations. First we will consider an algorithm that starts the iterative search froma candidate controller polynomial, the K variant, and then we will look at starting theiterative search from the candidate Lyapunov function, the V variant. Since we can notguarantee a feasible starting point for either algorithm, the K and the V variants can beconsidered two different algorithms.Algorithm 1 (State Feedback: Candidate K Variant) An iterative search to satisfythe SOS conditions of Lemma 7 starting from a candidate controller.

42Let i be the iteration index and set i = 1. Denote the candidate controller K (i=0) ,and pick the maximum degree of the controller and Lyapunov polynomials, d Kand d Vrespectively.1. Fix the controller polynomial K = K (i−1) , set l(x) = ‖x‖ d VdVand solve the followinglinesearch on γ where V ∈ R n,dV with V (0) = 0maxVγs.t.V − l−(γV + ∇V ∗ (f + gK))∈ Σ n∈ Σ n(3.7)Set V (i) = V . If γ max > 0, then the system (3.6) is semi-globally exponentially stablewith controller K (i−1) , else if −∞ < γ max ≤ 0 goto step 2. If γ max = −∞, theiteration is infeasible from the candidate controller K (i−1) , and no stability propertiesof the system (3.6) can be inferred.2. Fix the Lyapunov function V = V (i) and solve the semidefinite programming problemwhere K ∈ R m n,d Kwith K(0) = 0maxKγs.t.(3.8)−(γV + ∇V ∗ (f + gK)) ∈ Σ nSet K (i) = K. If γ max > 0, then the system (3.6) is semi-globally exponentially stablewith controller K (i) . If γ max ≤ 0, increment i and loop back to step 1.If we desire to start from a candidate Lyapunov function instead of a candidate

43controller we can follow the V variant algorithm, which is a trivial reordering of the stepsabove, but as noted in the remark after the algorithm, it is subtly different.Algorithm 2 (State Feedback: Candidate V Variant) An iterative search to satisfythe SOS conditions of Lemma 7 starting from a positive definite candidate Lyapunov function.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) , and pick the maximum degree of the controller and Lyapunov polynomials, d K andd Vrespectively.1. Fix the Lyapunov function V = V (i−1) and solve the semidefinite programming problemwhere K ∈ R m n,d Kwith K(0) = 0maxKγs.t.(3.9)−(γV + ∇V ∗ (f + gK)) ∈ Σ nSet K (i) = K. If γ max > 0, then the system (3.6) is semi-globally exponentially stablewith controller K (i) , if −∞ < γ max ≤ 0 goto to step 2. If γ max = −∞, the iterationis infeasible starting from the candidate Lyapunov function V (i−1) , and no stabilityproperties of the system (3.6) can be inferred.2. Fix the controller polynomial K = K (i) and set l(x) = ‖x‖ d VdV. Solve the following

44linesearch on γ where V ∈ R n,dV with V (0) = 0maxVγs.t.V − l−(γV + ∇V ∗ (f + gK))∈ Σ n∈ Σ n(3.10)Set V (i) = V . If γ max > 0, then the system (3.6) is semi-globally exponentially stablewith controller K (i) , else if γ max ≤ 0, increment i loop back to step 1.Remark 2 (Properties of the State Feedback Algorithms) :• If −∞ < γ max in step 1 of either algorithm, then the rest of the iteration’s searcheswill be feasible, however, this does not mean that a γ max > 0 will necessarily be found.• The degree of the Lyapunov function, d Vshould always be picked to be even, since itneeds to be positive definite.• Since deg f ≥ 1, deg(∇V ∗ (f + gK)) ≥ deg V . This implies that the value of d K needsto be picked so that the degree of ∇V ∗ (f +gK) is even to insure that −(γV +∇V ∗ (f +gK)) is SOS.• In general the V variant algorithm tends to be feasible when the K variant is not.A theoretical rational for this heuristic is as follows. Given a candidate controller,the “likelihood” that it semi-globally exponentially stabilizes the nonlinear system islow, so unless it does it is impossible to find a Lyapunov function.On the otherhand, given a positive definite candidate Lyapunov function, if the system can be

45semi-globally exponentially stabilized, then finding a controller that does so and workswith the candidate Lyapunov function is more likely.The polynomial controller design step of the V variant algorithm can also beviewed as treating the candidate Lyapunov function as a form of control Lyapunov function(CLF), see [17] for background. However, our controller design approach contrasts withthe standard CLF back-stepping procedure by using convex optimization to search for anypolynomial controller of fixed degree instead of matching the controller to counteract thesystem’s dynamics.3.4.2 State Feedback Design Example: Non-Holonomic SystemConsider the bilinear system from example 2 in [34],⎡ ⎤ ⎡⎤⎢ẋ 1⎥⎣ ⎦ = 3x 1 + 4x 2⎢⎥⎣⎦ uẋ 2 −20x 1 + 10x 2Clearly this system fits the format of (3.5) with f = 0, so we can use the algorithmsdesigned in the previous section to design a semi-globally exponentially stabilizing statefeedback controller.Since the system has f = 0, we can not find our candidate V and K by solvingthe control design problem on the system’s linearization. We will instead choose to designa controller by starting Algorithm 2 withV (x) = x 2 1 + x 2 2and setting d V = d K = 2. From this candidate Lyapunov function, we find a controller andLyapunov function pair that achieves γ max = 0.0078 after 2 iterations. The controller that

46the algorithm designs has both quadratic and linear terms, but the linear terms are bothsmaller in magnitude than 10 −6 . If we zero out these tiny terms we find that we still meetthe required SOS conditions. and we are left with the reduced controllerK(x) = 1 ()− 0.426x 2 1 + 2.275x 1 x 2 − 1.404x 2 2100With this controller we can not use the SOS conditions in Lemma 5 to find theinduced L 2 gain from disturbances to the system’s states, h(x) = x, since the closed loopsystem is a homogeneous cubic polynomial. This makes ˙V a homogeneous quartic polynomial,which lacks the necessary quadratic terms to counteract the −h(x) ∗ h(x) = −x ∗ xterm, and thus makes the SOS conditions always infeasible.3.4.3 State Feedback Design Example: Nonlinear Spring-Mass SystemWe will design a state feedback controller following Algorithms 1 and 2 for thespring-mass system given below where x 1 , x 3 represent the displacement of m 1 , m 2 respectively,the k’s identify the springs, d is a anti-damper, and u is a forcing input.k 1✲ u k 2✁❆❆❆✁✁ ✁❆ ❆ ❆ ✁✁❆❆❆✁✁ ✁❆ ❆ ❆ ✁m 1 ✲ x 1m 2 ✲ x 3dFor simplicity we will consider only unit masses, m 1 = m 2 = 1.Let k 1 be astiffening spring that generates the displacement dependent forceF k1 (x) = x 1 + 1 10 x3 1and let the forces generated by k 2 and d be linear in the relative displacement and velocity

47respectively with d chosen to provide negative damping to make the system unstableF k2 = x 3 − x 1F d = − 1 10 (x 4 − x 2 ).We can now write the system’s dynamics as⎡ ⎤ ⎡⎤ ⎡ ⎤ẋ 1x 20ẋ 2−(x 1 + 110 =x3 1 ) + (x 3 − x 1 ) − 110 (x 4 − x 2 )1+u (3.11)ẋ 3x 40⎢ ⎥ ⎢⎥ ⎢ ⎥⎣ ⎦ ⎣⎦ ⎣ ⎦ẋ 4 −(x 3 − x 1 ) + 110 (x 4 − x 2 )0} {{ } }{{}f(x)g(x)The linearization of the system (3.11) has eigenvalues with positive real part, so the uncontrolledsystem is unstable.We want to design a state feedback controller, u = K(x), to semi-globally exponentiallystabilize the systemẋ = f(x) + g(x)uwhere f, g are defined in (3.11). However, we need to start with either a candidate controlleror Lyapunov function, and we will find both by solving the linearized version of the problemto find a linear controller and a quadratic Lyapunov function.Letting A fand B g be the linearizations of f and g about the point x = 0 thelinearized system isẋ = A f x + B g u (3.12)It is possible to use the “feedback trick” from [2] to pose the problem of finding a linearstatic state feedback controller, u = K lin (x) = Kx, for system (3.12) with a quadratic

48Lyapunov function that demonstrates semi-global exponential stability, V quad (x) = x ∗ P x,as the following linesearch with LMI constraintsmaxγs.t.Q ≻ 0(3.13)−γQ≽ QA ∗ f + A f Q + L ∗ B ∗ g + B g Lwhere P = Q −1 and K = LP .Using K lin as found by (3.13) as the candidate controller to start Algorithm 1 withd V = 2 or d V = 4 makes the SOS conditions (3.7) infeasible, thus γ max = −∞. This impliesthat the K variant of the algorithm fails to tell us anything about the possibility of semigloballyexponentially stabilizing the unstable system (3.11). However if we start Algorithm2 with V quad as the candidate Lyapunov function and fix d K = 3, then a controller is foundin the first optimization (3.9) that makes γ max = 1.2311. However, if we start Algorithm2 with d K = 1, it fails which shows that we need to go to d K = 3 to achieve semi-globalexponential stability.The success of the V variant while the K variant fails is not that strange when weconsider what the SOS optimizations were trying to find. In the K case, we were looking fora Lyapunov function that demonstrates that the nonlinear system with the linear controllerwrapped around it is semi-global exponentially stable. Whereas, in the V case we wereusing the quadratic V in the manner of a control Lyapunov function and designing a thirdorder controller to make the necessary SOS condition hold.The controller found in the V case is a degree three polynomial in 4 variables, so ithas 34 terms. Since no part of the optimization tries to reduce the number of nonzero terms,

49all of them are used. If all the terms whose coefficients are less than 10 −6 in absolute valueare set to zero the number of used terms drops to 24 and the reduced controller continuesto make the system globally exponentially stable with the same value of γ max .Performance of the State Feedback ControllerNow that we have designed a polynomial state feedback controller for the examplesystem, we can analyze its performance by using Lemma 5 to compute a bound on theinduced L 2 gain from disturbances to the states by selecting h(x) = x.If we allow adisturbance to enter the system through the control channel, g w = g, and keep d V = 2,then following Lemma 5 we get the following performance bound‖x(t)‖ 2‖w(t)‖ 2≤ 0.686which shows that the system is non-expansive.3.5 Output Feedback Controller DesignWe can expand the results for state feedback controllers by allowing the controllerto be a dynamic system and by limiting the information that it receives. In many cases, theexisting output feedback controller design schemes (ie. back stepping [17]) formulate theproblem by making the Lyapunov function depend only on the system’s outputs, however,we will continue to require that the Lyapunov function depend on all the system’s and thecontroller’s states.Again, we will be searching for ways to validate the assumptions of Theorem 9using an iterative procedure to check SOS conditions. As in the state feedback case, we will

50not consider the globally asymptotic stability case, since its set up will again suffer fromthe infeasiblity properties that were illustrated in the previous section.Define the system to be controlled asẋy= f(x) + g(x)u= h(x)(3.14)for x ∈ R n with f ∈ R n n, f(0) = 0, u ∈ R m , g ∈ R n×mn with y ∈ R p , h ∈ R p n and h(0) = 0.If we allow u to be generated by an unknown n ξ -state dynamic output feedback controllerof the form˙ξu= A(ξ) + B(ξ)y= C(ξ) + D(ξ)y(3.15)for ξ ∈ R n ξwith A ∈ R n ξn ξ, A(0) = 0, B ∈ R n ξ×p, C ∈ R m n ξ, C(0) = 0 and D ∈ R m×p . Withn ξn ξthis controller structure, the closed loop system becomesẋ˙ξ= f(x) + g(x)(C(ξ) + D(ξ)h(x))= A(ξ) + B(ξ)h(x).(3.16)By the assumptions on f, h, A, C, we know that the combined system has a fixed point at[x; ξ] = [0; 0], and we can pose the following analog of Lemma 3 to test the closed loopsystem (3.16) for semi-global exponential stability.Lemma 8 Let n ξ be a fixed positive integer, and l([x; ξ]) = ‖[x; ξ]‖ d dwith d some integergreater than one. If there exists γ > 0, A ∈ R n ξn ξ, A(0) = 0, B ∈ R n ξ×pn ξ, C ∈ R m n ξ, C(0) = 0,D ∈ R m×pn ξand V ∈ R n+nξ with V (0) = 0 such thatV − l⎛ ⎡⎤⎞− ⎜⎝ γV + ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠A + Bh∈ Σ n+nξ∈ Σ n+n ξ

51then the system (3.14) is semi-globally exponentially stabilized by the controller (3.15). Additionally,γ/d is a rate of convergence for the closed loop system.As before this lemma can be proved along the lines of Lemma 2 and again it isbilinear in the elements of the controller system and the Lyapunov function. However, unlikethe earlier lemmas, when the systems are linear and the Lyapunov function is quadraticwe can not use the “feedback trick” to check the conditions in Lemma 8 with a singlesemidefinite program, although, by the separation theorem, we could design separately anobserver and a state feedback controller.3.5.1 Iterative Output Feedback Design AlgorithmsIn their stated forms, the SOS conditions of Lemma 8 do not fit into the set up forTheorem 3 since they are bilinear in the controller system and the Lyapunov function. Toget around this problem we will propose two iterative approaches that are rather similar toAlgorithms 1 and 2 that also start from the candidate controller, the A, B, C, D variant, orthe Lyapunov function, the V variant. As in the state feedback case, since neither algorithmcan be guaranteed to be feasible the V and the A, B, C, D variants can be considered twodifferent algorithms.Algorithm 3 (Output Feedback: Candidate A, B, C, D Variant) An iterative searchto satisfy the SOS conditions of Lemma 8 starting from a candidate controller.Let i be the iteration index and set i = 1. Denote the candidate controller A (i=0) ,B (i=0) , C (i=0) , D (i=0) , and pick the maximum degree of the controller and Lyapunov polynomials,d A , d B , d C , d D and d Vrespectively.

521. Fix the controller polynomials A = A (i−1) , B = B (i−1) , C = C (i−1) , D = D (i−1) , andset l([x; ξ]) = ‖[x; ξ]‖ d VdV. Solve the following linesearch on γ where V ∈ R n+nξ ,d VwithV (0) = 0maxVγs.t.V − l ∈ Σ n+nξ(3.17)⎛ ⎡⎤⎞− ⎜⎝ γV + ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠ ∈ Σ n+n ξA + BhSet V (i) = V .If γ max > 0, then the system (3.14) is semi-globally exponentiallystable with controller A (i−1) , B (i−1) , C (i−1) , D (i−1) , else if −∞ < γ max ≤ 0 goto step2. If γ max = −∞, the iteration is infeasible starting from the candidate controllerA (i−1) , B (i−1) , C (i−1) , D (i−1) , and no stability properties of the system (3.14) can beinferred.2. Fix the Lyapunov function V = V (i) , and solve the semidefinite programming problemwhere A ∈ R n ξn ξ ,d Awith A(0) = 0, B ∈ R n ξ×pn ξ ,d B, C ∈ R m n ξ ,d C, C(0) = 0, and D ∈ R m×pn ξ ,d DmaxA,B,C,Dγs.t.⎛ ⎡⎤⎞(3.18)− ⎜⎝ γV + ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠ ∈ Σ n+n ξA + BhSet A (i) = A, B (i) = B, C (i) = C, D (i) = D. If γ max > 0, then the system (3.14)is semi-globally exponentially stable with controller A (i) , B (i) , C (i) , D (i) , if γ max ≤ 0,increment i loop back to step 1.

53As with the state feedback case, if we want to start from a candidate Lyapunovfunction instead of a candidate controller we can reorder the steps above to follow the Vvariant of the algorithm.Algorithm 4 (State Feedback: Candidate V Variant) An iterative search to satisfythe SOS conditions of Lemma 8 starting from a positive definite candidate Lyapunov function.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) , and pick the maximum degree of the controller and Lyapunov polynomials, d A , d B ,d C , d D and d Vrespectively.1. Fix the Lyapunov function V = V (i−1) , solve the semidefinite programming problemwhere A ∈ R n ξn ξ ,d Awith A(0) = 0, B ∈ R n ξ×pn ξ ,d B, C ∈ R m n ξ ,d C, C(0) = 0, and D ∈ R m×pn ξ ,d DmaxA,B,C,Dγs.t.⎛ ⎡⎤⎞(3.19)− ⎜⎝ γV + ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠ ∈ Σ n+n ξA + BhSet A (i) = A, B (i) = B, C (i) = C, D (i) = D. If γ max > 0, then the system (3.14) issemi-globally exponentially stable with controller A (i) , B (i) , C (i) , D (i) , if −∞ < γ max ≤0 goto to step 2. If γ max = −∞, the iteration is infeasible starting from the candidateLyapunov function V , and no stability properties of the system (3.14) can be inferred.2. Fix the controller polynomials A = A (i) , B = B (i) , C = C (i) , D = D (i) and setl([x; ξ]) = ‖[x; ξ]‖ d VdV. Solve the following linesearch on γ where V ∈ R n+nξ ,d Vwith

54V (0) = 0maxVγs.t.V − l ∈ Σ n+nξ(3.20)⎛ ⎡⎤⎞− ⎜⎝ γV + ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠ ∈ Σ n+n ξA + BhSet V (i) = V . If γ max > 0, then the system (3.14) is semi-globally exponentially stablewith controller A (i) , B (i) , C (i) , D (i) , else if γ max ≤ 0, increment i and loop back to step1.Remark 3 (Properties of the Output Feedback Algorithms) :• If −∞ < γ max in step 1 of either algorithm, then the rest of the iteration’s searcheswill be feasible, however, this does not mean that a γ max > 0 will necessarily be found.• The degree of the Lyapunov function, d Vshould always be picked to be even, since itneeds to be positive definite.• Since deg f ≥ 1,⎛ ⎡⎤⎞deg ⎜⎝ ∇V ∗ ⎢f + gC + gDh⎥⎟⎣⎦⎠ ≥ deg VA + BhThis implies that the values of d A , d B , d C , d D need to be chosen so that the degree of∇V ∗ [f + gC + gDh; A + Bh] is even. If it were odd, then the SOS conditions requiredfor stability could never be satisfied.• In the output feedback case we can not employ the “feedback trick” so we can notstart the algorithm from its linear analog. Due to this, we must find other candidate

55controller and Lyapunov polynomials, and the selection of these greatly influences thefeasibility of the two algorithm variants. If a good controller starting point can found,then Algorithm 3 will tend to be feasible more often than Algorithm 4. Conversely, ifa good candidate Lyapunov function can be found, then the V variant will tend to befeasible more often.3.5.2 Output Feedback Design ExampleConsider again the spring mass system used in §3.4.3, whose dynamics are givenby (3.11). If we make the system’s output⎡ ⎤y = h(x) = ⎢x 1⎥⎣ ⎦x 2the problem fits the form of (3.14), so we can try to design an output feedback controllerfor the system using Algorithms 3 and 4.First, we need to find a candidate controller and a candidate Lyapunov functionso that we can compare the A, B, C, D and the V variants. Unlike the state feedback case,we can not just linearize the problem and solve the resulting LMIs, so we will first constructa robust linear controller and its Lyapunov function to start the algorithms.We choose to start the algorithms with a robust linear controller in the hopethat its robustness will compensate for the system’s nonlinearities which can be viewed asuncertainties. In this light, we will maximize our robustness against plant uncertainty bypicking the candidate controller (A, B, C, D) to minimize the H ∞ gain from d to e for theblock diagram shown in Figure 3.1, which will give optimal robustness as measured by thegap metric [10].

56e 1✻d 1 ✲ ❝ ✲✻A fC h 0B g✲ e 2ACBD✛❄❝ ✛ d 2Figure 3.1: The Gap Metric Block DiagramIf n ξ = n, we can use standard software to find the H ∞ controller (A, B, C, D)that minimizes the gain from d to e in the block diagram above. If this controller achievesa gain from d to e of β then if we denote the closed loop system A c , B c , C c , D c the positivedefinite matrix X that makes⎡⎤A ∗ cX + XA c XB c Cc∗ Bc ∗ X −βI Dc∗ ≺ 0 (3.21)⎢⎥⎣⎦C c D c −βIprovides a quadratic Lyapunov function [x; ξ] ∗ X[x; ξ] to use as a candidate Lyapunov function.If n ξ ≠ n, then we can devise a candidate controller and Lyapunov function as follows.First find a random n ξ state controller that stabilizes the linearized system (A f , B g , C h , 0).Since the closed loop system (A c , B c , C c , D c ) is linear in the controller (A, B, C, D), we canstart with the random stabilizing controller and iterate between adjusting the controllerand the Lyapunov function to minimize β subject to X ≻ 0 and the LMI constraint (3.21)With the candidate controller and Lyapunov function found by the above methodwe can use Algorithms 3 and 4. If we first search for a full order controller, n ξ = 4, we

57can find a linear controller, d A = d C = 1 and d B = d D = 0, that semi-globally exponentiallystabilizes the nonlinear system using both the V and the A, B, C, D variants of thealgorithm, as shown by Table 3.2.A, B, C, D variant V variantV (1) , A (0) , B (0) , C (0) , D (0) γ = −8.88 V (0) , A (1) , B (1) , C (1) , D (1) γ = −28.63V (1) , A (1) , B (1) , C (1) , D (1) γ = −6.41 V (1) , A (1) , B (1) , C (1) , D (1) γ = −12.46V (2) , A (1) , B (1) , C (1) , D (1) γ = −0.09 V (1) , A (2) , B (2) , C (2) , D (2) γ = −3.08V (2) , A (2) , B (2) , C (2) , D (2) γ = +0.12 V (2) , A (2) , B (2) , C (2) , D (2) γ = −0.09– V (2) , A (3) , B (3) , C (3) , D (3) γ = +0.06Table 3.2: The results of Algorithms 3&4 on (3.11) with h = [x 1 ; x 2 ] and n ξ = 4.If we reduce the number of controller states so that n ξ = 2, we can still find a linearcontroller that semi-globally exponentially stabilizes the system. However, only Algorithm3, the A, B, C, D variant, works while the V variant is infeasible. The final two state linearcontroller and closed loop Lyapunov function are below⎡⎡ ⎤1.37 −225.10 −240.10 −13.94A B⎢ ⎥⎣ ⎦ = 190.19 −2040.73 −2119.78 −122.79⎢C D ⎣715.60 3009.76 −3654.87 −3778.65⎤⎥⎦and⎡X =⎢⎣⎤2.67 −0.17 −0.69 0.21 0.65 −0.59−0.17 1.39 −0.34 0.39 1.45 2.56−0.69 −0.34 1.17 −0.23 0.07 −0.480.21 0.39 −0.23 1.35 −0.09 0.560.65 1.45 0.07 −0.09 93.84 88.98⎥⎦−0.59 2.56 −0.48 0.56 88.98 87.15

58while the progress of the iteration is shown in Table 3.3A, B, C, D variant n ξ = 2V (1) , A (0) , B (0) , C (0) , D (0) γ = −9.57V (1) , A (1) , B (1) , C (1) , D (1) γ = −7.62V (2) , A (1) , B (1) , C (1) , D (1) γ = −0.10V (2) , A (2) , B (2) , C (2) , D (2) γ = −0.10V (3) , A (2) , B (2) , C (2) , D (2) γ = −0.09V (3) , A (3) , B (3) , C (3) , D (3) γ = +1.45Table 3.3: Progress of Algorithm 3 on (3.11) with h = [x 1 ; x 2 ] and n ξ = 2.However, if we reduce the number of controller states again by setting n ξ = 1, wecan not find a linear controller to semi-globally exponentially stabilize the system.Also, if we set h = [x 3 ; x 4 ], which separates the control and the observations,both of the algorithms turn out to be infeasible for the full order controller case when thecontroller is allowed to be linear or cubic with a quadratic or quartic Lyapunov function.Performance of the Output Feedback ControllersAs with the state feedback design example, we can now look at the performanceof the controllers derived above, by bounding the system’s induced L 2 gain. We will againallow the disturbance to enter in the control channel making g w = g. However, we foundthat, in these examples, using the Lyapunov function that the controller design algorithmsreturned found the smallest values for √ α, the bound on the induced L 2 gain from wto y.The results are presented in Table 3.4, which shows that the performance of thecontroller designed by the V variant algorithm is vastly worse in this metric than either ofthe A, B, C, D variant controllers. It is interesting to note that the achieved values for γ max

59seem to have little or no relation to the performance bound. However, as expected, the fullorder controller exhibits better performance than the reduced order controller.Algorithm n ξ√ α4 4 45.6063 4 0.0433 2 0.111Table 3.4: Performance of the controllers designed in the example.3.6 Chapter SummaryIn this chapter we investigated convex optimization algorithms to construct Lyapunovfunctions that demonstrate either global asymptotic or semi-global exponential stability.As a measure of a system’s performance, we provided a convex optimization procedureto bound the induced L 2 gain from disturbances to output signals.We then applied the semi-global exponential stability results to derive iterative algorithmsto design both state and output feedback controllers. These synthesis approacheswere tested on examples and the resulting controllers’ performance were bounded by computinga bound on the system’s induced L 2 gain from a disturbance in the control channelto the system’s output.

60Chapter 4Local Stability and ControllerSynthesisIf we again consider the system (3.1)ẋ = f(x)for x ∈ R n with f ∈ R n n as well as f(0) = 0, we can pose local system theoretic questionsas searches for SOS polynomials in addition to the global ones considered in the previouschapter. Again, we will first make Lyapunov stability arguments and adapt them into SOSprogramming problems. Additionally, we will continue to use the shorthand ˙V := ∇V ∗ f.4.1 Local Stability BackgroundUsing the stability definitions from §3.1.1 we can state the basic local stabilityLyapunov function result based on [16, Theorem 3.1].

61Theorem 10 Let D ⊂ R n be a domain containing the equilibrium point x = 0 of the system(3.1). Let V : D → R be a continuously differentiable function such thatV (0) = 0V (x) > 0 on D \ {0}and˙V := ∇V ∗ f < 0 on D \ {0}then the system (3.1) is asymptotically stable about x = 0. Moreover, any region Ω β :={x ∈ R n |V (x) ≤ β} such that Ω β ⊆ D describes an positively invariant region contained inthe equilibrium point’s domain of attraction.Proof:Given ɛ > 0, choose r ∈ (0, ɛ) such thatB r := {x ∈ R n |‖x‖ ≤ r} ⊂ DLet a = min ‖x‖=r V (x). Then, since V (x) > 0 for x ≠ 0, a > 0. Take b ∈ (0, a) and letΩ b := {x ∈ B r |V (x) ≤ b}. Since 0 < b < a, the containment Ω b ⊂ B r holds. Since Ω b ⊂ D,we know that ˙V (x) ≤ 0 on all of Ω b . Thus, for x 0 ∈ Ω bV (φ T (x 0 )) ≤ V (x 0 ) ≤ bfor all T ≥ 0. Since V is continuous and V (0) = 0 there must exist δ ɛ such thatB δɛ := {x ∈ R n |‖x‖ ≤ δ ɛ } ⊂ Ω b ⊂ B rEnd to end, we now have,x 0 ∈ B δɛ ⇒ x 0 ∈ Ω b ⇒ φ T (x 0 ) ∈ Ω b ⇒ φ T (x 0 ) ∈ B r

62thus ‖x 0 ‖ < δ ɛ implies ‖φ T (x 0 )‖ < ɛ for all T ≥ 0, proving stability.To prove asymptotic stability, we need to show that every initial condition x 0 ∈ B rconverges to the origin, which is that for every c > 0 there exists a T c > 0 such that‖φ t (x 0 )‖ < c for all t > T c . From earlier we know that for every c > 0 there exists d > 0such that Ω d ⊂ B c , which makes it sufficient to show that V (φ t (x 0 )) → 0 as t → ∞. SinceV (φ t (x 0 )) is monotonically decreasing and bounded below by zero,V (φ t (x 0 )) → v ≥ 0as t → ∞. To show that v = 0, suppose that v > 0. By continuity of V there exists w > 0such that B w ⊂ Ω v . The limit V (φ t (x 0 )) → v > 0 implies that the state trajectory mustremain outside of B w for all time. Let −α = max w≤‖x‖≤r ˙V (x), which exists because ˙V iscontinuous over this compact set. Clearly −α < 0. Integrating ˙V from any x 0 ∈ B r wehave∫ tV (φ t (x 0 )) = V (x 0 ) +0˙V (φ τ (x 0 )) dτ ≤ V (x 0 ) − αtThe right hand side will eventually become negative, contradicting the assumption thatv > 0.Consider now the set Ω β .Since Ω β ⊂ D, if x ∈ Ω β \ {0} then V (x) > 0 and˙V (x) < 0.This shows, following the arguments above, that V (φ t (x)) → 0 and thus‖φ t (x)‖ → 0 as t → ∞ for any x ∈ Ω β \ {0}. It follows that Ω β is contained in theequilibrium point’s domain of attraction.✷

634.2 Convex Stability TestsUsing the Lyapunov conditions for local asymptotic stability in Theorem 10 wecan construct two SOS programming based algorithms to prove a fixed point’s stability.Additionally, these algorithms can be used to find and maximize the size of certain invariantsubsets of its region of attraction. Unlike Zubov’s theorem, [11], which finds the exact regionof attraction, our approach allows us to use convex optimization to begin to understand thesize and shape of the region of attraction by finding invariant subsets.The first algorithm presented below works to find the largest estimate of the fixedpoint’s region of attraction by expanding the set D and then finding the largest level set ofthe resulting Lyapunov function that is contained in D, which in line with Theorem 10 isinvariant and is contained in the region of attraction. The second algorithm approaches theproblem from somewhat of a converse point of view. It expands a region that is containedin a level set of the Lyapunov function on which all of the Lyapunov conditions hold.4.2.1 Expanding D AlgorithmTo fit the assumptions of Theorem 10 into an SOS programming framework wewill first restrict V ∈ R n with V (0) = 0, as in the global case. Additionally we will describeD with a semi-algebraic setD := {x ∈ R n |p(x) ≤ β}

64with p ∈ Σ n , positive definite, and β ≥ 0 to insure that D is connected and contains theorigin. Now the requirements of Theorem 10 for asymptotic stability become{x ∈ R n |p(x) ≤ β} \ {0} ⊆ {x ∈ R n | ˙V (x) < 0}{x ∈ R n |p(x) ≤ β} \ {0} ⊆ {x ∈ R n |V (x) > 0}If we can find a V to satisfy these conditions for a fixed p and any value of β > 0, then thesystem (3.1) is asymptotically stable about the fixed point x = 0. However, using this setup, the largest invariant set we can demonstrate that converges to the origin is the largestlevel set of V that is contained in D. In order to find this largest estimate of the region ofattraction, we will satisfy the Lyapunov conditions above for the largest D by fixing p andmaximizing β subject to the Lyapunov conditions. In this approach, the level sets of p givethe shape of the regions over which we will be checking the Lyapunov conditions.We pose the following optimization to search for V using the set emptiness formof the set containment constraints abovemaxV ∈R n,V (0)=0βs.t.{x ∈ R n |p(x) ≤ β, x ≠ 0, V (x) ≤ 0} = φ{x ∈ R n |p(x) ≤ β, x ≠ 0, ˙V (x) ≥ 0} = φThese conditions are not yet semi-algebraic as they contain the non-polynomial constraintx ≠ 0. We can get around this problem by using l 1 , l 2 ∈ Σ n , positive definite, and replacing

65x ≠ 0 with l(x) ≠ 0 to getmaxV ∈R n,V (0)=0βs.t.{x ∈ R n |p(x) ≤ β, l 1 (x) ≠ 0, V (x) ≤ 0} = φ{x ∈ R n |p(x) ≤ β, l 2 (x) ≠ 0, ˙V (x) ≥ 0} = φInvoking the Positivstellensatz (Theorem 4), we can rewrite the constraints to form theequivalent optimizationmaxV ∈R n,V (0)=0βs.t.s 1 + (β − p)s 2 − V s 3 − V (β − p)s 4 + l 2k 11 = 0s 5 + (β − p)s 6 + ˙V s 7 + ˙V (β − p)s 8 + l 2k 22 = 0where s 1 , . . . , s 8 ∈ Σ n , k 1 , k 2 ∈ Z + and f, p are given. These constraints can not be checkedwith SOS programming in this general form. However, if we specify convenient values forthe k’s, the s’s and fix p we can use an iterative SOS programming approach to find theLyapunov function that maximizes the size of D.To limit the degree of the problem, we pick k 1 = k 2 = 1.Additionally, sincethe product of two SOS polynomials is SOS, we can further simplify the problem by byreplacing s 1 , . . . , s 4 with s 1 l 1 , . . . , s 4 l 1 as well as s 5 , . . . , s 8 with s 5 l 2 , . . . , s 8 l 2 . We can then

66factor out the l terms to get the following optimization with SOS constraintsmaxV ∈R n,V (0)=0,s i ∈Σ nβs.t.(4.1)−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ nIf this optimization is feasible, then V shows the system to be asymptotically stable withthe Lyapunov requirements holding on all of D. However, these constraints are bilinearin V and the s’s, so we will have to solve the problem iteratively. An algorithm to solvethe optimization (4.1) and find the largest estimate of the fixed point’s region of attractionwithin D is given in Algorithm 5 which follows the discussion of finding the largest invariantset contained in DHaving found the largest D = {x ∈ R n |p(x) ≤ β} over which the Lyapunovconditions hold we can now formulate a search for the largest level set of V which it containsasmaxcs.t.{x ∈ R n |V (x) ≤ c} ⊆ {x ∈ R n |p(x) ≤ β}where as opposed to (4.1) both β and V are fixed. We begin the transformation to an SOSprogramming problem by forming empty set constraint versionmaxcs.t.{x ∈ R n |V (x) ≤ c, p(x) > β} = φ

67which is equivalent tomaxcs.t.{x ∈ R n |V (x) ≤ c, p(x) ≥ β, p(x) ≠ β} = φNow we use the Positivstellensatz to transform the constraint, and pick k = 1 for the monoidto get the SOS programming problemmaxŝ 2 ,ŝ 3 ,ŝ 4 ∈Σ ns.t.(4.2)−ŝ 2 (c − V ) − ŝ 3 (p − β) − ŝ 4 (p − β)(c − V ) − (p − β) 2 ∈ Σ nwhich can be solved with a bisection on c.We can now combine the optimization to maximize the size of D (4.1) with the levelset maximization (4.2) to form the following algorithm for estimating the asymptoticallystable fixed point’s domain of attraction.cAlgorithm 5 (Expanding D) An iterative search to expand the region D in Theorem 10starting from a candidate Lyapunov function.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and pick the maximum degree of the Lyapunov function, the SOS multipliers and thel polynomials to be d V , d s2 , d s3 , d s4 , d s6 , d s7 , d s8 and d l1 , d l2 respectively. Pick the maximumdegrees for the the level set maximization problem (4.2) and denote them dŝ2 , dŝ3 and dŝ4 .Fix l k (x) = ɛ ∑ nj=1 xd l kjfor k = 1, 2 and some ɛ > 0. Additionally set β (i=0) = 0.1. Set V = V (i−1) and solve the linesearch on β where s 2 ∈ Σ n,ds2 , s 3 ∈ Σ n,ds3 , s 4 ∈

68Σ n,ds4 , s 6 ∈ Σ n,ds6 , s 7 ∈ Σ n,ds7 and s 8 ∈ Σ n,ds8maxs 2 ,s 3 ,s 4 ,s 6 ,s 7 ,s 8β(4.3)−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ nSet s (i)3 = s 3 , s (i)4 = s 4 , s (i)7 = s 7 and s (i)8 = s 8 . Continue to step 2.2. Set s 3 = s (i)3 , s 4 = s (i)4 , s 7 = s (i)7 and s 8 = s (i)8 . Solve the linesearch on β wheres.t.V ∈ R n,dV with V (0) = 0, s 2 ∈ Σ n,ds2 and s 6 ∈ Σ n,ds6maxV,s 2 ,s 6β(4.4)−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ nSet β (i) = β and V (i) = V . If β (i) − β (i−1) is less than a specified tolerance go to step3, else increment i and go to step 1.s.t.3. Set V = V (i) and β = β (i) . Solve the linesearch on c where ŝ 2 ∈ Σ n,dŝ2 , ŝ 3 ∈ Σ n,dŝ3and ŝ 4 ∈ Σ n,dŝ4maxŝ 2 ,ŝ 3 ,ŝ 4cs.t.−ŝ 2 (c − V ) − ŝ 3 (p − β) − ŝ 4 (p − β)(c − V ) − (p − β) 2 ∈ Σ nSet c max = c. The set {x ∈ R n |V (x) ≤ c max } is the resulting estimate of the regionof attraction, for it is positively invariant, contained within D and all of its pointsconverge to the fixed point x = 0.

69Remark 4 (Properties of the expanding D algorithm) :• If the algorithm is started from a feasible point, which is a candidate Lyapunov functionsuch that there exist s i ’s that make (4.1) feasible, then the algorithm will always remainfeasible. If the system’s linearization is stable, then the linearization’s quadraticLyapunov function will always work, since it will stabilize the nonlinear system nearto the origin.• Note that V need not be positive definite, since it is required to be positive only onD \ {0}, which is the first constraint in (4.1). However, to be positive over a regionabout the origin V must have no linear terms and d Vshould be even. Clearly if Vwere chosen to be positive definite, then this constraint can just become V − l 1 ∈ Σ n .• Since the s’s, l’s and the ŝ’s are SOS, they must be of even degree. Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintFor the second SOS constraintmax{deg(ps 2 ), deg(V s 3 )} ≥ d l1max{deg(ps 2 ), deg(V s 3 )} ≥ deg(V ps 4 )For the c maximization constraintdeg(ps 6 ) ≥ d l2deg(ps 6 ) ≥ max{deg( ˙V s 7 ), deg( ˙V ps 8 )}max{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(p 2 )max{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(ŝ 3 p)

704.2.2 Expanding Interior AlgorithmThe previous algorithm looks to Theorem 10 and searches for estimates of theregion of attraction by enlarging D. However, since D is described by the level sets of pand the estimate of the fixed point’s region of attraction is the largest level set of V that iscontained in D, it is possible to have a large D that contains a much smaller largest levelset of V . This algorithm takes the other approach by requiring that the set whose sizeis maximized is contained within an invariant region. This algorithm originally appearedin the context of state feedback controller design in [13] which also appears in a modifiedversion in §4.4.2.Consider again the system (3.1). If we define a variable sized regionP β := {x ∈ R n |p(x) ≤ β}for p ∈ Σ n and positive definite, we can estimate the region of attraction by maximizing βsubject to the constraint that all of the points in P β converge to the origin under the flowof the system. Following Theorem 10, if we defineD := {x ∈ R n |V (x) ≤ 1}for some as of yet unknown candidate Lyapunov function, then P β must be contained D.Additionally for the theorem’s other assumptions{x ∈ R n |V (x) ≤ 1} \ {0} ⊆ {x ∈ R n | ˙V (x) < 0}and, since V and thus D is unknown, the only effective way to ensure that V is positive onD \ {0} it to require that V be positive everywhere away from x = 0.

71The problem of finding the best estimate of the region of attraction in this frameworkcan be written with set emptiness constraints asmaxV ∈R n,V (0)=0βs.t.{x ∈ R n |V (x) ≤ 0, x ≠ 0} = φ(4.5){x ∈ R n |p(x) ≤ β, V (x) ≥ 1, V (x) ≠ 1} = φ{x ∈ R n |V (x) ≤ 1, ˙V (x) ≥ 0, x ≠ 0} = φwhich, as with the expanding D version, is not semi-algebraic due to the x ≠ 0 constraints.If we use the same work around as before by replacing these constraints with l 1 (x) ≠ 0 andl 2 (x) ≠ 0 with l 1 , l 2 ∈ Σ n , positive definite, we havemaxV ∈R n,V (0)=0βs.t.{x ∈ R n |V (x) ≤ 0, l 1 (x) ≠ 0} = φ(4.6){x ∈ R n |p(x) ≤ β, V (x) ≥ 1, V (x) ≠ 1} = φ{x ∈ R n |V (x) ≤ 1, ˙V (x) ≥ 0, l 1 (x) ≠ 0} = φApplying the Positivstellensatz (Theorem 4) the optimization becomesmaxβV ∈ R n, V (0) = 0, k 1 , k 2 , k 3 ∈ Z +s 1 , . . . , s 10 ∈ Σ ns.t.s 1 − V s 2 + l 2k 11 = 0(4.7)s 3 + (β − p)s 4 + (V − 1)s 5 + (β − p)(V − 1)s 6 + (V − 1) 2k 2= 0s 7 + (1 − V )s 8 + ˙V s 9 + (1 − V ) ˙V s 10 + l 2k 32 = 0

72which is not amenable to SOS programming unless we pick convenient values for some ofthe s’s and the k’s. To keep the degree of the problem down, we pick k 1 = k 2 = k 3 = 1.To simplify the first constraint, we pick s 2 = l 1 and factor l 1 out of s 1 . The secondconstraint has the term (V −1) 2 which is quadratic in the coefficients of V and thus inhibitsoptimization over V , so set s 3 = s 4 = 0 and factor out a (V − 1) term. The third constrainthas the term (1 − V ) ˙V s 10 which is quadratic in the coefficients of V so we set s 10 = 0 andfactor out l 2 . These selections allow us to write the optimization in a form that is suitablefor an iterative algorithm.maxV ∈ R n, V (0) = 0βs 6 , s 8 , s 9 ∈ Σ ns.t.∈ Σ n(− (1 − V )s 8 + ˙V)s 9 + l 2(4.8)V − l 1 ∈ Σ n− ( (β − p)s 6 + (V − 1) ) ∈ Σ nWe now can propose an algorithm to find an estimate of the region of attractionfor the fixed point, x = 0 of the system (3.1) by iteratively solving the SOS constrainedoptimization (4.8).Algorithm 6 (Expanding Interior) An iterative search to expand the region P β and thusthe region D := {x ∈ R n |V (x) ≤ 1} in Theorem 10 starting from a positive definite candidateLyapunov function.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and pick the maximum degree of the Lyapunov function, the SOS multipliers and

73the l polynomials to be d V , d s6 , d s8 , d s9 and d l1 , d l2 respectively. Fix l k (x) = ɛ ∑ nj=1 xd l kjk = 1, 2 and some ɛ > 0. Additionally set β (i=0) = 0.for1. Set V = V (i−1) and solve the linesearch on β where s 6 ∈ Σ n,ds6 , s 8 ∈ Σ n,ds8 ands 9 ∈ Σ n,ds9maxs 6 ,s 8 ,s 9βSet s (i)8 = s 8 and s (i)9 = s 9 . Continue to step 2.s.t.− ( (β − p)s 6 + (V − 1) ) (4.9)∈ Σ n(− (1 − V )s 8 + ˙V)s 9 + l 2 ∈ Σ n2. Set s 8 = s (i)8 and s 9 = s (i)9 . Solve the linesearch on β where V ∈ R n,d Vwith V (0) = 0and s 6 ∈ Σ n,ds6maxV,s 6βs.t.V − l 1 ∈ Σ n(4.10)− ( (β − p)s 6 + (V − 1) ) ∈ Σ n(− (1 − V )s 8 + ˙V)s 9 + l 2 ∈ Σ nSet β (i) = β and V (i) = V . If β (i) − β (i−1) is less than a specified tolerance goto step3, else increment i and go to step 1.3. The set {x ∈ R n |V (i) (x) ≤ 1} contains P β (i) and is the largest estimate of the fixedpoint’s region of attraction.Remark 5 (Properties of the expanding interior algorithm) :

74• If the algorithm is started from a feasible point, which is a Lyapunov function suchthat there exist s i ’s that make (4.8) feasible, then the algorithm will always remainfeasible. If the system’s linearization is stable, then the linearization’s quadratic Lyapunovfunction will always work, since it will stabilize the nonlinear system near tothe origin.• Note that, unlike Algorithm 5, V must be positive definite, so it should have no linearterms and d Vshould be even.• Since the s’s, and the l’s are SOS, they must be of even degree.Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintd V = d l1For the second SOS constraintdeg(ps 6 ) ≥ d VFor the third SOS constraintdeg(V s 8 ) ≥ deg( ˙V s 9 )deg(V s 8 ) = d l24.2.3 Estimating the Region of Attraction ExampleTo compare the effectiveness of Algorithm 5 (Expanding D) and Algorithm 6(Expanding Interior) consider estimating the region of attraction for the following system,a damped pendulum where the sine term has been replaced with the first two terms of its

75Taylor series expansionẋ 1 = x 2ẋ 2 = − 810 x 2 −(x 1 − x3 16) (4.11)The system has three equilibrium points: (0, 0) and (± √ 6, 0). Looking at the linearizationsof the system about each equilibrium point we find, as expected, that the first is a stablesink while the other are saddles.To use either Algorithm 5 or 6 we need to pick p and find a suitable startingcandidate Lyapunov function. In order to demonstrate the way the two algorithms work,we will pick a completely uninspired polynomial to expand its level sets:p(x) = x 2 1 + x 2 2Additionally we will pick the uninspired candidate Lyapunov function V (x) = x ∗ P x whereP comes from the LMI feasibility problemP ≻ 0A ∗ f P + P A f ≺ 0with A f the linearization of the system (4.11). Solving this feasibility problem, we find⎡⎤P = ⎢⎥⎣⎦25.33 81.98which we will use to form the candidate Lyapunov function.Setting the both algorithm’s stopping tolerance to β (i) − β (i−1) = .01 and settingthe degrees as follows we can compare the two algorithms. For the expanding D algorithm,set d V = 2, d s2 = d s3 = d s6 = 2, d s4 = d s7 = d s8 = 0 and d l1 = d l2 = 4. For the expandinginterior algorithm set d V = 2, d s8 = 2, d s6 = d s9 = 0, d l1 = 2 and d l2 = 4.

76We can see the progress of the two algorithms in Table 4.1 which shows the radiusof the region being expanded ( √ β in both cases). Note that, D contains the estimate ofthe region of attraction while the estimate contains P β .Iteration Index D Radius P β Radius1 2.42 0.852 2.42 1.163 - 1.594 - 1.845 - 2.056 - 2.05Table 4.1: Results of applying Algorithms 5 and 6 to (4.11).The results of running both algorithms are shown in Figure 4.1. The large dotsindicate the system’s fixed points and the thin lines connecting them are the system’s stableand unstable manifolds. The dashed lines show the maximal D from the expanding D algorithmand the maximal P β from the expanding interior algorithm. The thick-lined ellipsesshow the two estimates of the region of attraction; the smaller ellipse that is contained inD comes from Algorithm 5, while the larger ellipse that contains P β comes from Algorithm6.Even though the Lyapunov function is quadratic and the SOS multipliers are oflow degree, we can look at Figure 4.1 and tell that, for both algorithms, the largest value ofβ possible has been found for the given p. In the expanding D case, the region D borders thesystem’s saddle points where f, and thus ˙V , is zero. Where as, in the expanding interiorcase, P β comes right up to two of the system’s stable manifolds, by which the region ofattraction must be bounded.

7743P β2D1x 20−1−2−3−4−4 −3 −2 −1 0 1 2 3 4x 1Figure 4.1: Phase plot of the system (4.11) with the largest D and P β as well as theirellipsoidal estimates of the region of attraction that they respectively circumscribe andinscribe.Note that in Figure 4.1 the region D crosses the two of the system’s stable manifolds.Clearly the region of attraction can not cross any of the system’s stable or unstablemanifolds, so all of the area of D outside the manifolds can not be part of the estimate ofthe region of attraction. This skews the Lyapunov function that Algorithm 5 designs, sinceit is expanding D over areas where the level sets can not follow. Additionally, it seemsunrealistic to assume that, for a general polynomial system, the region where ˙V < 0 isdescribed by the a level set of a low degree polynomial.Clearly both of these algorithms would perform better if p were chosen to have levelsets that more closely align with the shape of the region of attraction, but the uninspiredchoice of p here indicates what will happen in higher dimensions where we are unable tovisualize the region of attraction.

784.3 Disturbance AnalysisThe previous local SOS applications were centered on proving stability; we will nowshift gears to consider the local effect of external disturbances on a polynomial system. Awealth of interesting results of this type for linear systems are posed in an LMI framework in[5]. Using SOS programming we can, in general, formulate parallel versions for polynomialsystems.In most cases the generalization from linear systems and LMIs to polynomialsystems and SOS programs is straight forward, as is shown in the example below in §4.3.1.However, it is very important to point out that the LMI tests given in [5] provide conditionsfor the system to have a given characteristic globally, which as we have seen in the stabilityand controller synthesis cases may be impossible or just undesired. So, in general, it willmake sense to add a few set containment constraints to make the result hold only on somesubset of the whole space.4.3.1 Reachable Set Bounds under Unit Energy DisturbancesThe original problem in [5, §6.1.1] is, given the linear systemẋ = Ax + B w wwith x ∈ R n and w ∈ R nw , can we bound the set of points that are reachable from x 0 = 0in T time units if the disturbance is constrained to have unit energy, ∫ T0 w(t)∗ w(t) dt ≤ 1.The bounding approach used is Lyapunov function based. Let the Lyapunov function bequadraticV (x) = x ∗ P x

79with P an as of yet unknown positive definite matrix. Then, if˙V (x) = ∇V ∗ (Ax + B w w) ≤ w ∗ wfor all x, w, we can integrate both sides from 0 to T to findV (x(T )) − V (x 0 ) ≤∫ T0w(t) ∗ w(t) dt ≤ 1which, since x 0 = 0 implies that the set {x ∈ R n |x ∗ P x ≤ 1} contains x(T ). The positivedefiniteness of V and the inequality bound on ˙V can be, written in an LMI setup as P ≻ 0and⎡⎢A∗P + P A⎣BwP∗⎤P B w⎥⎦ ≼ 0−IThese conditions provide a set that contains the reachable set from x 0 = 0 with unit energydisturbances, and, with a cost function dependent on the eigenvalues of P , we could optimizethe bound for tightness.Approach for Polynomial SystemsWe can now follow the reasoning above, as originally done in [13], to mimic thelinear system’s reachable set bound for the following polynomial system,ẋ = f(x) + g w (x)w (4.12)with x(t) ∈ R n , w(t) ∈ R nw , f ∈ R n n, f(0) = 0 and g w ∈ R n×nwn . The two central constraintsof the Lyapunov based approach are that V ∈ R n must be positive definite, and that∇V (x) ∗ (f(x) + g w (x)w) ≤ w ∗ w

80for all x, w.If we can find a V that satisfies these constraints, we know that the setΩ 1 := {x ∈ R n |V (x) ≤ 1} bounds the unit energy reachable set. At this point, thetransition of the problem from linear to polynomial system is complete and we would justneed to write the constraints above as SOS conditions.However, in this set up the ˙V constraint is required to hold for all x, w, whichseems extreme since φ t (x 0 ) remains in Ω 1 for all t ∈ [0, T ]. Additionally, we have no easyway to minimize the size of the bounding set.To get around the problems presented above, we will introduce a fixed, user chosen,function p ∈ Σ n that is positive definite and make the following definitionP β := {x ∈ R n |p(x) ≤ β}of a set that will contain Ω 1 . Additionally, since the state trajectory always remains in Ω 1 ,we will only require that the ˙V inequality hold for (x, w) ∈ Ω 1 × R nw . Now we can pose thefollowing optimization to minimize the size of set that bounds the unit energy reachableset, Ω 1 , where the constraints mentioned above have been turned into their set emptinessformsminVβ⎧⎪⎨⎪⎩s.t.{x ∈ R n |V (x) ≤ 0, l(x) ≠ 0} = φ{x ∈ R n |V (x) ≤ 1, p(x) ≥ β, p(x) ≠ β} = φ⎫∣ ∣∣∣∣∣∣∣∣∣∣∣V (x) ≤ 1,x ∈ R nx ∈ R nw∇V (x) ∗ (f(x) + g w (x)w) ≥ w ∗ w,∇V (x) ∗ (f(x) + g w (x)w) ≠ w ∗ w⎪⎬⎪⎭= φ

81with l ∈ Σ n , positive definite. Using the Positivstellensatz, this becomesminVβs.t.s 1 − V s 2 + l 2k 1= 0s 3 + (1 − V )s 4 + (p − β)s 5 + (1 − V )(p − β)s 6 + (p − β) 2k 2= 0⎛()⎞s 7 + ∇V (x) ∗ (f(x) + g w (x)w) − w ∗ w s 8 + (1 − V )s 9()+(1 − V ) ∇V (x) ∗ (f(x) + g w (x)w) − w ∗ w s 10= 0⎜⎟⎝()+ ∇V (x) ∗ (f(x) + g w (x)w) − w ∗ 2k3 ⎠wwith s 1 , . . . , s 6 ∈ Σ n , s 7 , . . . , s 10 ∈ Σ n+nw and k 1 , k 2 , k 3 ∈ Z + . To begin the transitionfrom set emptiness constraints into SOS constraints, pick k 1 = k 2 = k 3 = 1. For the firstconstraint, set s 2 = l and factor out an l. For the second constraint, we can just rearrange it.()And for the third constraint, we pick s 7 = s 9 = 0 and factor out a ∇V ∗ (f + g w w) − w ∗ w .In all this gives us the optimizationminVβV − l ∈ Σ n(4.13)− (1 − V )s 10 + ∇V ∗ (f + g w w) − w ∗ w ∈ Σ n+nw− ( (1 − V )s 4 + (p − β)s 5 + (1 − V )(p − β)s 6 + (p − β) 2) ∈ Σ n(())which we can solve iteratively by alternating linesearches on the s i ’s and V . For furtherdetails and a numeric example see [13, §III].s.t.

824.3.2 Set Invariance under Peak Bounded DisturbancesConsidering again a polynomial system subject to disturbances as in (4.12),ẋ = f(x) + g w (x)wwe can now look at finding the maximum peak disturbance value such that a given set remainsinvariant under these bounded disturbances and the action of the system’s dynamics.Let the peak of w be bounded by‖w(t)‖ ∞ ≤ √ γand define the invariant set asΩ 1 := {x ∈ R n |V (x) ≤ 1}for some fixed V ∈ R n , positive definite. We know that if ˙V (x, w) ≤ 0 on the boundary ofΩ 1 for all w meeting the peak bound, then the flow of the system from any point in Ω 1 cannot ever leave Ω 1 , which makes it invariant. In set containment terms we can write thisrelationship as{x ∈ R n , w ∈ R nw |V (x) = 1} ∩ {x ∈ R n , w ∈ R nw |w ∗ w ≤ γ}⊆ {x ∈ R n , w ∈ R nw | ˙V (x, w) ≤ 0} (4.14)which can be rewritten in set emptiness form as{x ∈ R n , w ∈ R nw |V (x) − 1 = 0, γ − w ∗ w ≥ 0, ˙V (x, w) ≥ 0, ˙V (x, w) ≠ 0} = φEmploying the Positivstellensatz, this becomess 0 + s 1 (γ − w ∗ w) + s 2 ( ˙V ) + s 3 (γ − w ∗ w)( ˙V ) + ( ˙V ) 2k + q(V − 1) = 0

83with k ∈ Z + , q ∈ R n+nw and s 0 , s 1 , s 2 , s 3 ∈ Σ n+nw .Using our standard approach of k = 1, we can write the following SOS constraintthat guarantees invariance under bounded w,−s 1 (γ − w ∗ w) − s 2 ( ˙V ) − s 3 (γ − w ∗ w)( ˙V ) − ˙V 2 − q(V − 1) ∈ Σ n+nw (4.15)Notice that this SOS condition has terms that are not linear in the monomials of V , andthus there is no way to use our convex optimization approach to adjust V while checkingthis condition. Since (4.15) in linear in γ we can search for the maximum peak disturbancefor which the set is invariant, by searching over q and the s i ’s to maximize γ subject to(4.15). We will need to have the following degree relationship hold to make (4.15) possiblyfeasible( ) ( )max deg(s 1 ) + 2, deg(s 2 ˙V ), deg(qV ) ≥ max deg(s 3 ˙V ) + 2, 2 deg( ˙V )If we set x 0 = 0, then the invariant set Ω 1 bounds the system’s reachable setunder disturbances with peak less than γ. This bound is similar, but less stringent, thanthe bound for linear systems given in [5].In §4.4.3 we design two state feedback controllers to make the largest region possibleattracted to the origin, and then we provide an example of the technique above byfinding the largest peak disturbance that these controllers can reject.Effect of ‖w(t)‖ ∞ on ‖x(t)‖ ∞Using the bounded peak disturbances techniques above to find the largest disturbancepeak value for which Ω 1 is invariant, we can then bound the peak size of the system’s

84state to get a relationship that is similar to the induced L ∞ → L ∞ norm from disturbanceto state for this invariant set.For a given V , we solve the optimization to find the largest γ such that (4.15) isfeasible. Then we can bound the size of the state by optimizing to find the smallest α suchthatΩ 1 = {x ∈ R n |V (x) ≤ 1} ⊆ {x ∈ R n |x ∗ x ≤ α}This containment constraint is easily solved with a generalized S-procedure following from§2.2.1. From this point we know that the following implication holds‖w(t)‖ ∞ ≤ √ γ ⇒ ‖x(t)‖ ∞ ≤ √ αas long as x 0 ∈ Ω 1 , which provides our induced norm-like bound.As with the previous disturbance analysis technique, we demonstrate this one in§4.4.3 on the designed state feedback controllers.4.3.3 Induced L 2 → L 2 GainConsider the disturbance driven system with outputs (3.3),ẋ = f(x) + g w (x)wy = h(x)with x(t) ∈ R n , w(t) ∈ R nw , y(t) ∈ R p , f ∈ R n n, f(0) = 0, g w ∈ R n×nwn , and h ∈ R p n withh(0) = 0. In §3.3 we studied this system’s global induced L 2 → L 2 gain from w to y, andnow we look to bound this gain on some invariant region.For a region, Ω 1= {x ∈ R n |V (x) ≤ 1} as in §4.3.2, that is invariant underdisturbances with ‖w(t)‖ ∞ ≤ √ γ, we can bound the induced L 2 → L 2 gain from w to y on

85this invariant set by finding a positive definite H ∈ R n and β ≥ 0 such that the followingset containment holds{x ∈ R n , w ∈ R nw |w ∗ w ≤ γ} ∩ {x ∈ R n , w ∈ R nw |V (x) ≤ 1}⊆ {x ∈ R n , w ∈ R nw |Ḣ(x, w) + h(x)∗ h(x) − βw ∗ w ≤ 0} (4.16)If we can find a β, H pair to make (4.16) hold, then we can follow the steps from §3.3 toshow thatprovided that x 0 = 0 and ‖w(t)‖ ∞ ≤ √ γ.‖y(t)‖ 2‖w(t)‖ 2≤ √ βWe can search for the tightest bound on the induced norm by employing a generalizedS-procedure to satisfy (4.16) and solving the following optimizationminH∈R nβs.t.(4.17)H − l ∈ Σ n)−(Ḣ(x, w) + h(x) ∗ h(x) − βw ∗ w − s 1 (γ − w ∗ w) − s 2 (1 − V ) ∈ Σ n+nwwith s 1 , s 2 ∈ Σ n+nwand l ∈ Σ n , positive definite.and s 2 so thatIn an effort to make the optimization (4.17) feasible we will pick the degrees of s 1deg(s 1 ) + 2 ≥ deg(Ḣ + h∗ h)anddeg(s 2 ) + deg(V ) ≥ deg(Ḣ + h∗ h)Once more, we will exhibit this technique to study the performance of the statefeedback controllers that we design in §4.4.3.

864.4 State Feedback Controller DesignIn the previous section, we found two SOS programming based approaches to proveasymptotic stability for a polynomial system and estimate the domain of attraction of thefixed point at the origin. We now expand these approaches for state feedback controllerdesign.Consider again the system (3.5)ẋ = f(x) + g(x)ufor x ∈ R n with f ∈ R n n, f(0) = 0 and u ∈ R m with g ∈ R n×mn . If we allow u to begenerated by a state feedback controller K ∈ R m nwith K(0) = 0, we get the closed loopsystem (3.6)ẋ = f(x) + g(x)K(x)where K is still unknown. We can now look to find state feedback controller design algorithmsthat parallel Algorithms 5 and 6.4.4.1 Expanding D Algorithm for State Feedback DesignWe will follow the steps of the development of the expanding D algorithm in §4.2.1to develop a state feedback controller design algorithm. As before we restrict V ∈ R n withV (0) = 0 and describe D with a semi-algebraic setD := {x ∈ R n |p(x) ≤ β}

87with p ∈ Σ n , positive definite, and β ≥ 0.Now the requirements of Theorem 10 forasymptotic stability for the closed loop system (3.6) become{x ∈ R n |p(x) ≤ β} \ {0} ⊆ {x ∈ R n |V (x) > 0}{x ∈ R n |p(x) ≤ β} \ {0} ⊆ {x ∈ R n |∇V (x) ∗ (f(x) + g(x)K(x) < 0}If we can find V, K to satisfy these conditions for a fixed p and any value of β > 0, then thesystem (3.1) is asymptotically stable about the fixed point x = 0.As in Algorithm 5, the largest invariant set we can demonstrate that converges tothe origin is the largest level set of V that is contained in D. Following the manipulationsthat got us to SOS constraints in stability case we find that maximizing D under the setcontainments above is equivalent tomaxV ∈ R n, V (0) = 0βK ∈ R m n , K(0) = 0s i ∈ Σ ns.t.(4.18)−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ nAgain, since the SOS conditions are trilinear in V, K and the s’s they will haveto be checked iteratively. Also, as before, the largest estimate of the region of attractionis the largest invariant set contained in D and is found using the same optimization as in§4.2.1. The following algorithm uses a nested iterative approach to maximize the size ofD by adjusting V , K and the s’s and then finds largest level set of V contained in D toestimate the region of attraction.

88Algorithm 7 (Expanding D State Feedback Design) An iterative search to expandthe region D in Theorem 10 starting from a candidate Lyapunov function and a candidatecontroller.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and the candidate controller K (i=0) .Pick the maximum degree of the Lyapunovfunction, the controller, the SOS multipliers and the l polynomials to be d V , d K ,d s2 , d s3 , d s4 , d s6 , d s7 , d s8 and d l1 , d l2 respectively. Pick the maximum degrees for the the levelset maximization problem (4.2) and denote them dŝ2 , dŝ3and dŝ4 . Fix l k (x) = ɛ ∑ nj=1 xd l kjfor k = 1, 2 and some ɛ > 0. Additionally set β (i=0) = 0.1. SOS Weight Iteration:Set V = V (i−1) and K = K (i−1) and solve the linesearch on β where s 2 ∈ Σ n,ds2 ,s 3 ∈ Σ n,ds3 , s 4 ∈ Σ n,ds4 , s 6 ∈ Σ n,ds6 , s 7 ∈ Σ n,ds7and s 8 ∈ Σ n,ds8maxs 2 ,s 3 ,s 4 ,s 6 ,s 4 ,s 8βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n−(β − p)s 6 − ∇V ∗ (f + gK)s 7 − ∇V ∗ (f + gK)(β − p)s 8 − l 2 ∈ Σ nSet s (i)7 = s 7 , s (i)8 = s 8 and continue to step 2.2. Controller Iteration:Set V = V (i−1) , s 7 = s (i)7 and s 8 = s (i)8 . Solve the linesearch on β where s 2 ∈ Σ n,ds2 ,

89s 3 ∈ Σ n,ds3 , s 4 ∈ Σ n,ds4 , s 6 ∈ Σ n,ds6 and K ∈ R m n,d Kwith K(0) = 0maxK,s 2 ,s 3 ,s 4 ,s 6βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n−(β − p)s 6 − ∇V ∗ (f + gK)s 7 − ∇V ∗ (f + gK)(β − p)s 8 − l 2 ∈ Σ nSet K (i) = K, s (i)3 = s 3 and s (i)4 = s 4 . Continue to step 3.3. Lyapunov function iterationSet K = K (i) , s 3 = s (i)3 , s 4 = s (i)4 , s 7 = s (i)7 and s 8 = s (i)8 . Solve the linesearch on βwhere s 2 ∈ Σ n,ds2 , s 6 ∈ Σ n,ds6 and V ∈ R n,dV with V (0) = 0maxV,s 2 ,s 6βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n−(β − p)s 6 − ∇V ∗ (f + gK)s 7 − ∇V ∗ (f + gK)(β − p)s 8 − l 2 ∈ Σ nSet V (i) = V and continue to step 4.4. If β (i) − β (i−1) is less than a specified tolerance goto step 5, else increment i and gotostep 1.5. Set V = V (i) and β = β (i) . Solve the linesearch on c where ŝ 2 ∈ Σ n,dŝ2 , ŝ 3 ∈ Σ n,dŝ3and ŝ 4 ∈ Σ n,dŝ4maxŝ 2 ,ŝ 3 ,ŝ 4cs.t.−ŝ 2 (c − V ) − ŝ 3 (p − β) − ŝ 4 (p − β)(c − V ) − (p − β) 2 ∈ Σ n

90Set c max = c. The set {x ∈ R n |V (x) ≤ c max } is the resulting estimate of the regionof attraction, for it is positively invariant, contained within D and all of its pointsconverge to the fixed point x = 0.Remark 6 (Properties of the expanding D state feedback algorithm) :• As with the stability version of the album, if the algorithm is started from a feasiblepoint it will always remain feasible. If the system’s linearization is controllable, thena linear controller that stabilizes the linearized system and the resulting quadraticLyapunov function will always work, since they will stabilize the nonlinear systemnear to the origin. Since we can be guaranteed of a feasible starting point, we do notneed to consider K and V variants of the algorithm.• As with the stability version, V need not be positive definite, since it is required to bepositive only on D \{0}. However to be positive over a region about the origin V musthave no linear terms and d Vshould be even. Clearly if V were chosen to be positivedefinite, then the first constraint in (4.18) can just become V − l 1 ∈ Σ n .• Since the s’s, l’s and the ŝ’s are SOS, they must be of even degree. Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintFor the second SOS constraintmax{deg(ps 2 ), deg(V s 3 )} ≥ d l1max{deg(ps 2 ), deg(V s 3 )} ≥ deg(V ps 4 )deg(ps 6 ) ≥ d l2deg(ps 6 ) ≥ max {deg (∇V ∗ (f + gK)s 7 ) , deg (∇V ∗ (f + gK)ps 8 )}

91For the c maximization constraintmax{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(p 2 )max{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(ŝ 3 p)4.4.2 Expanding Interior Algorithm for State Feedback DesignAs with the expanding D algorithm, we can now adapt the expanding interioralgorithm to search for state feedback controllers to stabilize (3.5) as was proposed in [13].As with the stability version, we define a variable sized regionP β := {x ∈ R n |p(x) ≤ β}for p ∈ Σ n and positive definite. We then defineD := {x ∈ R n |V (x) ≤ 1}for some as of yet unknown candidate Lyapunov function. Following the developments forthe stability case we know that we need{x ∈ R n |p(x) ≤ β} ⊆ {x ∈ R n |V (x) ≤ 1}{x ∈ R n |V (x) ≤ 1} \ {0} ⊆({x ∈ R n |∇V ∗ (x))f(x) + g(x)K(x) < 0}{x ∈ R n |V (x) ≤ 0, x ≠ 0} = φ

92with V (0) = 0 to satisfy the assumptions of Theorem 10. Maximizing the size of P β subjectto these set emptiness and containment constraints can be reformulated asmaxV ∈ R n, V (0) = 0βK ∈ R m n , K(0) = 0s 6 , s 8 , s 9 ∈ Σ ns.t.(4.19)− ( (1 − V )s 8 + ∇V ∗ )(f + gK)s 9 + l 2 ∈ ΣnV − l 1 ∈ Σ n− ( (β − p)s 6 + (V − 1) ) ∈ Σ nWe now can propose an algorithm to design a controller to enlarge our estimate ofthe region of attraction for the fixed point, x = 0, of (3.5) by solving the SOS constrainedoptimization (4.19) with a nested iteration. For the controller synthesis part of the iterationa slight modification is made to the third constraint of (4.19) by introducing an intermediatevariable α and expanding the Lyapunov level set {x ∈ R n |V (x) < α}. After the controlleris designed, the Lyapunov function constraint is rescaled by α.Algorithm 8 (Expanding Interior State Feedback Design) An iterative search to expandthe region P β and thus the region D := {x ∈ R n |V (x) ≤ 1} in Theorem 10 startingfrom a positive definite candidate Lyapunov function and a candidate state feedback controller.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and the candidate state feedback controller K (i=0) . Pick the maximum degree of theLyapunov function, the controller, the SOS multipliers and the l polynomials to be d V , d K ,d s6 , d s8 , d s9 and d l1 , d l2 respectively. Fix l k (x) = ɛ ∑ nj=1 xd l kjfor k = 1, 2 and some ɛ > 0.

93Additionally set l (i=0)2 = l 2 , and β (i=0) = 0.1. SOS Weight iteration:Set V = V (i−1) , K = K (i−1) and l 2 = l (i−1)2 . Solve the linesearch on α where s 8 ∈Σ n,ds8 and s 9 ∈ Σ n,ds9maxs 8 ,s 9s.t.()−((α − V )s 8 + ∇V ∗ f + gK)s 9 + l 2α∈Σ nSet s (i)9 = s 9 and continue to step 2.2. Controller iteration:Set V = V (i−1) , l 2 = l (i−1)2 , and s 9 = s (i)9 . Solve the linesearch on α where s 8 ∈ Σ n,ds8and K ∈ R m n,d Kwith K(0) = 0maxK,s 8s.t.()−((α − V )s 8 + ∇V ∗ f + gK)s 9 + l 2α∈Σ nSet K (i) = K, s (i)8 = s 8 and l (i)2 = l (i−1)2 /α. Continue to step 2.3. Lyapunov function iteration:Set K = K (i) , s 8 = s (i)8 , s 9 = s (i)9 and l 2 = l (i)2 . Solve the linesearch on β where

94s 6 ∈ Σ n,ds6 and V ∈ R n,dV with V (0) = 0maxV,s 6βs.t.V − l 1 ∈ Σ n− ( (β − p)s 6 + (V − 1) ) ∈ Σ n(− (1 − V )s 8 + ˙V)s 9 + l 2 ∈Set V (i) = V and β (i) = β. Continue to step 4.Σ n4. If β (i) − β (i−1) is less than a specified tolerance, the set {x ∈ R n |V (i) (x) ≤ 1} containsP β (i) and is the largest estimate of the fixed point’s region of attraction, else incrementi and goto step 1.Remark 7 (Properties of the expanding interior algorithm) :• If the algorithm is started from a feasible point, then it will always remain feasible.If the system’s linearization is controllable, then any linear controller that stabilizesthe linearized system and the corresponding quadratic Lyapunov function will alwayswork, since they will stabilize the nonlinear system near to the origin. Again, thisfeasibility property lets us not consider both a V and a K variant.• Note that, unlike Algorithm 5, V must be positive definite, so it should have no linearterms and d Vshould be even.• Since the s’s, and the l’s are SOS, they must be of even degree.Additionally, thedegrees need to be chosen so that following relations hold:

95For the first SOS constraintd V = d l1For the second SOS constraintdeg(ps 6 ) ≥ d VFor the third SOS constraint)deg(V s 8 ) ≥ deg(∇V ∗ (f + gK)s 9deg(V s 8 ) = d l24.4.3 State Feedback Design ExampleConsider the spring-mass-damper system from §3.4.3, and recall that it was possibleto design a degree three polynomial state feedback controller to make the systemsemi-globally exponentially stable. However, no lower degree controller could be found thatmet the stability requirements. Also, if the linear spring were replaced with a nonlinearspring that behaved the same as the one that fixed m 1 to the wall, then no polynomialcontroller can be found to make the system semi-globally exponentially stable for d K ≤ 5and d V ≤ 6.We now look to find a local controller to demonstrate stability of the spring-massdampersystem with two nonlinear springs⎡ ⎤ ⎡⎤ ⎡ ⎤ẋ 1x 20ẋ 2−(x 1 + 110 =x3 1 ) + ( (x 3 − x 1 ) + 110 (x 3 − x 1 ) 3) − 110 (x 4 − x 2 )1+u (4.20)ẋ 3x 40⎢ ⎥ ⎢⎥ ⎢ ⎥⎣ ⎦ ⎣ẋ 4 − ( (x 3 − x 1 ) + 110 (x 3 − x 1 ) 3) ⎦ ⎣ ⎦+ 110 (x 4 − x 2 )0} {{ } }{{}f(x)g(x)

96As before, the linearization of the system is unstable and we will use the linear controllerand quadratic Lyapunov function from the “feedback trick” LMI (3.13) as the candidatecontroller and Lyapunov function. To show an additional use for Algorithms 7 & 8 we willalso tack on the constraint that the system model (4.20) is only valid when ‖x‖ ≤ 10.From the perspective of the expanding D algorithm, this model validity constraintcan be handled by making D := x 1 1 + x2 2 + x2 3 + x2 4and setting the upper bound on theβ linesearches to be 100. With this choice of D, we know that the largest level set of theresulting Lyapunov function will be within the region where the model is valid, so anyinitial condition within this level set will converge to the origin and always remain withinthe region where the model is valid.To handle the model validity constraint within the context of the expanding interioralgorithm, set p = x 1 1 + x2 2 + x2 3 + x2 4and the upper bound on β in all the linesearches to100. Unlike the expanding D case, this does not yet take care of the constraint. When thealgorithm finishes we can do a bisection on a generalized S-procedure to find the largestvalue r such that{x ∈ R n |V (x) ≤ r} ⊆ {x ∈ R n |x 2 1 + x 2 2 + x 2 3 + x 2 4 ≤ 100}Then all initial conditions in the set where V is less than or equal to r converge to the originand always remain in the region where the model is valid.Using these approaches we can solve for a state feedback controller for (4.20). Weuse Algorithm 7 with d V = 2, d K = 1, d s2 = d s3 = 2, d s4 = 0, d s6 = 4, d s7 = d s8 = 0,d l1 = d l2 = 4, with dŝ2 = dŝ3 = 2 and dŝ4 = 0. For Algorithm 8 we use d V = 2, d K = 1,d s6 = d s8 = 2, d s9 = 0, with d l1 = 2 and d l2 = 4. For both algorithms we set the tolerance

97to 0.01.After 4 iterations Algorithm 7 converges to β = 100 with c max = 15.10, whileAlgorithm 8 reaches the β tolerance limit after 24 iterations at β = 14.96 with r = 10.01.For clarity, let the controller and Lyapunov function found by the expanding Dalgorithm be K d and V d . Likewise, let the results from the expanding interior algorithm beK i and V i . To summarize the results, under the linear controller⎡ ⎤x 1[]x 2K d (x) = −62.58 −26.31 36.08 −27.06x 3⎢ ⎥⎣ ⎦x 4the system (4.20) is asymptotically stable and the region Ω d := {x ∈ R n |V d (x) ≤ c max }with⎡ ⎤x 1x 2V d (x) =x 3⎢ ⎥⎣ ⎦x 4∗ ⎡⎢⎣4.82 0.72 −1.83 2.470.72 0.27 −0.40 0.34−1.83 −0.40 2.54 −0.312.47 0.34 −0.31 3.02⎤ ⎡ ⎤x 1x 2x 3⎥ ⎢ ⎥⎦ ⎣ ⎦x 4is an invariant set contained in the fixed point’s region of attraction as well as the set wherethe model is valid, {x ∈ R n |‖x‖ ≤ 10}. Additionally under the controller⎡ ⎤x 1[]x 2K i (x) = −35.57 −7.70 22.40 −14.56x 3⎢ ⎥⎣ ⎦x 4

98the system (4.20) is asymptotically stable and the region Ω i := {x ∈ R n |V i (x) ≤ r} with⎡ ⎤∗ ⎡⎤ ⎡ ⎤x 14.33 0.35 −2.19 1.89x 1x 20.35 0.14 −0.25 0.19x 2V i (x) =x 3−2.19 −0.25 2.45 −0.63x 3⎢ ⎥ ⎢⎥ ⎢ ⎥⎣ ⎦ ⎣⎦ ⎣ ⎦x 4 1.89 0.19 −0.63 2.36 x 4is an invariant set contained in the fixed point’s region of attraction as well as the set wherethe model is valid, {x ∈ R n |‖x‖ ≤ 10}.Comparing the two controllers we find that the volume of Ω d is 779.74, while Ω ihas smaller volume of 535.75.Interestingly, Ω d does not contain Ω i , and r would haveto scaled back by 30% to make Ω i fit inside of Ω d . Since Ω d has a larger volume, K d isthe preferred controller, unless there is specific information about the initial condition thatshows that it is in Ω i \ Ω d .Disturbance Rejection Qualities of the ControllersUsing the disturbance analysis techniques of §4.3, we can now quantify the performanceof the controllers K d and K i designed above. The approaches of §4.3.2 and §4.3.3allow us to find the largest peak disturbance under which Ω d and Ω i are invariant, thepeak norm of the state that this implies, and the L 2 → L 2 gains from a disturbance to thesystem’s states on these invariant regions.For all these analyses we will assume that the disturbance enters additively in thecontrol channel, which makes g w (x) = g(x). Additionally, since we are considering a statefeedback system, we will set h(x) = x for the induced L 2 → L 2 gain.To compute, √ γ, the bound on ‖w(t)‖ under which Ω d and Ω i are invariant, the

99K dK i√ γ 18.57 4.42√ α 10.07 10.00√ β 0.06 0.27Table 4.2: State Feedback Controller Disturbance Rejection Results.condition (4.15) becomes, at minimum, a search for SOS polynomials of degree 8 in 5variables, since n = 4, n w = 1 and deg( ˙V ) = 4. Polynomials in Σ 5,8 have 1287 coefficients.To work around searching for polynomials with so many coefficients, we will setq = ˙V 2ˆq and s i = ˙V 2 ŝ i for i = 1, 2, 3. This allows us to factor out a ˙V 2 term to get thefollowing condition−ŝ 1 (γ − w ∗ w) − ŝ 2 ( ˙V ) − ŝ 3 (γ − w ∗ w)( ˙V ) − 1 − ˆq(V − 1) ∈ Σ n+nwwhich, if we pick deg(ŝ 1 ) = 4, deg(ŝ 2 ) = 2, deg(ŝ 3 ) = 0 and deg(ˆq) = 4, becomes a searchfor a polynomial of degree 6 in 5 variables. Polynomials in Σ 5,6 have 462 terms, which is avery good improvement in problem size.Using this approach to reduce the problem size of (4.15) and picking deg H = 2and deg s 1 = deg s 2 = 0 in (4.17), we get the results of the three disturbance rejectionanalyses that are shown in Table 4.2. Note that the bounds are as follows, the respectiveinvariant set is invariant under‖w(t)‖ ∞ ≤ √ γand‖w(t)‖ ∞ ≤ √ γ ⇒ ‖x(t)‖ ∞ ≤ √ α

100as well as, if ‖w(t)‖ ∞ ≤ √ γ then‖x(t)‖ 2‖w(t)‖ 2≤ √ βThe results shown in the table illustrate that under all three criterion considered, thecontroller designed by the expanding D algorithm out performs the controller designed bythe expanding interior algorithm. The superior disturbance rejection together with the factthat Ω d is much larger than Ω i point to K d being the preferred controller for this example.4.5 Output Feedback Controller DesignAs in the global case, we can now expand the state feedback results by allowingthe controller to be dynamic. As with the local asymptotic stability and state feedbackcontroller design problems, we will provide an expanding D approach as well as an expandinginterior approach to design a controller to demonstrate the largest estimate of the region ofattraction.Following the global case, define the same system to be controlled (3.14)ẋy= f(x) + g(x)u= h(x)for x ∈ R n with f ∈ R n n, f(0) = 0, u ∈ R m , g ∈ R n×mn with y ∈ R p , h ∈ R p n and h(0) = 0.u is generated by an unknown n ξ -state dynamic output feedback controller (3.15)˙ξu= A(ξ) + B(ξ)y= C(ξ) + D(ξ)yfor ξ ∈ R n ξwith A ∈ R n ξn ξ, A(0) = 0, B ∈ R n ξ×p, C ∈ R m n ξ, C(0) = 0 and D ∈ R m×p . Withn ξn ξ

101this controller structure, the closed loop system becomes (3.16)ẋ˙ξ= f(x) + g(x)(C(ξ) + D(ξ)h(x))= A(ξ) + B(ξ)h(x).By the assumptions on f, h, A and C, we know that the combined system has a fixed pointat [x; ξ] = [0; 0].From this point we can adapt Algorithms 7 and 8 to solve for a stabilizing outputfeedback controller. Again, as in the global output feedback case, when the systems arelinear and the Lyapunov function is quadratic we can not use the “feedback trick” to solvea single LMI for a feasible candidate controller and Lyapunov function pair.4.5.1 Expanding D Algorithm for Output Feedback DesignFollowing the development of the state feedback expanding D algorithm we candevelop a output feedback controller design algorithm. First we fix the number of states inthe output feedback controller, n ξ ∈ Z + , restrict the Lyapunov function V ∈ R n+nξwithV (0) = 0 and define D with a semi-algebraic setD := {[x; ξ] ∈ R n+n ξ| p([x; ξ]) ≤ β}with p ∈ Σ n+nξ , positive definite, and β ≥ 0.The requirements of Theorem 10 for asymptotic stability of the closed loop system

102(3.16) become{[x; ξ] ∈ R n+n ξ| p([x; ξ]) ≤ β} \ {0} ⊆ {[x; ξ] ∈ R n+n ξ| V ([x; ξ]) > 0}{[x; ξ] ∈ R n+n ξ| p([x; ξ]) ≤ β} \ {0} ⊆⎧⎡⎤⎪⎨⎫⎪ [x; ξ] ∈ R n+n ξ∇V ([x; ξ]) ∗ ⎢f(x) + g(x)(C(ξ) + D(ξ)h(x))⎬⎥⎣⎦⎪⎩< 0 ⎪∣A(ξ) + B(ξ)h(x)⎭The system is asymptotically stable about the fixed point [x; ξ] = 0, if any V, A, B, C, Dcan be found to satisfy these requirements for a fixed p and any β > 0.Following Algorithm 5, the largest estimate of the region of attraction that wecan demonstrate is the largest level set of V that is contained in D.Using the samemanipulations as in the state feedback case we get that maximizing the size of the regionD under the set containments above is equivalent tomaxβV ∈ R n+nξ , V (0) = 0, s i ∈ Σ n+nξA ∈ R n ξn ξ, A(0) = 0, B ∈ R n ξ ×pn ξC ∈ R m n ξ, C(0) = 0, D ∈ R m×pn ξs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n+nξ⎡⎤(4.21)−(β − p)s 6 − ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 7A + Bh⎡⎤−∇V ∗ ⎢f + gC + gDh⎥⎣⎦ (β − p)s 8 − l 2 ∈ Σ n+nξA + BhSince the SOS conditions are not linear in the decision variables, we will have to solve theoptimization iteratively.

103Algorithm 9 (Expanding D Output Feedback Design) An iterative search to expandthe region D in Theorem 10 starting from a candidate Lyapunov function and a candidateoutput feedback controller with n ξ states.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and the candidate controller A (i=0) , B (i=0) , C (i=0) , D (i=0) . Pick the maximum degreeof the Lyapunov function, the controller, the SOS multipliers and the l polynomials to bed V , d A , d B , d C , d D , d s2 , d s3 , d s4 , d s6 , d s7 , d s8 and d l1 , d l2 respectively. Pick the maximumdegrees for the the level set maximization problem (4.2) and denote them dŝ2 , dŝ3 and dŝ4 .( ∑nFix l k (x) = ɛj=1 xd l kj+ ∑ )n ξj=1 ξd l kjfor k = 1, 2 and some ɛ > 0. Additionally setβ (i=0) = 0.1. SOS Weight Iteration:Set V = V (i−1) , A = A (i−1) , B = B (i−1) , C = C (i−1) and D = D (i−1) . Solve thelinesearch on β where s 2 ∈ Σ n+nξ ,d s2, s 3 ∈ Σ n+nξ ,d s3, s 4 ∈ Σ n+nξ ,d s4, s 6 ∈ Σ n+nξ ,d s6,s 7 ∈ Σ n+nξ ,d s7and s 8 ∈ Σ n+nξ ,d s8maxs 2 ,s 3 ,s 4 ,s 6 ,s 4 ,s 8βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n+nξ⎡⎤−(β − p)s 6 − ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 7A + Bh⎡⎤−∇V ∗ ⎢f + gC + gDh⎥⎣⎦ (β − p)s 8 − l 2 ∈ Σ n+nξA + BhSet s (j)7 = s 7 and s (j)8 = s 8 . Continue to step 2.

1042. Controller Iteration:Set V = V (i−1) , s 7 = s (i)7 and s 8 = s (i)8 . Solve the linesearch on β where s 2 ∈ Σ n+nξ ,d s2,s 3 ∈ Σ n+nξ ,d s3, s 4 ∈ Σ n+nξ ,d s4, s 6 ∈ Σ n+nξ ,d s6, A ∈ R n ξn ξ ,d A, A(0) = 0, B ∈ R n ξ×pn ξ ,d B,C ∈ R m n ξ ,d Cwith C(0) = 0 and D ∈ R m×pn ξ ,d DmaxA, B, C, Dβs 2 , s 3 , s 4 , s 6s.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n+nξ⎡⎤−(β − p)s 6 − ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 7A + Bh⎡⎤−∇V ∗ ⎢f + gC + gDh⎥⎣⎦ (β − p)s 8 − l 2 ∈ Σ n+nξA + BhSet A (i) = A, B (i) = B, C (i) = C, D (i) = D, s (i)3 = s 3 and s (i)4 = s 4 . Continue to step3.3. Lyapunov function iterationSet A = A (i) , B = B (i) , C = C (i) , D = D (i) , s 3= s (i)3 , s 4 = s (i)4 , s 7 = s (i)7and s 8 = s (i)8 . Solve the linesearch on β where s 2 ∈ Σ n+nξ ,d s2, s 6 ∈ Σ n+nξ ,d s6and

105V ∈ R n+nξ ,d Vwith V (0) = 0maxV,s 2 ,s 6βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1 ∈ Σ n+nξ⎡⎤−(β − p)s 6 − ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 7A + Bh⎡⎤−∇V ∗ ⎢f + gC + gDh⎥⎣⎦ (β − p)s 8 − l 2 ∈ Σ n+nξA + BhSet β (i) = β and V (i) = V . Continue to step 4.4. If β (i) − β (i−1) is less than a specified tolerance goto step 5, else increment i and gotostep 1.5. Set V = V (i) and β = β (i) . Solve the linesearch on c where ŝ 2 ∈ Σ n+nξ ,dŝ2 , ŝ 3 ∈Σ n+nξ ,dŝ3 and ŝ 4 ∈ Σ n+nξ ,dŝ4maxŝ 2 ,ŝ 3 ,ŝ 4cs.t.−ŝ 2 (c − V ) − ŝ 3 (p − β) − ŝ 4 (p − β)(c − V ) − (p − β) 2 ∈ Σ nSet c max = c. The set {[x; ξ] ∈ R n+n ξ|V ([x; ξ]) ≤ c max } is the resulting estimate ofthe region of attraction, for it is positively invariant, contained within D and all ofits points converge to the fixed point x = 0.Remark 8 (Properties of the expanding D output feedback algorithm) :

106• As with the stability and state feedback versions, if the algorithm is started from afeasible point it will always remain feasible. If the system’s linearization is controllable,then a linear output feedback controller that stabilizes the linearized system and theresulting quadratic Lyapunov function will always work, since they will stabilize thenonlinear system near to the origin. Again, this allows us to not consider separatealgorithms starting from the candidate controller and from the candidate Lyapunovfunction.• As with the previous versions, V need not be positive definite, since it is required to bepositive only on D \{0}. However to be positive over a region about the origin V musthave no linear terms and d Vshould be even. Clearly if V were chosen to be positivedefinite, then the first constraint in (4.21) can just become V − l 1 ∈ Σ n+nξ .• Since the s’s, l’s and the ŝ’s are SOS, they must be of even degree. Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintFor the second SOS constraintmax{deg(ps 2 ), deg(V s 3 )} ≥ d l1max{deg(ps 2 ), deg(V s 3 )} ≥ deg(V ps 4 )deg(ps 6 ) ≥ d l2⎛ ⎡⎤ ⎞deg(ps 6 ) ≥ deg ⎜⎝ ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s ⎟7⎠A + Bh⎛ ⎡⎤ ⎞deg(ps 6 ) ≥ deg ⎜⎝ ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ ps ⎟8⎠A + Bh

107For the c maximization constraintmax{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(p 2 )max{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(ŝ 3 p)4.5.2 Expanding Interior Algorithm for Output Feedback DesignWe now adapt the expanding interior algorithm proposed in [13] to search fordynamic output feedback controllers to stabilize (3.14). First we fix the dynamic controller’snumber of states, n ξ ∈ Z + , and we define a variable sized regionP β := {[x; ξ] ∈ R n+n ξ| p([x; ξ]) ≤ β}for p ∈ Σ n+nξand positive definite. We then defineD := {[x; ξ] ∈ R n+n ξ| V ([x; ξ]) ≤ 1}for some as of yet unknown candidate Lyapunov function. Following the developments forthe state feedback case{[x; ξ] ∈ R n+n ξ| p([x; ξ]) ≤ β} ⊆ {[x; ξ] ∈ R n+n ξ| V ([x; ξ]) ≤ 1}{[x; ξ] ∈ R n+n ξ| V ([x; ξ]) ≤ 1} \ {0} ⊆⎧⎡⎤⎪⎨⎫⎪ [x; ξ] ∈ R n+n ξ∇V ∗ ([x; ξ]) ⎢f(x) + g(x)(C(ξ) + D(ξ)h(x))⎬⎥⎣⎦⎪⎩< 0 ⎪∣A(ξ) + B(ξ)h(x)⎭{[x; ξ] ∈ R n+n ξ| V ([x; ξ]) ≤ 0, [x; ξ] ≠ 0} = φ

108with V (0) = 0. Via the Positivstellensatz, maximizing the size of P β subject to the constraintsabove becomesmaxβV ∈ R n+nξ , V (0) = 0, s i ∈ Σ n+nξA ∈ R n ξn ξ, A(0) = 0, B ∈ R n ξ ×pn ξC ∈ R m n ξ, C(0) = 0, D ∈ R m×pn ξs.t.(4.22)V − l 1 ∈ Σ n+nξ− ( (β − p)s 6 + (V − 1) ) ∈ Σ n+nξ⎛⎡⎤ ⎞− ⎜⎝ (1 − V )s 8 + ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 9 + l 2⎟⎠ ∈ Σ n+n ξA + BhWe now can propose an algorithm to prove that the fixed point [x; ξ] = 0 is asymptoticallystable and estimate its region of attraction by iteratively solving (4.22). Again,for the controller synthesis part of the iteration we will have to introduce the intermediatevalue α.Algorithm 10 (Expanding Interior Output Feedback Design)An iterative searchto expand the region P β and thus the region D := {x ∈ R n |V (x) ≤ 1} in Theorem 10 startingfrom a positive definite candidate Lyapunov function and a n ξ state candidate dynamicoutput feedback controller.Let i be the iteration index and set i = 1. Denote the candidate Lyapunov functionV (i=0) and the candidate output feedback controller A (i=0) , B (i=0) , C (i=0) , D (i=0) . Pick themaximum degree of the Lyapunov function, the controller, the SOS multipliers and the lpolynomials to be d V , d A , d B , d C , d D , d s6 , d s8 , d s9 and d l1 , d l2 respectively. Fix l k ([x; ξ]) =( ∑nɛj=1 xd l kj+ ∑ )n ξj=1 ξd l kjfor k = 1, 2 and some ɛ > 0. Additionally set l (i=0)2 = l 2 and

109β (i=0) = 0.1. SOS Weight Iteration:Set V = V (i−1) , A = A (i−1) , B = B (i−1) , C = C (i−1) , D = D (i−1) and l 2 = l (i−1)2 .Solve the linesearch on α where s 8 ∈ Σ n+nξ ,d s8and s 9 ∈ Σ n+nξ ,d s9maxs 8 ,s 9αs.t.⎛⎡⎤ ⎞− ⎜⎝ (α − V )s 8 + ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 9 + l 2⎟⎠ ∈ Σ n+n ξA + BhSet s (i)9 = s 9 . Continue to step 2.2. Controller Iteration:Set V = V (i−1) , l 2 = l (i−1)2 and s 9 = s (i)9 . Solve the linesearch on α where s 8 ∈ Σ n,dA8 ,A ∈ R n ξn ξ ,d A, A(0) = 0, B ∈ R n ξ×pn ξ ,d B, C ∈ R m n ξ ,d C, C(0) = 0 and D ∈ R m×pn ξ ,d DmaxA,B,C,D,s 8αs.t.⎛⎡⎤ ⎞− ⎜⎝ (α − V )s 8 + ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 9 + l 2⎟⎠ ∈ Σ n+n ξA + BhSet A (i) = A, B (i) = B, C (i) = C, D (i) = D, s (i)8 = s 8 and l (i)2 = l (i−1)2 /α. Continueto step 3.3. Lyapunov function iteration:Set A = A (i) , B = B (i) , C = C (i) , D = D (i) , s 8 = s (i)8 , s 9 = s (i)9 and l 2 = l (i)2 . Solve

110the linesearch on β where s 6 ∈ Σ n,ds6 and V ∈ R n+nξ ,d Vwith V (0) = 0maxV,s 6βs.t.V − l 1 ∈ Σ n+nξ− ( (β − p)s 6 + (V − 1) ) ∈ Σ n+nξ⎛⎡⎤ ⎞− ⎜⎝ (1 − V )s 8 + ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s 9 + l 2⎟⎠ ∈ Σ n+n ξA + BhSet V (i) = V and β (i) = β. Continue to step 4.4. If β (i) − β (i−1) is less than a specified tolerance, the set {x ∈ R n |V (i) (x) ≤ 1} containsP β (i) and is the largest estimate of the fixed point’s region of attraction, else incrementi and goto step 1.Remark 9 (Properties of the expanding interior algorithm) :• If the algorithm is started from a feasible point it will always remain feasible.Ifthe system’s linearization is controllable, then any linear controller that stabilizes thelinearized system and the corresponding quadratic Lyapunov function will always work,since they will stabilize the nonlinear system near to the origin.Once more, thisfeasibility property allows us to consider only a V variant of the algorithm.• Note that, unlike Algorithm 5, V must be positive definite, so it should have no linearterms and d Vshould be even.• Since the s’s, and the l’s are SOS, they must be of even degree.Additionally, thedegrees need to be chosen so that following relations hold:

111For the first SOS constraintd V = d l1For the second SOS constraintdeg(ps 6 ) ≥ d VFor the third SOS constraint⎛ ⎡⎤ ⎞deg(V s 8 ) ≥ deg ⎜⎝ ∇V ∗ ⎢f + gC + gDh⎥⎣⎦ s ⎟9⎠A + Bhdeg(V s 8 ) = d l24.5.3 Output Feedback Design ExampleWe can demonstrate Algorithms 9 and 10 with the double nonlinear spring-massdampersystem from §4.4.3 if we introduce an output function, h. We will consider twooutput functions⎡ ⎤h 1 (x) = ⎢x 3⎥⎣ ⎦x 4andh 2 (x) = x 4Both of these output functions provide measurements from the second mass, m 2 , that isnot directly actuated by u.In the example for semi-global exponential stability outputfeedback algorithms, we had to use h(x) = [x 1 ; x 2 ], which makes the observations andactuation colocated, to make the algorithms feasible.Using the system dynamics given by (4.20) with either h 1 or h 2 above we can finda linear controller and quadratic Lyapunov function candidate pair by employing the linear

112gap metric design steps shown in §3.5.2.In order to keep the size of the computations down, as well as, pose a challengingcontrol problem, we will fix n ξ = 1. For both algorithms and also both output functions wepickp([x; ξ]) = x 1 1 + x 2 2 + x 2 3 + x 2 4 + ξ 2 1We can now solve for an output feedback controller for (4.20) with each of theoutput functions. For Algorithm 9 set d V = 2, d A = d C = 1, d B = d D = 0, d s2 = d s3 = 2,d s4 = 0, d s6 = 4, d s7 = 2, d s8 = 0, d l1 = d l2 = 4, with dŝ2 = dŝ3 = 2 and dŝ4 = 0. ForAlgorithm 10 we use d V = 2, d A = d C = 1, d B = d D = 0, d s6 = d s8 = 2, d s9 = 0, withd l1 = 2 and d l2 = 4.In this set up Algorithm 9 will involve searches for SOS polynomials in 5 variablesof degree 4, so we will reduce the computational load by setting the tolerance to 0.1. While,since Algorithm 10 has SOS weights in 5 variables of degree 2 we set the tolerance to 0.01.Using h 1After 11 iterations Algorithm 9 converges to β = 11.9 with c max = 25.9, whileAlgorithm 10 reaches the β tolerance limit after 4 iterations at β = 0.22.For clarity, let the controller and Lyapunov function found by the expanding Dalgorithm be A d , B d , C d , D d and V d . Likewise, let the results from the expanding interioralgorithm be A i , B i , C i , D i and V i . To summarize the results, under the linear controller⎡ ⎤ ⎡⎤A d B d⎢ ⎥⎣ ⎦ = −1.50 −0.22 −1.55⎢⎥⎣⎦D d 3.36 1.37 2.20C d

113the system (4.20) is asymptotically stable and the region Ω d := {x ∈ R n |V d (x) ≤ c max }with⎡ ⎤x 1x 2V d ([x; ξ]) =x 3x⎢ 4⎥⎣ ⎦ξ 1∗ ⎡⎢⎣59.64 −8.65 −60.76 −31.19 −70.42−8.65 38.02 28.73 34.70 65.70−60.76 28.73 81.82 59.34 112.81−31.19 34.70 59.34 67.29 104.82−70.42 65.70 112.81 104.82 194.25⎤ ⎡ ⎤x 1x 2x 3x⎥ ⎢ 4⎥⎦ ⎣ ⎦ξ 1is an invariant set contained in the fixed point’s region of attraction. Additionally underthe controller⎡⎢⎣A iC i⎤ ⎡B i⎥⎦ = ⎢⎣D i−0.88 0.70 −1.190.99 0.10 1.06⎤⎥⎦the system (4.20) is asymptotically stable and the region Ω i := {x ∈ R n |V i (x) ≤ 1} withV i (x) = 1100⎡ ⎤x 1x 2x 3x⎢ 4⎥⎣ ⎦ξ 1⎤ ⎡ ⎤1.27 −0.03 −0.70 −0.09 −0.41x 1−0.03 1.27 −0.03 1.31 0.90x 2−0.70 −0.03 0.70 0.19 0.21x 3−0.09 1.31 0.19 2.86 1.48x⎢⎥ ⎢ 4⎥⎣⎦ ⎣ ⎦−0.41 0.90 0.21 1.48 1.03 ξ 1∗ ⎡is an invariant set contained in the fixed point’s region of attraction.Comparing the two controllers using h 1 , we find the volume of Ω d is 13.06, whileΩ i has a volume of 22.34 with no containment of Ω d . From this information, it would seemthat the controller (A i , B i , C i , D i ) would be preferable, however this comparison is basedon the size of the entire region including the contribution from the controller’s state. Inmost control implementations, we would pick the initial condition ξ 1 = 0, and if we look

114at the sizes of the invariant regions, with this restriction on ξ 1 ’s initial condition we findsomething very interesting. The volume of Ω d with ξ 1 = 0 is 6.65, while the volume of Ω iwith ξ 1 = 0 is only 5.80. This reversal in volumes shows that as long as we are starting thecontroller with initial condition ξ 1 = 0, we should use the controller (A d , B d , C d , D d ).Using h 2After 4 iterations Algorithm 9 converges to β = 1.1 with c max = 0.3, while Algorithm10 reaches the β tolerance limit after 4 iterations at β = 0.08.For clarity, let the controller and Lyapunov function found by the expanding Dalgorithm be A d , B d , C d , D d and V d . Likewise, let the results from the expanding interioralgorithm be A i , B i , C i , D i and V i . To summarize the results, under the linear controller⎡ ⎤ ⎡⎤A d B d⎢ ⎥⎣ ⎦ = −0.66 1.13⎢⎥⎣⎦D d −1.09 0.60C dthe system (4.20) is asymptotically stable and the region Ω d := {x ∈ R n |V d (x) ≤ c max }with⎡ ⎤x 1x 2V d ([x; ξ]) =x 3x⎢ 4⎥⎣ ⎦ξ 1∗ ⎡⎢⎣4.88 −0.23 −1.85 −0.58 2.61−0.23 3.05 0.35 1.33 −1.29−1.85 0.35 1.37 0.50 −1.17−0.58 1.33 0.50 1.75 −2.482.61 −1.29 −1.17 −2.48 2.46⎤ ⎡ ⎤x 1x 2x 3x⎥ ⎢ 4⎥⎦ ⎣ ⎦ξ 1is an invariant set contained in the fixed point’s region of attraction. Additionally under

115the controller⎡⎢⎣A iC i⎤ ⎡B i⎥⎦ = ⎢⎣D i−0.92 −2.230.82 0.88⎤⎥⎦the system (4.20) is asymptotically stable and the region Ω i := {x ∈ R n |V i (x) ≤ 1} with⎡ ⎤x 1x 2V i (x) =x 3x⎢ 4⎥⎣ ⎦ξ 1∗ ⎡⎢⎣⎤ ⎡ ⎤6.67 −0.31 −1.35 −0.73 −2.81x 1−0.31 5.63 0.70 4.72 2.36x 2−1.35 0.70 1.78 2.61 2.74x 3−0.73 4.72 2.61 5.16 2.70x⎥ ⎢ 4⎥⎦ ⎣ ⎦−2.81 2.36 2.74 2.70 2.60 ξ 1is an invariant set contained in the fixed point’s region of attraction.Comparing the two controllers using h 2 , we find the volume of Ω d is 0.11, while Ω ihas a volume of 1.58 with no containment of Ω d . Again this would seem to imply that thecontroller (A i , B i , C i , D i ) would be preferable. When we look at setting ξ 1 = 0 to reflectthe choice of controller initial condition, the volume of Ω d with ξ 1 = 0 is 0.13, while thevolume of Ω i with ξ 1 = 0 is still larger at 0.72 and also it contains the restricted version ofΩ d . Note that these volumes are now in R 4 while the original volumes were in R 5 , so thevolume increase for Ω i is legitimate.Since Ω i is larger in both the restricted and non-restricted cases, (A i , B i , C i , D i ) isthe preferred controller. Note though, that the volumes for the invariant regions using h 2are smaller than those achievable from h 1 , which in turn are smaller than those in the statefeedback case. Clearly, the more information that is available, the larger the set of initialconditions for which the controller will asymptotically stabilize the system.

1164.6 Chapter SummaryIn this chapter we looked at using SOS optimization to do local system analysisand controller synthesis. We derived two algorithms, expanding D and expanding interior,that are at the heart of our approach to both stability analysis and controller synthesis.Additionally they provide ways to optimize the size of positively invariant regions about afixed point.Also, we provided methods to analyze the effects of external disturbances on agiven system, by bounding the system’s unit energy reachable set set, finding the largestpeak disturbance for which a given set is invariant, and bounding the induced L 2 → L 2gain from disturbance to output on an invariant set.We then used the expanding D and expanding interior algorithms to design bothstate feedback and output feedback controllers.Using these approaches we design statefeedback controllers for an example system and applied the disturbance analysis techniquesto study the controllers’ ability to reject disturbances. Additionally, we applied our algorithmsto design single state output feedback controllers for the same example system withdifficult output functions. As compared to the semi-global exponential stability designs,these local algorithms can stabilize much more difficult systems, but the results only holdon smaller regions of the state space.

117Chapter 5Discrete Time Containment &StabilityUp to this point all of the applications of SOS programming that we have consideredhave been for continuous time polynomial systems; now, we will look at similarproblems for discrete time polynomial systems. As opposed to the continuous time case, indiscrete time we can easily make arguments about set invariance without using Lyapunovfunctions. However, these set invariance results do not provide any information about thesystem’s stability, so we will use a Lyapunov function approach to prove stability. Unfortunately,due to the way the discrete time Lyapunov approach proves stability, we will not beable to use SOS programming to design controllers for discrete time polynomial systems.

1185.1 Set Invariance for Discrete Time SystemsConsider the systemx k+1 = f(x k ) (5.1)with f ∈ R n n and f(0) = 0. We would like to know if a region of the state space X ⊂ R n isinvariant under the action of f. We know that X is invariant under f if x ∈ X ⇒ f(x) ∈ X.If we define X with a semialgebraic setX := {x ∈ R n |p(x) ≤ β}we can check invariance with the following set containment constraint{x ∈ R n | p(x) ≤ β} ⊆ {x ∈ R n | p(f(x)) ≤ β} (5.2)We can check this constraint with a generalized S-procedure from §2.2.1, which asks if thereexists s ∈ Σ n such that(β − p ◦ f) − s(β − p) ∈ Σ n (5.3)To have a chance at making this constraint feasible we will need to pick s such thatdeg(sp) ≥ deg(p ◦ f)which insures that the positive term sp has the highest degree. Additionally, we can lookto find the largest invariant region by adding a cost function of β to the feasibility problem(5.3).5.1.1 Set Invariance ExampleConsider the systemx k+1 = −x 4 k + x2 k

119The system has fixed points at x = {0, −1.325}, and its linearization about the origin givesno information about the system. Fixing p(x) = x 2 , we would like to find the largest valueof β such that the constraint (5.3) holds. With d s = 6, making both terms in (5.3) of degree8, we find the largest value of β to be 1.75, which implies that the set{x ∈ R| − 1.322 ≤ x ≤ 1.322}is invariant under the system’s dynamics.Since this region pushes up against the fixedpoint, we know that we have indeed expanded the set {x ∈ R|p(x) ≤ β} as much as ispossible.5.1.2 Set Invariance under DisturbancesThe check above is useful to determine if a region is invariant under a system’sdynamics, however, it is often more useful to know if the region remains invariant whena disturbance is introduced into the system. For example, in the receding horizon controlstrategy put forward in [14], one of the requirements for it to work is that there must existregion of state space that is invariant under the action of the dynamics and some set ofdisturbances.We can expand the previous invariance check to include external disturbances bychecking if the systemx k+1 = f(x k , w k ) (5.4)with w k ∈ W ⊂ R m , f ∈ R n n+m and f(0, 0) = 0 satisfiesx ∈ X, w ∈ W ⇒ f(x, w) ∈ X

120If we describe W with a semialgebraic setW := {w ∈ R m |q(w) ≤ γ}then we can write an analog of (5.2){x ∈ R n , w ∈ R m |p(x) ≤ β} ∩ {x ∈ R n , w ∈ R m |q(w) ≤ γ}⊆ {x ∈ R n , w ∈ R m |p(f(x, w)) ≤ β}which, again, can be checked with a generalized S-procedure, from §2.2.1,∃s 1 , s 2 ∈ Σ n+ms.t.(5.5)(β − p ◦ f) − s 1 (β − p) − s 2 (γ − q) ∈ Σ n+mWe can also add a cost function to (5.5) to find the maximum allowable disturbance(max γ), the smallest invariant set (min β), or any other combination of these objectives.However, for feasibility we will need the highest degree SOS term to be positive ormax {deg(s 1 p), deg(s 2 q)} ≥ deg(p ◦ f)This inequality presents us again with potentially very large optimization problems, sinces 1 , s 2 have potentially large degree and are now polynomials in both x and w.5.1.3 Set Invariance under Disturbances ExampleConsider the previous containment example, except now subject it to a disturbancex k+1 = −x 4 k + x2 k + w k

121Fixing p(x) = x 2 , and q(w) = w 2 , we will fix values of β ∈ [0, 1.75] and for each of thesevalues we will search for the largest value of γ such that the constraint (5.5) holds.Additionally, since w enters the dynamics linearly, we can solve for the exact valueof γ for a given β. Noting that x 2 ≤ β and w 2 ≤ γ are equivalent to |x| ≤ √ β and |w| ≤ √ γ,respectively, we know that for |x| ≤ √ β the invariance relation becomes− √ β + √ γ ≤ −x 4 + x 2 ≤ √ β − √ γThus we can solve for the exact value of γ withγ(β) =(max(√β − max|x|≤ √ β∣∣−x 4 + x 2∣ )) 2∣ , 0We can now compare the SOS programming approach to the exact results. Figure5.1 gives the exact results with a solid line, while the largest values of γ using the optimizationapproach of (5.5), with d s1 = d s2 = 6, are plotted as stars. From the plot, we can tellthat the SOS approach provides a very good lower bound and that we should pick β ≈ 1.2to maximize the magnitude of the disturbance that the system can reject while remainingin its original invariant set.5.2 Discrete Time Stability BackgroundRemembering that the system under consideration is (5.1)x k+1 = f(x k )with f ∈ R n n and f(0) = 0, we can form a Lyapunov argument for asymptotic stabilitywith the following theorem. The discrete time version of the asymptotic stability theorem

1220.80.70.60.5γ0.40.30.20.100 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8βFigure 5.1: The maximal γ for which (5.5) is feasible for given β are plotted as stars, whilethe solid line shows the exact results.is identical to the continuous time one, Theorem 10, except that the conditions on ˙V havebeen replaced with conditions on (V ◦ f − V ).Theorem 11 Let D ⊂ R n be a domain containing the equilibrium point x = 0 of the system(5.1). Let V : D → R be a continuously differentiable function such thatV (0) = 0V (x) > 0 on D \ {0}andV (f(x)) − V (x) < 0 on D \ {0}then the system (5.1) is asymptotically stable about x = 0. Moreover, any region Ω β :={x ∈ R n |V (x) ≤ β} such that Ω β ⊆ D describes an positively invariant region contained inthe equilibrium point’s domain of attraction.

124Let i be the iteration number starting at one.Denote the candidate Lyapunovfunction V (i=0) and pick the maximum degree of the Lyapunov function, the SOS multipliersand the l polynomials to be d V , d s2 , d s3 , d s4 , d s6 , d s7 , d s8 and d l1 , d l2 respectively. Pick themaximum degrees for the the level set maximization problem (4.2) and denote them dŝ2 , dŝ3and dŝ4 . Fix l i (x) = ɛ ∑ nj=1 xd l ijfor i = 1, 2 and some ɛ > 0. Additionally set β (i=0) = 0.1. Set V = V (i−1) and solve the linesearch on β where s 2 ∈ Σ n,ds2 , s 3 ∈ Σ n,ds3 , s 4 ∈Σ n,ds4 , s 6 ∈ Σ n,ds6 , s 7 ∈ Σ n,ds7 and s 8 ∈ Σ n,ds8maxs 2 ,s 3 ,s 4 ,s 6 ,s 7 ,s 8βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1−(β − p)s 6 − (V ◦ f − V )s 7 − (V ◦ f − V )(β − p)s 8 − l 2∈∈Σ nΣ n(5.6)Set s (i)3 = s 3 , s (i)4 = s 4 , s (i)7 = s 7 and s (i)8 = s 8 . Continue to step 2.2. Set s 3 = s (i)3 , s 4 = s (i)4 , s 7 = s (i)7 and s 8 = s (i)8 . Solve the linesearch on β whereV ∈ R n,dV with V (0) = 0, s 2 ∈ Σ n,ds2 and s 6 ∈ Σ n,ds6maxV,s 2 ,s 6βs.t.−(β − p)s 2 + V s 3 + V (β − p)s 4 − l 1−(β − p)s 6 − (V ◦ f − V )s 7 − (V ◦ f − V )(β − p)s 8 − l 2∈∈Σ nΣ n(5.7)Set β (i) = β and V (i) = V . If β (i) − β (i−1) is less than a specified tolerance go to step3, else increment i and go to step 1.3. Set V = V (i) and β = β (i) . Solve the linesearch on c where ŝ 2 ∈ Σ n,dŝ2 , ŝ 3 ∈ Σ n,dŝ3

125and ŝ 4 ∈ Σ n,dŝ4maxŝ 2 ,ŝ 3 ,ŝ 4cs.t.−ŝ 2 (c − V ) − ŝ 3 (p − β) − ŝ 4 (p − β)(c − V ) − (p − β) 2 ∈ Σ nSet c max = c. The set {x ∈ R n |V (x) ≤ c max } is the resulting estimate of the regionof attraction, for it is positively invariant, contained within D and all of its pointsconverge to the fixed point x = 0.Remark 10 (Properties of the expanding D algorithm) :• If the algorithm is started from a feasible point it will always remain feasible. If thesystem’s linearization is stable, then the linearization’s quadratic Lyapunov functionwill always work, since it will stabilize the nonlinear system near to the origin.• Note that V need not be positive definite, since it is required to be positive only onD \ {0}. However, to be positive over a region about the origin V must have no linearterms and d Vshould be even. Clearly if V were chosen to be positive definite, thenthe first constraint in (5.6) and (5.7) can just become V − l 1 ∈ Σ n .• Since the s’s, l’s and the ŝ’s are SOS, they must be of even degree. Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintmax{deg(ps 2 ), deg(V s 3 )} ≥ d l1max{deg(ps 2 ), deg(V s 3 )} ≥ deg(V ps 4 )

126For the second SOS constraintdeg(ps 6 ) ≥ d l2deg(ps 6 ) ≥ max{deg((V ◦ f − V )s 7 ), deg((V ◦ f − V )ps 8 )}For the c maximization constraintmax{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(p 2 )max{deg(ŝ 2 V ), deg(ŝ 4 pV )} ≥ deg(ŝ 3 p)5.3.2 Expanding Interior AlgorithmWe can also adapt Algorithm 6 for discrete time systems by following the stepsthat we used to derive it in §4.2.2. Define a variable sized regionP β := {x ∈ R n |p(x) ≤ β}for p ∈ Σ n , positive definite, and we estimate the region of attraction by maximizing βsubject to the constraint that all of the points in P βconverge to the origin under thesystem’s dynamics. Following Theorem 11, if we defineD := {x ∈ R n |V (x) ≤ 1}for an unknown candidate Lyapunov function, then P β must be contained D. Additionallywe need,{x ∈ R n |V (x) ≤ 1} \ {0} ⊆ {x ∈ R n |V (f(x)) − V (x) < 0}Since V is unknown, to ensure that V is positive on D \ {0} we require that V be positiveeverywhere away from x = 0. With this set of requirements, we can propose the discretetime version of the expanding interior algorithm.

127Algorithm 12 (Expanding Interior) An iterative search to expand the region P β andthus the region D := {x ∈ R n |V (x) ≤ 1} in Theorem 10 starting from a positive definitecandidate Lyapunov function.Let i be the iteration number starting at one.Denote the candidate Lyapunovfunction V (i=0) and pick the maximum degree of the Lyapunov function, the SOS multipliersand the l polynomials to be d V , d s6 , d s8 , d s9 and d l1 , d l2 respectively. Fix l i (x) = ɛ ∑ nj=1 xd l ijfor i = 1, 2 and some ɛ > 0. Additionally set β (i=0) = 0.1. Set V = V (i−1) and solve the linesearch on β where s 6 ∈ Σ n,ds6 , s 8 ∈ Σ n,ds8 ands 9 ∈ Σ n,ds9maxs 6 ,s 8 ,s 9β(5.8)− ( (β − p)s 6 + (V − 1) ) ∈ Σ nSet s (i)8 = s 8 and s (i)9 = s 9 . Continue to step 2.s.t.2. Set s 8 = s (i)8 and s 9 = s (i)9 . Solve the linesearch on β where V ∈ R n,d Vwith V (0) = 0and s 6 ∈ Σ n,ds6maxV,s 6βs.t.V − l 1 ∈ Σ n(5.9)− ( )(1 − V )s 8 + (V ◦ f − V )s 9 + l 2 ∈ Σn− ( (β − p)s 6 + (V − 1) ) ∈ Σ n

128Set β (i) = β and V (i) = V . If β (i) − β (i−1) is less than a specified tolerance goto step3, else increment i and go to step 1.3. The set {x ∈ R n |V (i) (x) ≤ 1} contains P β (i) and is the largest estimate of the fixedpoint’s region of attraction.Remark 11 (Properties of the expanding interior algorithm) :• If the algorithm is started from a feasible point it will always remain feasible. If thesystem’s linearization is stable, then the linearization’s quadratic Lyapunov functionwill always work, since it will stabilize the nonlinear system near to the origin.• Note that, unlike Algorithm 11, V must be positive definite, so it should have no linearterms and d Vshould be even.• Since the s’s, and the l’s are SOS, they must be of even degree.Additionally, thedegrees need to be chosen so that following relations hold:For the first SOS constraintd V = d l1For the second SOS constraintdeg(ps 6 ) ≥ d VFor the third SOS constraintdeg(V s 8 ) ≥ deg((V ◦ f − V )s 9 )deg(V s 8 ) = d l2

1295.3.3 Estimating the Region of Attraction ExampleTo compare the effectiveness of Algorithm 11 (Expanding D) and Algorithm 12(Expanding Interior) consider estimating the region of attraction for the following system,a sampled version of the damped pendulum system (4.11),⎡ ⎤⎢x 1⎥⎣ ⎦x 2k+1⎡ ⎤= ⎢x 1⎥⎣ ⎦x 2k⎡+ t s⎢⎣− 810 x 2 −x 2(x 1 − x3 16)⎤⎥⎦k(5.10)where t s is the sampling time. As with the continuous time version, the system has threeequilibrium points: (0, 0) and (± √ 6, 0) and the linearization about the first is stable whilethe others are not. Following the continuous time version we pickp(x) = x 2 1 + x 2 2and we use the system’s linearization A f to construct a quadratic Lyapunov function V (x) =x ∗ P x whereSolving this feasibility problem we findP ≻ 0A ∗ f P A f − P ≺ 0⎡⎤P = ⎢2.07 0.87⎥⎣ ⎦0.87 2.13which we will use to form the candidate Lyapunov function.If we set the stopping tolerances to β (i) − β (i−1) = .01, t s = 110, and set the degreesas below, we can compare Algorithms 11 and 12 with their continuous time counterparts,Algorithms 5 and 6 . For the expanding D algorithm, set d V = 2, d s2 = d s3 = d s7 = 2,

130d s4 = d s8 = 0, d s6 = 6, d l1 = 4 and d l2 = 8. For the expanding interior algorithm setd V = 2, d s8 = 4, d s6 = d s9 = 0, d l1 = 2 and d l2 = 6.The expanding D algorithm converges to have a radius of √ β = 2.41 after 2iterations, while the expanding interior algorithm converges so that P β has a radius of √ β =2.05 after 12 iterations. These numbers are essentially the same as in the continuous timecase, which shows that the expanding regions are both as large as possible. Additionally theinvariant regions given by the Lyapunov function level sets that contain and are containedby P β and D, respectively, are visually identical their continuous time analogs shown inFigure 4.1.Since the discrete time and continuous time results line up almost perfectly we cansee that the Lyapunov design techniques work in similar manners for both. In the continuoustime case, it was straight forward to extend the Lyapunov design algorithms to includecontroller design, and we would like to extend this discrete/continuous stability parallelto controller design.However, we can not extend these techniques to the discrete timecontroller synthesis problem. Since we would be searching for K to make (V ◦(f +gK)−V )negative on some region, which is not linear in the coefficients of K even when V is fixed,we would not have an SOS problem.5.4 Chapter SummaryIn this chapter we investigated using SOS optimization techniques to answer systemtheoretic questions for discrete time systems with polynomial system maps. First, webroke with the continuous time approach by studying set invariance without using a Lya-

131punov function. We illustrated a way to directly show that a given set is invariant under thesystem’s dynamics as well as under a bounded disturbance. Simple examples were providedfor both approaches to demonstrate their utility. In the set invariance under disturbancesexample we also found the exact solution to the given problem to illustrate the quality ofour approach.We then extended the local stability algorithms, expanding D and expanding interior,to discrete time systems. We used a sampled version of the continuous time localstability example to show that the algorithms function in essentially the same manner. Additionally,we showed the fundamental limitation of our Lyapunov based SOS optimizationapproach when it is applied to discrete time systems, which keeps us from being able tomove from investigating a system’s stability to designing controllers for it.

132Chapter 6Conclusions and RecommendationsThis thesis considered the problem of using a Lyapunov based approach to analyzethe stability and performance of polynomial systems and to synthesize polynomialstate feedback and output feedback controllers. The polynomial Lyapunov approach wasmade computationally tractable by invoking the Positivstellensatz to form suitable sufficientconditions that could be iteratively solved with convex optimization.In Chapter 3, we illustrated a iterative approach to finding Lyapunov functions andcontrollers to prove that a closed loop polynomial system was semi-globally exponentiallystable. Additionally, we provided a Lyapunov base approach to bound the system’s inducedL 2 gain from disturbance to output. However the global nature of these results somewhatlimits their application, since in many cases the system either has fixed points away fromthe origin or is a model of an underlying system that is only valid on a fixed region of statespace.In light of these restrictions, we developed two different algorithmic approaches

133to do localized analysis and controller synthesis in Chapter 4. The Expanding D approachworks to find the largest estimate of the system’s region of attraction by expanding the regionwhere the Lyapunov conditions hold, while the Expanding Interior approach searchesfor an invariant set that contains an expanding region. The algorithms both have strengthsand weaknesses and these are often complimentary. Also, we expanded our approach todisturbance analysis by providing bounds on the reachable set under a unit energy disturbance,proving set invariance under peak bounded disturbances and introducing a localinduced L 2 gain bound.We then extended the local continuous time stability results, using the two differentapproaches, to discrete time systems in Chapter 5. Due to the nature of the Lyapunovtheorem in discrete time, we were unable to use similar methods to do controller design.However, due to the discrete time set up we were able to find non-Lyapunov based conditionsto insure that a region of initial conditions was invariant under the system’s dynamics anda given set of disturbances.The results presented in this thesis leave several topics open for future research.Two of them are discussed below• Integrate Performance Measures into Controller Design:In this thesis, we have constructed a series of algorithms that find Lyapunov functionsand controllers to demonstrate either global or local stability, as well as techniquesfor analyzing a system’s performance under disturbances.However, we have notintegrated these performance measures into the controller design procedures.If controller design and performance analysis were performed simultaneously, then it

134would be possible to use the design algorithm to optimize some aspect of the system’sperformance, instead of just checking the performance after the controller is designed.With a joint approach to these problems we could provide a valuable tool that wouldallow us to design robust controllers for polynomial systems with built-in performanceguarantees.• Minimize the Number of Non-Zero Controller TermsIf we were to design a high degree polynomial controller, we would be confrontedwith the problem of trying to implement a hugely complex polynomial with a largenumber of terms. Are all these terms necessary, or can some of them be set to zero?Clearly, we can run the design algorithm once including all of the monomials, afterwhich we can run the algorithm again with all of the monomials that have smallcoefficients removed.In the global state feedback controller design example, thistechnique worked, but this approach does not guarantee that the given controller hasthe fewest number of non-zero terms. If we could find a way to setup the controllerdesign algorithm to minimize the number of non-zero controller terms, then we could,possibly greatly, reduce the complexity of the controller implementation.

135Bibliography[1] C. Berg. The multidimensional moment problem and semigroups. In Proceedings ofSymposia in Applied Mathematics, volume 37, pages 110–124, 1987.[2] J. Bernussou, P. Peres, and J. Geromel. A linear programming oriented procedurefor quadratic stabilization of uncertain systems. Systems & Control Letters, 13:65–72,1989.[3] J. Bochnak, M. Coste, and M-F. Roy. Géométrie Algébrique Réelle. Springer, Berlin,1986.[4] N. Bose and C. Li. A quadratic form representation of polynomials of several variablesand its applications. IEEE Transactions on Automatic Control, 14:447–448, 1968.[5] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities inSystem and Control Theory. SIAM, Philadelphia, 1994.[6] C. Caramanis. Non-convex optimization via real algebraic geometry. Available fromhttp://web.mit.edu/6.962/www/www fall 2001/schedule.html, 2002.

136[7] M. Choi, T. Lam, and B. Reznick. Sums of squares of real polynomials. In Proceedingsof Symposia in Pure Mathematics, volume 58(2), pages 103–126, 1995.[8] J. Doyle and C. Chu. Matrix interpolation and H ∞ performance bounds. In Proceedingsof the American Control Conference, pages 129–134, 1985.[9] K. Gatermann and P. Parrilo. Symmetry groups, semidefinite programs, and sums ofsquares. Technical Report arXiv:math.AC/0211450, ETH Zurich, 2002.[10] T. Georgiou and M. Smith. Optimal robustness in the gap metric. IEEE Transactionson Automatic Control, 35(6):673–686, 1990.[11] W. Hahn. Stability of Motion. Springer-Verlag, 1967.[12] D. Henrion and J. Lasserre. GloptiPoly: Global optimization over polynomials withMatlab and SeDuMi. In Proceedings of the Conference on Decision and Control, pages747–752, 2002.[13] Z. Jarvis-Wloszek, R. Feeley, W. Tan, K. Sun, and A. Packard. Some controls applicationsof sum of squares programming. In Proceedings of the Conference on Decisionand Control, pages 4676–4681, 2003.[14] Z. Jarvis-Wloszek, D. Philbrick, M. Kaya, A. Packard, and G. Balas. Control with disturbancepreview and on-line optimization. IEEE Transactions on Automatic Control,to appear.[15] T. Johansen. Computation of Lyapunov functions for smooth nonlinear systems usingconvex optimization. Automatica, 36:1617–1626, 2000.

137[16] H. Khalil. Nonlinear Systems. Prentice Hall, Upper Saddle River, NJ, 2nd edition,1996.[17] P. Kokotović. Constructive nonlinear control: Progress in the 90’s. In Proceedings ofthe IFAC World Congress, pages 49–77, 1999.[18] J. Lasserre. Global optimization with polynomials and the problem of moments. SIAMJournal of Optimization, 11(3):796–817, 2000.[19] T. Motzkin. The arithmetic-geometric inequality. In Inequalities, Proceedings of theSymposium at Wright-Patterson AFB, 1965, pages 202–224. Academic Press, 1967.[20] A. Papachristodoulou and S. Prajna. On the construction of Lyapunov functions usingthe sum of squares decomposition. In Proceedings of the Conference on Decision andControl, pages 3482–3487, 2002.[21] P. Parrilo. Semidefinite programming relaxations for semialgebraic problems. MathematicalProgramming Ser. B, 96:2:293–320, 2003.[22] V. Powers and T. Wörmann. An algorithm for sums of squares of real polynomials.Journal of pure and applied algebra, 127:99–104, 1998.[23] S. Prajna, A. Papachristodoulou, and P. Parrilo. Introducing SOSTOOLS: A generalpurpose sum of squares programming solver.In Proceedings of the Conference onDecision and Control, pages 741–746, 2002.[24] S. Prajna, A. Papachristodoulou, and P. A. Parrilo. SOSTOOLS:

138Sum of squares optimization toolbox for Matlab, 2002. Available fromhttp://www.cds.caltech.edu/sostools.[25] M. Putinar. Positive polynomials on compact semialgebraic sets. Indiana UniversityMathematical Journal, 42:969–984, 1993.[26] A. Rantzer. On convexity in stabilization of nonlinear systems. In Proceedings of theConference on Decision and Control, pages 2941–2945, 2000.[27] A. Rantzer. A dual to Lyapunov’s stability theorem. Systems & Control Letters,42:161–168, 2001.[28] B. Reznick. Extremal psd forms with few terms. Duke Mathematical Journal, 45:363–374, 1978.[29] B. Reznick. Some concrete aspects of Hilbert’s 17th problem. In Contemporary Mathematics,volume 253, pages 251–272, 2000.[30] K. Schmüdgen. The k-moment problem for compact semialgebraic sets. MathematischeAnnalen, 289:203–206, 1991.[31] G. Stengle. A nullstellensatz and a positivstellensatz in semialgebraic geometry. MathematischeAnnalen, 207:87–97, 1974.[32] G. Stengle. Complexity estimates for the Schmüdgen postivstellensatz. Journal ofComplexity, 12:167–174, 1996.[33] J. Sturm. Using SeDuMi 1.02, a Matlab toolbox for optimizaton over symmetric

139cones. Optimization Methods and Software, 11-12:625–653, 1999. Available fromhttp://fewcal.cub.nl/sturm/software/sedumi.html.[34] S. Tzafestas and K. Anagnostou. Stabilization of singularly perturbed strictly bilinearsystems. IEEE Transactions on Automatic Control, 10:943–946, 1984.[35] L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49–95,1996.[36] A. Zelentsovsky. Nonquadratic Lyapunov functions for robust stability analysis of linearuncertain systems. IEEE Transactions on Automatic Control, 39(1):135–138, 1994.

140Appendix ASemidefinite ProgrammingSemidefinite programming (SDP) considers the following optimization problemminx∈R nc ∗ xs.t. F (x) := F 0 + x 1 F 1 + · · · + x n F n ≽ 0with c ∈ R n , and F i = F ∗i∈ R p×p for i = 0, 1, . . . , n. The most important theoretical propertyof SDP is its convexity, which allows it to be solved numerically with great efficiently[33]. SDP has many other useful theoretical properties that are covered in detail in thesurvey [35].Often SDP is used to solve the feasibility problem: does there exist x ∈ R n suchthat F (x) ≽ 0? The generalized inequality F (x) ≽ 0 is linear, or strictly speaking affine,in x, so the feasibility problem is often referred to as a linear matrix inequality (LMI). Oneproperty of LMIs is that a set of them can be turned into single larger block diagonal LMI.

141Example 4 Consider the LMIs F (x) ≽ 0 and G(x) ≽ 0 they can be written as⎡⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢G(x)⎥⎣⎦ = ⎢G 0⎥⎣ ⎦ + x ⎢G 1⎥1 ⎣ ⎦ + · · · + x ⎢G n⎥n ⎣ ⎦ ≽ 0F (x) F 0 F 1 F nTo illustrate a less obvious use of LMIs consider the following Lyapunov stability argument.Example 5 Consider the linear system ẋ = Ax with x ∈ R n and the Lyapunov functionV (x) = x ∗ P x, with P unknown. If a P can be picked such that V (x) is positive definite and− ˙V (x) is positive semidefinite, then the system is stable. Noting that ˙V (x) = x ∗ (A ∗ P +P A)x, the problem can be posed as: does there exist a P such thatP ≻ 0−(A ∗ P + P A) ≽ 0If P is represented on the standard basis as ∑ mi=1 p iE i , with m := (n + 1)n/2, then theLMIs above become⎡∑ m i=1⎢p iE i − ɛI⎣−(A ∗ ∑ mi=1 p iE i + ∑ ⎥⎦ ≽ 0mi=1 p iE i A)⎤for any ɛ > 0. The question of what value to use for ɛ can also be incorporated to give thefollowing SDP in the variables ɛ, p 1 , . . . , p mmaxp 1 ,...,p mɛ⎡⎤⎡⎤ ⎡s.t. p 1⎢E 1 ⎥⎣⎦ + . . . + p ⎢E m ⎥m ⎣⎦ + ɛ ⎢−I⎣−(A∗E 1 + E 1 A)−(A∗E m + E m A)⎤⎥⎦ ≽ 00which, if the maximum value for ɛ is strictly positive, gives a matrix P such that the Lyapunovfunction V demonstrates the stability of A.