Notes on Lagrange Multipliers
Notes on Lagrange Multipliers
Notes on Lagrange Multipliers
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<str<strong>on</strong>g>Notes</str<strong>on</strong>g> <strong>on</strong> <strong>Lagrange</strong> <strong>Multipliers</strong> and the C<strong>on</strong>sumer Problem<br />
A. Understanding the Lagrangian Optimizati<strong>on</strong> Problem<br />
A typical c<strong>on</strong>sumer problem has form<br />
Max u(x, y)<br />
s.t. px + qy < W.<br />
The Lagrangian corresp<strong>on</strong>ding to this c<strong>on</strong>sumer problem is<br />
L(x, y, λ) = u(x, y) + λ(W - px - qy).<br />
The Lagrangian optimizati<strong>on</strong> problem is based <strong>on</strong> finding a combinati<strong>on</strong> of values<br />
(x*, y*, λ*) that maximize the objective functi<strong>on</strong> L(x, y, λ) in (x, y) and minimize the<br />
objective functi<strong>on</strong> L(x, y, λ) in λ subject to the requirements x, y, λ > 0.<br />
The format of this optimizati<strong>on</strong> problem may seem unfamiliar; the maximizati<strong>on</strong> and<br />
minimizati<strong>on</strong> comp<strong>on</strong>ents of the problem may appear to work at cross-purposes. The<br />
easiest way to think about Lagrangian optimizati<strong>on</strong> is to c<strong>on</strong>sider subproblems related to<br />
the optimal choices (x*, y*, λ*).<br />
(1) Given (y*, λ*), the choice of x* must maximize L(x, y, λ).<br />
(2) Given (x*, λ*), the choice of y* must maximize L(x, y, λ).<br />
(3) Given (x*, y*), the choice of λ* must minimize L(x, y, λ).<br />
The “soluti<strong>on</strong>” to the Lagrangian problem is a combinati<strong>on</strong> of values (x*, y*, λ*) that<br />
solve subproblems (1) through (3) simultaneously.<br />
The choice of λ is best illustrated in a graph of bundles relative to the budget c<strong>on</strong>straint.<br />
Figure 1 graphs the budget line y = 40 – 2x, corresp<strong>on</strong>ding to (say) W = 40, p = 2, q = 1.
50<br />
45<br />
40<br />
35<br />
30<br />
25<br />
20<br />
15<br />
10<br />
5<br />
Figure 1:<br />
The <strong>Lagrange</strong> Multplier and the Budget C<strong>on</strong>straint<br />
A<br />
Cost < W<br />
B<br />
Cost > W<br />
0<br />
0 5 10 15 20 25<br />
Subproblem 3: Choosing λ given (x, y) to minimize L(x, y, λ).<br />
Bundles in set A have cost < W, meaning px + qy < W, or equivalently W - px - qy > 0.<br />
Since the product λ(W - px - qy) in the Lagrangian objective functi<strong>on</strong> is positive in set A,<br />
λ*(x, y) = 0 minimizes L(x, y, λ) for any combinati<strong>on</strong> of values (x, y) in set A.<br />
Similarly, bundles in set B have cost < W, implying that W - px - qy > 0. Since the<br />
product λ(W - px - qy) is negative in set B, λ*(x, y) → ∞ minimizes L(x, y, λ) for any<br />
combinati<strong>on</strong> of values (x, y) in set B.<br />
Finally, bundles <strong>on</strong> the budget line have cost = W, implying that W - px - qy = 0. Since<br />
the product λ(W - px - qy) is zero for any value of λ <strong>on</strong> the budget line, the choice of λ is<br />
unrestricted (at least in terms of producing a minimum value of the objective functi<strong>on</strong>)<br />
for any combinati<strong>on</strong> of values (x, y) in set B.<br />
Combining Subproblems 1&2 with Subproblem 3.<br />
We can simply work our way through the implicati<strong>on</strong>s of choices (x*, y*) in set A, set B<br />
and <strong>on</strong> the budget line to determine where the soluti<strong>on</strong> to the Lagrangian problem must<br />
lie in the graph. In essence, we are using a form of the “C<strong>on</strong>sistent C<strong>on</strong>jectures” (a fancy<br />
term for “Guess and Verify”) method for identifying the properties of the soluti<strong>on</strong> to a<br />
particular mathematical problem. In this case, we c<strong>on</strong>jecture in turn that the soluti<strong>on</strong><br />
(x*, y*, λ*) to the Lagrangian problem has values of (x*, y*) that fall in Set A, in Set B,<br />
or fall <strong>on</strong> the budget line. For each of these three possibilities, we attempt to verify<br />
whether it is possible to solve subproblems 1 to 3 simultaneously with (x*, y*) in the<br />
given regi<strong>on</strong>.
C<strong>on</strong>jectured Soluti<strong>on</strong> in Set A: For combinati<strong>on</strong>s of (x, y) in Set A, λ*(x, y) = 0.<br />
But given λ = 0, the Lagrangian objective functi<strong>on</strong> is simply u(x, y), which takes its<br />
unc<strong>on</strong>strained maximum at values (x, y) with x → ∞, y → ∞. In words, if there is no<br />
penalty for going bey<strong>on</strong>d the budget c<strong>on</strong>straint, then the c<strong>on</strong>sumer should choose a<br />
bundle in set B as the soluti<strong>on</strong> to subproblems (1) and (2) – but this c<strong>on</strong>tradicts the<br />
c<strong>on</strong>jecture of (x, y) in set A.<br />
C<strong>on</strong>jectured Soluti<strong>on</strong> in Set B: For combinati<strong>on</strong>s of (x, y) in Set B, λ*(x, y) = ∞.<br />
But given λ = ∞, the Lagrangian objective functi<strong>on</strong> dramatically penalizes expenditures<br />
bey<strong>on</strong>d budget W while dramatically rewarding expenditures less then budget W. In this<br />
case, the c<strong>on</strong>sumer should choose x* = y* = 0 (i.e. a bundle in set A) as the soluti<strong>on</strong> to<br />
subproblems (1) and (2). Once again, this c<strong>on</strong>tradicts the c<strong>on</strong>jecture of (x, y) in set B.<br />
C<strong>on</strong>jectured Soluti<strong>on</strong> <strong>on</strong> Budget Line: From our analysis above, λ*(x, y) is unrestricted<br />
<strong>on</strong> the budget line – at least λ*(x, y) is unrestricted by subproblem 3 as any choice of λ<br />
will solve subproblem 3 when (x, y) is <strong>on</strong> the budget line. In this case, it is possible to<br />
find a specific value of λ that will induce the c<strong>on</strong>sumer to choose values of x*, y* in<br />
subproblems (1) and (2) that are <strong>on</strong> the budget line.<br />
In sum, we tried out three different c<strong>on</strong>jectures about where soluti<strong>on</strong>s (x*, y*) to the<br />
Lagrangian problem could lie in the graph in Figure 1 relative to the budget line. We<br />
c<strong>on</strong>clude that combinati<strong>on</strong>s of (x*, y*) below the budget line (Set A) and above the<br />
budget line (Set B) cannot be part of simultaneous soluti<strong>on</strong> to subproblems (1) to (3).<br />
However, a combinati<strong>on</strong> of (x*, y*) <strong>on</strong> the budget line can be part of a simultaneous<br />
soluti<strong>on</strong> to subproblems (1) to (3) for an appropriate choice of λ.<br />
The tricky element of this soluti<strong>on</strong> (x*, y*, λ*) to the Lagrangian problem is that the<br />
value of λ is chosen to support a particular soluti<strong>on</strong> of x*, y* (i.e. a combinati<strong>on</strong> <strong>on</strong> the<br />
budget line) rather than to solve subproblem 3. However, this is still c<strong>on</strong>sistent with the<br />
mathematical c<strong>on</strong>diti<strong>on</strong> of solving subproblems 1 to 3 simultaneously. Secti<strong>on</strong> B<br />
provides further interpretati<strong>on</strong> of the value λ*.
B. Interpreting λ* as a Shadow Price or Penalty Term<br />
The technical advantage of the Lagrangian formulati<strong>on</strong> is that it c<strong>on</strong>verts a c<strong>on</strong>strained<br />
maximizati<strong>on</strong> problem into an unc<strong>on</strong>strained maximizati<strong>on</strong> problem. That is, the<br />
Lagrangian problem formally allows choices of (x, y) that are not in the budget set.<br />
For the Lagrangian objective functi<strong>on</strong> to match the objective functi<strong>on</strong> u(x, y), the term<br />
λ(px + qy - W) must end up equal to zero. That is, either λ = 0 or px + qy = W; this<br />
combinati<strong>on</strong> of two possibilities is sometimes called a complementary slackness<br />
c<strong>on</strong>diti<strong>on</strong>. Further since we know that the soluti<strong>on</strong> to the c<strong>on</strong>sumer problem is <strong>on</strong> the<br />
budget line, we want to find a Lagrangian soluti<strong>on</strong> that satisfies px + qy = W<br />
Intuitively, the new term in the Lagrangian objective functi<strong>on</strong> must provide incentives for<br />
the c<strong>on</strong>sumer to choose (x, y) <strong>on</strong> the budget line. Let’s rewrite the Lagrangian with a<br />
negative sign <strong>on</strong> λ, meaning that we also have to reverse the signs <strong>on</strong> W, px, and qy.<br />
L(x, y, λ) = u(x, y) - λ(px + qy - W).<br />
This format for L(x, y, λ) encourages the interpretati<strong>on</strong> of λ as a “penalty” per unit of<br />
expenditure bey<strong>on</strong>d W. But since px + qy – W can be negative, λ also serves as a<br />
“reward” per unit of expenditure less than W.<br />
Naturally, if λ is too large, the c<strong>on</strong>sumer will choose (x, y) below the budget line, while if<br />
λ is too small, the c<strong>on</strong>sumer will choose (x, y) above the budget line. The <strong>on</strong>ly way to<br />
balance rewards and penalties and induce the c<strong>on</strong>sumer to stay <strong>on</strong> the budget line is to<br />
choose λ* = [∂u / ∂x] / p = [∂u / ∂y] / q for a particular point <strong>on</strong> the budget line. For the<br />
bundle (x*, y*) that is the best choice in terms of utility <strong>on</strong> the budget line, this value of<br />
λ* provides exactly the right incentives for the c<strong>on</strong>sumer.<br />
One last comment is that this entire c<strong>on</strong>structi<strong>on</strong> is tied to small changes in bundle from<br />
(x*, y*) <strong>on</strong> the budget line. Reducing or increasing expenditures for either x or y by a<br />
small amount ε would reduce change the value of the utility functi<strong>on</strong> u(x, y) by λ* units,<br />
but this change would be offset by a reward or penalty from the Lagrangian λ term.<br />
What about a much larger change in either x or y from (x*, y*)? Partial derivative<br />
approximati<strong>on</strong>s no l<strong>on</strong>ger apply for changes in quantities that are more than infinitesimal.<br />
But because of the decreasing returns c<strong>on</strong>diti<strong>on</strong> (essentially by the sec<strong>on</strong>d-order<br />
c<strong>on</strong>diti<strong>on</strong> for maximizati<strong>on</strong> in (x, y)), we know that the effect <strong>on</strong> the utility functi<strong>on</strong> from<br />
marginal changes in x and y decreases as x and y increase. That is, a large increase in<br />
expenditure <strong>on</strong> x or y from (x*, y*) would yield an increase in u(x, y) of less than λ* per<br />
additi<strong>on</strong>al $ spent, but with c<strong>on</strong>stant cost λ* units per $ spent from the Lagrangian term.<br />
In words, the first-order c<strong>on</strong>diti<strong>on</strong> indicates that utility gains and losses from small<br />
changes in expenditure from (x*, y*) are exactly offset by appropriate choice of λ* in the<br />
Lagrangian term. In additi<strong>on</strong>, the sec<strong>on</strong>d-order c<strong>on</strong>diti<strong>on</strong> (decreasing MRS) indicates<br />
that utility gains and losses from large changes in expenditure are more than offset by the<br />
appropriate choice of λ* in the Lagrangian term.