The Mathematics of Optimization

1
The Mathematics of Optimization
• Many economic theories begin with the
assumption that an economic agent is
seeking to find the optimal value of some
function
– consumers seek to maximize utility
– firms seek to maximize profit
• This set of notes introduces the
mathematics common to these problems
2
Maximization of a Function of One
Variable
• Simple example: Manager of a firm wishes
to maximize profits
  f (q)

Maximum profits
* (*) occur at q*
 = f(q)

Quantity
q*
3
Maximization of a Function of One
Variable
• The manager will likely try to vary q to see
where the maximum profit occurs
– an increase from q1 to q2 leads to a rise in 


0
*

q
2  = f(q)

1

Quantity
q1 q2 q*
4
Maximization of a Function of One
Variable
• If output is increased beyond q*, profit will
decline
– an increase from q* to q3 leads to a drop in 



0
*

 = f(q)
q
3

Quantity
q* q3
5
 Derivatives
• The derivative of  = f(q) is the limit of
/q for very small changes in q

d df f (q1  h)  f (q1 )
  lim
dq dq h  0 h

• The value of this ratio depends on the

value of q1

6
Value of a Derivative at a Point
• The evaluation of the derivative at the
point q = q1 can be denoted
d
dq q  q1

• In our previous example,

d d d
0 0 0
dq q q
1
dq q q
3
dq q  q *
7
 First Order Condition for a
Maximum
• For a function of one variable to attain its
maximum value at some point, the derivative at
that point must be zero

df
0
dq q q*

8
 Second Order Conditions
• The first order condition (d/dq) is a
necessary condition for a maximum, but
it is not a sufficient condition

If the profit function was u-shaped,
the first order condition would result
in q* being chosen and  would
be minimized

*

Quantity
q*
9
 Second Order Conditions
• This must mean that, in order for q* to be
the optimum,

d d
 0 for q  q * and  0 for q  q *
dq dq

• Therefore, at q*, d/dq must be

decreasing

10
 Second Derivatives
• The derivative of a derivative is called a
second derivative
• The second derivative can be denoted by

d 2 d 2f
2
or 2
or f " (q )
dq dq

11
 Second Order Condition
• The second order condition to represent a
(local) maximum is

d 
2

2
 f " (q ) q  q *  0
dq q q *

12
Example of Profit Maximization
• Suppose that the relationship between
profit and output is
 = 1,000q - 5q2
• The first order condition for a maximum is
d/dq = 1,000 - 10q = 0
q* = 100
• Since the second derivative is always
-10, q = 100 is a global maximum
13
Functions of Several Variables
• Most goals of economic agents depend
on several variables
– trade-offs must be made
• The dependence of one variable (y) on
a series of other variables (x1,x2,…,xn) is
denoted by

y  f ( x1, x 2 ,..., x n )
14
 Partial Derivatives

• The partial derivative of y with respect

to x1 is denoted by
y f
or or fx or f1
x1 x1 1

• It is understood that in calculating the

partial derivative, all of the other x’s are
held constant
15
 Young’s Theorem
• Under general conditions, the order in
which partial differentiation is conducted to
evaluate second-order partial derivatives
does not matter

fij  f ji

16
 First-Order Condition for a
Maximum (or Minimum)
• A necessary condition for a maximum (or
minimum) of the function f(x1,x2,…,xn) is
that dy = 0 for any combination of small
changes in the x’s
• The only way for this to be true is if
f1  f2  ...  fn  0
• A point where this condition holds is
called a critical point 17
 Finding a Maximum
• Suppose that y is a function of x1 and x2
y = - (x1 - 1)2 - (x2 - 2)2 + 10
y = - x12 + 2x1 - x22 + 4x2 + 5
• First-order conditions imply that

y
 2 x1  2  0 x1*  1
x1 OR
y x2  2
*

 2 x 2  4  0
x 2
18
 Production Possibility Frontier

• Earlier example: 2x2 + y2 = 225

• Can be rewritten: f(x,y) = 2x2 + y2 - 225 = 0
• Because fx = 4x and fy = 2y, the opportunity
cost trade-off between x and y is

dy  fx  4x  2x
  
dx fy 2y y

19
 Constrained Maximization

• What if all values for the x’s are not

feasible?
– the values of x may all have to be positive
– a consumer’s choices are limited by the
amount of purchasing power available
• One method used to solve constrained
maximization problems is the Lagrangian
multiplier method
20
 Lagrangian Multiplier Method
• Suppose that we wish to find the values
of x1, x2,…, xn that maximize
y = f(x1, x2,…, xn)
subject to a constraint that permits only
certain values of the x’s to be used
g(x1, x2,…, xn) = 0

21
 Lagrangian Multiplier Method
• The Lagrangian multiplier method starts
with setting up the expression
L = f(x1, x2,…, xn ) + g(x1, x2,…, xn)
where  is an additional variable called
a Lagrangian multiplier
• When the constraint holds, L = f
because g(x1, x2,…, xn) = 0

22
 Lagrangian Multiplier Method
• First-Order Conditions
L/x1 = f1 + g1 = 0
L/x2 = f2 + g2 = 0
.
.
.
L/xn = fn + gn = 0
L/ = g(x1, x2,…, xn) = 0
23
 Lagrangian Multiplier Method
• The first-order conditions can generally
be solved for x1, x2,…, xn and 

• The solution will have two properties:

– the x’s will obey the constraint
– these x’s will make the value of L (and
therefore f) as large as possible

24
 Lagrangian Multiplier Method
• The Lagrangian multiplier () has an
important economic interpretation
• The first-order conditions imply that
f1/-g1 = f2/-g2 =…= fn/-gn = 
– the numerators above measure the
marginal benefit that one more unit of xi will
have for the function f
– the denominators reflect the added burden
on the constraint of using more xi
25
 Lagrangian Multiplier Method
• At the optimal choices for the x’s, the
ratio of the marginal benefit of increasing
xi to the marginal cost of increasing xi
should be the same for every x
•  is the common cost-benefit ratio for all
of the x’s
marginal benefit of x i

marginal cost of x i
26
 Lagrangian Multiplier Method
• If the constraint was relaxed slightly, it
would not matter which x is changed
• The Lagrangian multiplier provides a
measure of how the relaxation in the
constraint will affect the value of y
•  provides a “shadow price” to the
constraint

27
 Lagrangian Multiplier Method
• A high value of  indicates that y could
be increased substantially by relaxing
the constraint
– each x has a high cost-benefit ratio
• A low value of  indicates that there is
not much to be gained by relaxing the
constraint
• =0 implies that the constraint is not
binding
28
 Duality

• Any constrained maximization problem

has associated with it a dual problem in
constrained minimization that focuses
attention on the constraints in the
original problem

29
 Duality
• Individuals maximize utility subject to a
budget constraint
– dual problem: individuals minimize the
expenditure needed to achieve a given level
of utility
• Firms minimize the cost of inputs to
produce a given level of output
– dual problem: firms maximize output for a
given cost of inputs purchased
30
 Constrained Maximization
• Suppose a farmer had a certain length of
fence (P) and wished to enclose the
largest possible rectangular shape
• Let x be the length of one side
• Let y be the length of the other side
• Problem: choose x and y so as to
maximize the area (A = x·y) subject to the
constraint that the perimeter is fixed at P =
2x + 2y
31
 Constrained Maximization
• Setting up the Lagrangian multiplier
L = x·y + (P - 2x - 2y)
• The first-order conditions for a maximum
are
L/x = y - 2 = 0
L/y = x - 2 = 0
L/ = P - 2x - 2y = 0

32
 Constrained Maximization
• Since y/2 = x/2 = , x must be equal to y
– the field should be square
– x and y should be chosen so that the ratio of
marginal benefits to marginal costs should be
the same
• Since x = y and y = 2, we can use the
constraint to show that
x = y = P/4
 = P/8
33
 Constrained Maximization
• Interpretation of the Lagrangian multiplier
– if the farmer was interested in knowing how
much more field could be fenced by adding an
extra yard of fence,  suggests that he could
find out by dividing the present perimeter (P)
by 8

– thus, the Lagrangian multiplier provides

information about the implicit value of the
constraint
34
 Constrained Maximization
• Dual problem: choose x and y to minimize
the amount of fence required to surround
the field
minimize P = 2x + 2y subject to A = x·y

• Setting up the Lagrangian:

LD = 2x + 2y + D(A - xy)

35
 Constrained Maximization
• First-order conditions:
LD/x = 2 - D·y = 0
LD/y = 2 - D·x = 0
LD/D = A - x·y = 0
• Solving, we get
x = y = A1/2
• The Lagrangian multiplier (D) = 2A-1/2

36
 Second Order Conditions -
Functions of One Variable
• Let y = f(x)
• A necessary condition for a maximum is
that
dy/dx = f ’(x) = 0
• To ensure that the point is a maximum, y
must be decreasing for movements away
from it
37
 Second Order Conditions -
Functions of One Variable
• The total differential measures the change
in y
dy = f ’(x) dx
• To be at a maximum, dy must be
decreasing for small increases in x
• To see the changes in dy, we must use the
second derivative of y
38
 Second Order Conditions -
Functions of One Variable
d [f ' ( x )dx ]
d y
2
 dx  f " ( x )dx  dx  f " ( x )dx 2

dx

• Note that d 2y < 0 implies that f ’’(x)dx2 < 0

• Since dx2 must be positive, f ’’(x) < 0

• This means that the function f must have a

concave shape at the critical point
39
 Second Order Conditions -
Functions of Two Variables
• Suppose that y = f(x1, x2)
• First order conditions for a maximum are
y/x1 = f1 = 0
y/x2 = f2 = 0
• To ensure that the point is a maximum, y
must diminish for movements in any
direction away from the critical point
40
 Second Order Conditions -
Functions of Two Variables
• The slope in the x1 direction (f1) must be
diminishing at the critical point
• The slope in the x2 direction (f2) must be
diminishing at the critical point
• But, conditions must also be placed on the
cross-partial derivative (f12 = f21) to ensure
that dy is decreasing for all movements
through the critical point
41
 Second Order Conditions -
Functions of Two Variables
• The total differential of y is given by
dy = f1 dx1 + f2 dx2
• The differential of that function is
d 2y = (f11dx1 + f12dx2)dx1 + (f21dx1 + f22dx2)dx2
d 2y = f11dx12 + f12dx2dx1 + f21dx1 dx2 + f22dx22
• By Young’s theorem, f12 = f21 and
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
42
 Second Order Conditions -
Functions of Two Variables
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
• For this equation to be unambiguously
negative for any change in the x’s, f11 and
f22 must be negative
• If dx2 = 0, then d 2y = f11 dx12
– for d 2y < 0, f11 < 0
• If dx1 = 0, then d 2y = f22 dx22
– for d 2y < 0, f22 < 0 43
 Second Order Conditions -
Functions of Two Variables
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
• If neither dx1 nor dx2 is zero, then d 2y will
be unambiguously negative only if
f11 f22 - f122 > 0
– the second partial derivatives (f11 and f22) must
be sufficiently negative so that they outweigh
any possible perverse effects from the cross-
partial derivatives (f12 = f21)
44
 Second Order Conditions -
Constrained Maximization
• Suppose we want to choose x1 and x2
to maximize
y = f(x1, x2)
• subject to the linear constraint
c - b1x1 - b2x2 = 0
• We can set up the Lagrangian
L = f(x1, x2) + (c - b1x1 - b2x2)

45
 Constrained Maximization

• The first-order conditions are

f1 - b1 = 0
f2 - b2 = 0
c - b1x1 - b2x2 = 0

46
 Constrained Maximization
• To ensure we have a maximum, the
function has be quasi-concave, that is,
any two points within the set of points for
which f  k, where k is a constant, can be
joined by a line contained completely in
the set (the set is convex)
• For quasi-concavity of the function f,
f11 f22 - 2f12f1f2 + f22f12 < 0

47
 Homogeneous Functions
• A function f(x1,x2,…xn) is said to be
homogeneous of degree k if
f(tx1,tx2,…txn) = tk f(x1,x2,…xn)
– when a function is homogeneous of degree
one (k=1), a doubling (t=2) of all of its
arguments doubles the value of the function
itself
– when a function is homogeneous of degree
zero, a doubling of all of its arguments leaves
the value of the function unchanged
48
 Homothetic Functions
• A homothetic function is one that is formed
by taking a monotonic transformation of a
homogeneous function
• For both homogeneous and homothetic
functions, the implicit trade-offs among the
variables in the function depend only on
the ratios of those variables, not on their
absolute values

49
 Homothetic Functions
• Suppose we are examining the simple,
two variable implicit function f(x,y) = 0
• The implicit trade-off between x and y for a
two-variable function is
dy/dx = -fx/fy
• If we assume f is homogeneous of degree
k, its partial derivatives will be
homogeneous of degree k-1
50
 Important Points to Note:
• Using mathematics provides a
convenient, short-hand way for
economists to develop their models
– implications of various economic
assumptions can be studied in a
simplified setting through the use of such
mathematical tools

51
 Important Points to Note:
• Derivatives are often used in economics
because economists are interested in
how marginal changes in one variable
affect another
– partial derivatives incorporate the ceteris
paribus assumption used in most economic
models

52
 Important Points to Note:
• The mathematics of optimization is an
important tool for the development of
models that assume that economic
agents rationally pursue some goal
– the first-order condition for a maximum
requires that all partial derivatives equal
zero

53
 Important Points to Note:
• Most economic optimization
problems involve constraints on the
choices that agents can make
– the first-order conditions for a
maximum suggest that each activity be
operated at a level at which the ratio of
the marginal benefit of the activity to its
marginal cost

54
 Important Points to Note:
• The Lagrangian multiplier is used to
help solve constrained maximization
problems
– the Lagrangian multiplier can be
interpreted as the implicit value (shadow
price) of the constraint

55
 Important Points to Note:
• First-order conditions are necessary
but not sufficient for ensuring a
maximum or minimum
– second-order conditions that describe
the curvature of the function must be
checked

56
 Important Points to Note:
• Certain types of functions occur in many
economic problems
– quasi-concave functions obey the second-
order conditions of constrained maximum or
minimum problems when the constraints are
linear
– homothetic functions have the property that
implicit trade-offs among the variables depend
only on the ratios of these variables

57