·
oneminenta Prsanitarie
Unconstrained Optimization
Anin Rupp
1
Extrema of a Function
To optimize a function means to find its largest or smallest possible value. There are many applications to
optimization, in the business world it is usually desirable to maximize profit and/or minimize cost.
badap
One necessary condition for optimization is the presence of a continuous function. Recall that a continuous
function is a function that has no holes or jumps (discontinuity).
2
Extrema of a Function
Now consider the following graph of a function
Highest local Max
local max
local min
lowest
local min
You can clearly see that 𝐴 is the highest point on the graph and 𝐺 is the lowest point. Thusly, 𝑓(𝑎) and 𝑓(𝑔) are
the highest and lowest 𝑦-values, respectively. This function has the maximum value at 𝑥 = 𝑎 and minimum value
at 𝑥 = 𝑔.
3
Extrema of a Function · asse ou miss
We can generalize the definition for maximum and minimum value (sometimes called extrema):
A function 𝑓 has 𝑓(𝑎) as its maximum value if 𝑓(𝑎) ≥ 𝑓(𝑥) for all 𝑥 in its domain, a
function 𝑓 has 𝑓(𝑔) as its minimum value if 𝑓(𝑔) ≤ 𝑓(𝑥) for all 𝑥 in its domain.
This simply means that the maximum value is the largest possible 𝑦-value attain by the function over its domain.
Similarly, the minimum value is the smallest possible 𝑦-value attained by the function over its domain.
4
Extrema of a Function
Not only is it important for optimization that the function be continuous, but also for the interval to be closed
and bounded (interval includes the end points).
20 5]
,
Extreme Value Theorem:
I 3
If the function defined by y=f(x) is continuous on the closed, bounded interval a≤x≤b,
then it attains both a maximum and minimum value on this interval.
Max Min
Note that this theorem only tells us that the extreme values exist, not what they are. Going back to Figure 2,
point 𝐶 and 𝐸 are referred to as local maximums. Likewise, point 𝐵 and 𝐷 are referred to as local minimums.
⑨
cancave
What does local mean? At any point at which the function has a peak, it has a local maximum, and at any point
at which it has a valley it has a local minimum.
· canvex
5
Extrema of a Function
Max , Min
There is a simple way to determine the local extrema. We notice that at the peak, as we move from left to right,
the 𝑦-values increase and then it decreases. At the valley, the 𝑦-values decrease and then it increases. Remember
that a function may have multiple local extrema.
How do we fine the local extrema? Two things occur at the local extremum: 1) the tangent line is horizontal
(point 𝐵 and 𝐶); 2) the curve has a sharp corner where tangent line doesn’t exist (point 𝐷 and 𝐸).
Manzini = O
When the tangent line is horizontal, it means that the derivative (its slope) is zero. However, this is not enough.
There are many functions where the horizontal tangent line as a point is not the local extremum. Thus, finding
all the point where the derivative is zero (or fails to exist) is only the first step.
Any number in the domain of the function at which the derivative is either zero or fails to exist is called a
critical number. If 𝑐 is a critical number for the function defined by 𝑦 = 𝑓(𝑥) then the point 𝑐, 𝑓(𝑐) is called a
critical point. Every local extremum is a critical point, however, not every critical point is a local extremum.
6
.
1
an critical number
1 an function
25%
Extrema of a Function 0
=
3 Heimtosine
.
2 an critical point
- con critical number Georg
Example 1
Find the critical points of 𝑓 𝑥 = 𝑥 3 − 3𝑥 2 − 9𝑥 + 15
f'(X) = 3X2 bX 9
- - find y f(3) = 32 3(3)2
-
9(3) + 13
?
0 3(X 3)
=
-
2x -
= -
12
0 =
3(X + 1)(X -
3) f( x
-
= - 13 -
3) -
1)2
-
9) 1)
-
+ 15
X -
= 1 3
, =
28
Critical Point (3 -12)
, (-1 ,
20)
,
7
Extrema of a Function
Observe that the maximum and minimum values of a continuous function on a closed interval can occur either at
an endpoint of the interval or at critical point. All we have to do is compute the 𝑦-values of all of these points
and determine which is the maximum and which is the minimum.
Example 2 Domain
Find the extrema of 𝑓 𝑥 = 𝑥 3 − 3𝑥 2 − 9𝑥 + 15 on 0 ≤ 𝑥 ≤ 4
f((X) = 3X2 6X -
9 fin y
1)(X 3)
3(X 33 3(23
-
+
0 - (3) 9(3)
=
=
-
-
+ 15
X = -1 , 3 vin Domain =
-
122 Minimum point 13 , -12)
f(0) =
02 -
3(03) -
9(0) + 15
=
152 Maximum Point (0 15)
,
f(x) = k
2
-
3(23 -
9(4) + 15
=
- 52 (4 , -3)
8
The First Derivative Test
There are three possibilities to classify a critical point. Either the point is a local maximum, local minimum, or
neither. We can classify the critical point by examining the sign of the derivative.
The function is decreasing (increasing) as we approach the minimum (maximum) from the left and increasing
(decreasing) to its right.
9
The First Derivative Test
Notice that for the increasing function, each tangent line has positive slope. Every point where a function is
increasing (decreasing) and the derivative exists it is positive (negative). If a function is neither increasing nor
decreasing, then it is a constant, we know the derivative will be zero.
In any interval on which 𝑓 ′ 𝑥 > 0 the function is increasing; in any interval on which
𝑓 ′ 𝑥 < 0 the function is decreasing; in any interval on which 𝑓 ′ 𝑥 = 0 the function
is a constant
This means that if 𝑀 is a local maximum (minimum), the derivative of the function is positive (negative) just to
its left and negative (positive) just to its right. This is called the First Derivative Test (or First Order Condition).
10
The First Derivative Test
Example
Find and classify the critical point(s) of 𝑓 𝑥 = 𝑥 2 − 6𝑥
find order derivative point minimum point
f'(X) 2x 6 0
32 6(3)
=
f(3)
- =
= -
X =
3
-
=
G
(3 ,
-
9)
je 201
↑(2) =
-
2 F'(4) = 2
↓
Min
11
Economic and Finance Application
Suppose that the relationship between price and demand for a certain brand of computer is given by the equation
𝑝 = −3𝑥 + 1200, where 𝑝 is the price of a set in dollars and 𝑥 is the number of sets demanded. What price
should be charged per set if the total revenue is to be maximized.
0 X R = p. X 0 3(200) + 1 , 200 600$
R .
-
=
= =
R =
( 3X + 1200)X & =
-
3(2003 + 1200/200) = 120 , 000
R =
-
3x2 + 1200X
0 XX180
& (x) = 6X + 1208
↑r
-
200 200
X = ,
R(199) >O R'(201)/8
12
13
Concavity and the Second Derivative
Think about the way a curve looks as the function nears its minimum. At first, it may be decreasing very quickly
But, as it nears the low point, its rate of decrease must slow until it stops decreasing altogether at the minimum
(to zero). After passing the minimum, the function must begin to increase; first slowly and then more rapidly.
14
Concavity and the Second Derivative
What does this mean? Its rate of change is itself changing. In order to analyze this, we need higher order
derivatives.
Suppose 𝑦 = 𝑓 𝑥 = 4𝑥 3 − 2𝑥 2 − 7𝑥 + 9
We know that 𝑓′ 𝑥 = 12𝑥 2 − 4𝑥 − 7…the derivative is itself a function of 𝑥
In order to determine the rate of change of this tangent line (which is a derivative of the original function), we
need to take a derivative of a derivative…this is called the second derivative and is represented by
𝑑2 𝑦
𝑓 ′′ 𝑥 or .
𝑑𝑥 2
𝑑
12𝑥 2 − 4𝑥 − 7 = 24𝑥 − 4
𝑑𝑥
15
Concavity and the Second Derivative
Example 1
Find the second derivative of 𝑓 𝑥 = (𝑥 2 + 1)1/2
f'(x) = 1(X2 + 152 2X .
f"(x) = (x2 + 1) 2 + Xo -
1(X2 + 1)-2 .
2X
X
-"(x) = (x2 1
+
2 + X2 .
-
(x2 + 1
16
17
Concavity and the Second Derivative
The second derivative can help us determine concavity or convexity. Consider the convex function whose graph
is given in Figure 4a. Notice that as we follow the graph from left to right the slope of the tangent lines gets
larger. Thus, the slope (derivative) is an increasing function which means that the derivative of the derivative
(second derivative) must be positive 𝑓 ′′ 𝑥 > 0.
18
Concavity and the Second Derivative
Similarly, a concave function will see the slope decreasing and second derivative must be negative 𝑓 ′′ 𝑥 < 0 as
in Figure 4b. This is the Second Derivative Test (or Second Order Condition)
In any interval on which 𝑓 ′′ 𝑥 > 0, the function is convex. In any interval on which
𝑓 ′′ 𝑥 < 0 the function is concave.
The second derivative can be used to determine local minimum or local maximum
19
Concavity and the Second Derivative
Example 2
Use the Second Derivative Test to classify the critical points of 𝑓 𝑥 = 𝑥 3 − 3𝑥 2 − 9𝑥 + 15
local minimum
f(x) = 3X2-6X-9 (3 ,
-12)
3 1
X
-
= ,
f"(X) =
6X -
6
f"(3) = 18 -
6
Min
f"(3) = 12
6
f"( 6
-
-
1) =
-
= - 12 -
Max
20
21
Economic and Finance Application
Suppose that the price and demand for a particular luxury automobile are related by the demand equation 𝑝 +
10𝑥 = 200,000; where 𝑝 is the price per car in dollars and 𝑥 is the number of cars that will be purchased at that
price. What price should be charged per car if the total revenue is to be maximized?
& = XP
& = -
18X2+ 200 , 000X
R (x) =
-
20 X + 200 ,
000
"
& (x) = -
20 Maximum
↑ = - 10(10 , 000) + 200 , 000
P = 100 ,
000 $
22
23
Optimizing Functions of Two Variables
Suppose Samsung produces two models of smartphones, the Galaxy S Series and the Galaxy Note Series, and
that the total cost of producing 𝑥 units of the Galaxy S Series and 𝑦 units of the Galaxy Note Series is given by
the function 𝐶(𝑥, 𝑦). How would you find the level of production 𝑥 = 𝑎 and 𝑦 = 𝑏 that results in minimal cost?
The difference between this section and the previous section is that we must determine the optima for two
variables rather than one.
The function 𝑓(𝑥, 𝑦) is said to have a local maximum at the point (𝑎, 𝑏) in the domain of 𝑓 if 𝑓(𝑎, 𝑏) ≥ 𝑓(𝑥, 𝑦)
for all points (𝑥, 𝑦) in a circular disk centered at (𝑎, 𝑏).
The function 𝑓(𝑥, 𝑦) is said to have a local minimum at the point (𝑐, 𝑑) in the domain of 𝑓 if 𝑓(𝑐, 𝑑) ≤ 𝑓(𝑥, 𝑦)
for all points (𝑥, 𝑦) in a circular disk centered at (𝑐, 𝑑).
24
Optimizing Functions of Two Variables
There is a local maximum of 𝑓(𝑥, 𝑦) at (𝑎, 𝑏) if the surface 𝑧 = 𝑓(𝑥, 𝑦) has a peak at the point
(𝑎, 𝑏, 𝑓 𝑎, 𝑏 )…which means that (𝑎, 𝑏, 𝑓 𝑎, 𝑏 ) is at least as high as the nearby point on the surface.
25
Optimizing Functions of Two Variables
Suppose 𝑓(𝑥, 𝑦) has a local maximum at 𝑎, 𝑏 . Similar to optimizing a single variable, the critical point is
when both 𝑓𝑥 𝑎, 𝑏 = 0 and 𝑓𝑦 𝑎, 𝑏 = 0. raise o
26
Optimizing Functions of Two Variables
Although all the local extrema of a function must occur at critical points, not every critical point of a function
corresponds to a relative extremum…take 𝑓 𝑥, 𝑦 = 𝑦 2 − 𝑥 2
f(X y) ,
= -
2X = 0
Fy(X , y) =
2y = 0
f x
(0 ,
b) -
fy)0 ,
0) = 0
27
Optimizing Functions of Two Variables
It turns out that when we graph this function it looks like this:
28
Optimizing Functions of Two Variables
The surface 𝑧 = 𝑦 2 − 𝑥 2 has a local
maximum in the 𝑥 direction and a local
minimum in the 𝑦 direction. Instead of having
a peak or valley above the critical point 0,0 ,
the surface is shaped like a saddle…this is
what is called a saddle surface.
For a critical point to correspond to a local
extremum, the same extreme behavior
(maximum or minimum) must occur in all
directions (which is not the case here). Any
critical point (like the origin in this example)
where there is a relative maximum in one
direction and a relative minimum in another
direction is called a saddle point.
29
The Second Partial Test
So how do we know if a given critical point is a local extremum (maximum or minimum) or saddle point?
Luckily there is a procedure involving second-order partial derivatives that you can use.
Let 𝑓(𝑥, 𝑦) be a function of 𝑥 and 𝑦 whose partial derivatives 𝑓𝑥 , 𝑓𝑦 , 𝑓𝑥𝑥 , 𝑓𝑦𝑦 , and 𝑓𝑥𝑦 all exist, and let 𝐷(𝑥, 𝑦)
be the function:
30
31
The Second Partial Test
Example
Go back to 𝑓 𝑥, 𝑦 = 𝑦 2 − 𝑥 2 …we saw that there is a saddle point, use the second partial test to see if this is
indeed true
32
33
Economic and Finance Application:
Maximizing Profit
Ajarn Chanon manages a grocery store that carries two brands of cat food, a local brand obtained at the cost of
30 cents per can and a well-known national brand obtained for 40 cents per can. He estimates that if the local
brand is sold for 𝑥 cents per can and the national brand for 𝑦 cents per can, then approximately 70 − 5𝑥 + 4𝑦
cans of the local brand and 80 + 6𝑥 − 7𝑦 of the national brand will be sold each day. How should Ajarn Chanon
price each brand to maximize total daily profit from the sale of cat food?
Total Profit = Profit from Local Brand + Profit from National Brand
Total Profit = (Local Brand Sold)(Profit per Item) + (National Brand Sold)(Profit per Item)
f(x y)
,
= (10 -
3x + xy)(X 3) -
+ (80 + 6X -
y)(y 40)
+ -
=
nox 210 3x3+ 13X + 4xy 1y2 +
12xy
-
8y 24X
- -
+ 320 + bxy 2804
-
- -
34
35
36
37
