International Series in Operations Research

& Management Science

Volume 149

Series Editor
Frederick S. Hillier
Stanford University, CA, USA

Special Editorial Consultant

Camille C. Price
Stephen F. Austin State University, TX, USA

For further volumes:

http://www.springer.com/series/6161
Eric V. Denardo

Linear Programming and

Generalizations
A Problem-based Introduction with
Spreadsheets

1  3
Eric V. Denardo
Yale University
P.O. Box 208267
New Haven CT 06520-8267
USA
[email protected]

Additional material to this book can be downloaded from http://extra.springer.com.

ISSN 0884-8289
ISBN 978-1-4419-6490-8â•…â•…â•…â•… e-ISBN 978-1-4419-6491-5
DOI 10.1007/978-1-4419-6491-5
Springer New York Dordrecht Heidelberg London

Library of Congress Control Number: 2011920997

© Springer Science+Business Media, LLC 2011

All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY
10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connec-
tion with any form of information storage and retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are
not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject
to proprietary rights.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

 Preface

The title of this book adheres to a well-established tradition, but “linear

programming and generalizations” might be less descriptive than “models of
constrained optimization.” This book surveys models that optimize some-
thing, subject to constraints. The simplest such models are linear, and the
ideas used to analyze linear models generalize easily.

Over the past half century, dozens of excellent books have appeared on
this subject. Why another? This book fuses five components:

• It uses examples to introduce general ideas.

• It engages the student in spreadsheet computation.

• It surveys the uses of constrained optimization.

• It presents the mathematics that relates to constrained optimization.

• It links the subject to economic reasoning.

Each of these components can be found in other books. Their fusion

makes constrained optimization more accessible and more valuable. It stimu-
lates the student’s interest, it quickens the learning process, it helps students
to achieve mastery, and it prepares them to make effective use of the material.

A well-designed example provides context. It can illustrate the applicabil-

ity of the model. It can reveal a concept that holds in general. It can introduce
the notation that will be needed for a more general discussion.

Examples mesh naturally with spreadsheet computation. To compute on

a spreadsheet is to learn interactively – the spreadsheet gives instant feedback.
Spreadsheet computation also takes advantage of the revolution that has oc-
curred in computer hardware and software. Decades ago, constrained optimi-
zation required specialized knowledge and access to huge computers. It was
a subject for experts. That is no longer the case. Constrained optimization

v
vi Linear Programming and Generalizations

has become vastly easier to learn and to use. Spreadsheets help the student to
become facile with the subject, and it helps them use it to shape their profes-
sional identities.

Constrained optimization draws upon several branches of mathemat-

ics. Linear programming builds upon linear algebra. Its generalizations draw
upon analysis, differential calculus, and convexity. Including the relevant
math in a course on constrained optimization helps the student to master the
math and to use it effectively.

Nearly every facet of constrained optimization has a close link to eco-

nomic reasoning. I cite two examples, among many: A central theme of eco-
nomics is the efficient allocation of scarce resources, and the canonical model
for allocating scarce resources is the linear program. Marginal analysis is a
key concept in economics, and it is exactly what the simplex method accom-
plishes. Emphasizing the links between constrained optimization and eco-
nomics makes both subjects more comprehensible, and more germane.

The scope of this book reflects its components. Spreadsheet computation

is used throughout as a teaching-and-learning aide. Uses of constrained opti-
mization are surveyed. The theory is dovetailed with the relevant mathemat-
ics. The links to economics are emphasized.

The book is designed for use in courses that focus on the applications of
constrained optimization, in courses that emphasize the theory, and in cours-
es that link the subject to economics. A “use’s guide” is provided; it takes the
form of a brief preview of each of the six Parts that comprise this book.
 Acknowledgement

This book’s style and content have been shaped by decades of interac-
tion with Yale students. Their insights, reactions and critiques have led me
toward a problem-based approach to teaching and writing. With enthusiasm,
I acknowledge their contribution. This book also benefits from interactions
with my colleagues on the faculty. I am deeply indebted to Uriel G. Rothblum,
Kurt Anstreicher, Ludo Van der Heyden, Harvey M. Wagner, Arthur J. Swer-
sey, Herbert E. Scarf and Donald J. Brown, whose influences are evident here.

vii
 Contents

Part I – Prelude

Chapter 1. Introduction to Linear Programs����������������������������������尓�������╅╇ 3

Chapter 2. Spreadsheet Computation����������������������������������尓�������������������╅ 33

Chapter 3. Mathematical Preliminaries����������������������������������尓����������������╅ 67

Part II – The Basics

Chapter 4. The Simplex Method, Part 1����������������������������������尓���������������╇ 113

Chapter 5. Analyzing Linear Programs����������������������������������尓����������������╇ 153

Chapter 6. The Simplex Method, Part 2����������������������������������尓���������������╇ 195

Part III – Selected Applications

Chapter 7. A Survey of Optimization Problems����������������������������������尓��╇ 221

Chapter 8. Path Length Problems and Dynamic Programming���������╇ 269

Chapter 9. Flows in Networks����������������������������������尓��������������������������������╇ 297

Part IV – LP Theory

Chapter 10. Vector Spaces and Linear Programs����������������������������������尓╇ 331

Chapter 11. Multipliers and the Simplex Method���������������������������������╇ 355

Chapter 12. Duality����������������������������������尓������������������������������������尓�������������╇ 377

Chapter 13. The Dual Simplex Pivot and Its Uses���������������������������������╇ 413

ix
x Linear Programming and Generalizations

Part V – Game Theory

Chapter 14. Introduction to Game Theory����������������������������������尓����������╇ 445

Chapter 15. A Bi-Matrix Game����������������������������������尓�����������������������������╇ 479

Chapter 16. Fixed Points and Equilibria����������������������������������尓��������������╇ 507

Part VI – Nonlinear Optimization

Chapter 17. Convex Sets����������������������������������尓������������������������������������尓�����╇ 545

Chapter 18. Differentiation����������������������������������尓������������������������������������尓╇ 565

Chapter 19. Convex Functions����������������������������������尓������������������������������╇ 581

Chapter 20. Nonlinear Programs����������������������������������尓��������������������������╇ 617

 Part I–Prelude

This book introduces you, the reader, to constrained optimization. This

subject consists primarily of linear programs, their generalizations, and their
uses. Part I prepares you for what is coming.

Chapter 1. Introduction to Linear Programs

In this chapter, a linear program is described, and a simple linear program

is solved graphically. Glimpses are provided of the uses to which linear pro-
grams can be put. The limitations that seem to be inherent in linear programs
are identified, each with a pointer to the place in this book where it is skirted.

Chapter 2. Spreadsheet Computation

Chapter 2 contains the facets of Excel that are used in this book. Also
discussed in Chapter 2 is the software that accompanies this text. All of the
information in it is helpful, and some of it is vital.

Chapter 3. Mathematical Preliminaries

Presented in Chapter 3 is the mathematics on which an introductory ac-

count of linear programming rests. A familiar method for solving a system of
linear equations is described as a sequence of “pivots.” An Excel Add-In can
be used to execute these pivots.
Chapter 1: Introduction to Linear Programs

1.╅ Preview ����������������������������������尓������������������������������������尓�������������������������� 3

2.╅ An Example . ����������������������������������尓������������������������������������尓�������������� 4
3.╅ Generalizations . ����������������������������������尓������������������������������������尓������ 10
4.╅ Linearization ����������������������������������尓������������������������������������尓�������������� 12
5.╅ Themes ����������������������������������尓������������������������������������尓������������������������ 21
6.╅ Software ����������������������������������尓������������������������������������尓���������������������� 24
7.╅ The Beginnings ����������������������������������尓������������������������������������尓���������� 25
8.╅ Review ����������������������������������尓������������������������������������尓������������������������ 28
9.╅ Homework and Discussion Problems ����������������������������������尓������������ 30

1.  Preview

The goals of this chapter are to introduce you to linear programming and
its generalizations and to preview what’s coming. The chapter itself is orga-
nized into six main sections:

• In the first of these sections, the terminology that describes linear pro-
grams is introduced and a simple linear program is solved graphically.

• In the next section, several limitations of linear programs are discussed,

and pointers are provided to places in this book where these limitations
are skirted.

• The third section describes optimization problems that seem not to be

linear programs, but can be converted into linear programs.

• The fourth section introduces four themes that pervade this book.

E. V. Denardo, Linear Programming and Generalizations, International Series 3

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_1, © Springer Science+Business Media, LLC 2011
4 Linear Programming and Generalizations

• The fifth section introduces the computer codes that are used in this
text.

• The sixth section consists of a brief account of the origins of the field.

Linear programming and its generalizations is a broad subject. It has a

wide variety of uses. It has links to several academic fields. It is united by
themes that are introduced here and are developed in later chapters.

2.  An Example

A “linear program” is a disarmingly simple object. Its definition entails

the terms, “linear expression” and “linear constraint.” A linear expression ap-
pears below; its variables are x, y and z, and the dependence of this expression
on x, y and z is linear.
√
3 x3x− –2.5
2.5y
y ++ 2 zz ≤ 6 ,
− 5 y inequality
A linear constraint requires2axlinear + z = 3 , to take any one of the
three forms that are illustrated below: x≥0.
√
3 x3x
− –2.5
2.5y
y ++ 2 â•›zz ≤≤6,6 ,
╅╇ 2 x2x−–55y
y++ zz ==3,3 ,
╅╅╛╛xx ≥≥0.0 .
In other words, a linear constraint requires a linear expression to be less
than or equal to a number, to be equal to a number, or to be greater than or
equal to a number. The linear constraint xâ•›≥â•›0 requires the number x to be
nonnegative, for instance.

A linear program either maximizes or minimizes a linear expression

subject to finitely many linear constraints. An example of a linear program is:

Program 1.1.╇ z*╛=╛Maximize {2x╛+╛ 2y} subject to the constraints

1x + 2y ≤ 4,

3x + 2y ≤ 6,
x â•› ≥ â•›0,
╅╅╇ y ≥ 0.
 Chapter 1: Eric V. Denardo 5

The decision variables in a linear program are the quantities whose val-
ues are to be determined. Program 1.1 has two decision variables, which are
x and y. Program 1.1 has four constraints, each of which is a linear inequality.

A big deal?

A linear program seems rather simple. Can something this simple be im-
portant? Yes! Listed below are three reasons why this is so.

• A staggeringly diverse array of problems can be posed as linear pro-

grams.

• A family of algorithms that are known as the simplex method solves

nearly all linear programs with blinding speed.

• The ideas that underlie the simplex method generalize readily to situa-
tions that are far from linear and to settings that entail several decision
makers, rather than one.

Linear programming describes the family of mathematical tools that

are used to analyze linear programs. In tandem with the digital computer,
linear programming has made mathematics vastly more useful. Linear pro-
gramming also provides insight into a number of academic disciplines, which
include mathematics, economics, computer science, engineering, and opera-
tions research. These insights are glimpsed in this chapter and are developed
in later chapters.

Feasible solutions

Like any field, linear programming has its own specialized terminology
(jargon). Most of these terms are easy to remember because they are sug-
gested by normal English usage. A feasible solution to a linear program is
a set of values of its decision variables that satisfies each of its constraints.
Program 1.1 has many feasible solutions, one of which xâ•›=â•›1 and yâ•›=â•›0. The
feasible region of a linear program is its set of feasible solutions. Program 1.1
has only two decision variables, so its feasible region can be represented on
the plane. Figure 1.1 does so.
6 Linear Programming and Generalizations

Figure 1.1.   Feasible region for Program 1.1.

y
3

1x + 2y = 4
2

3x + 2y = 6

1
feasible region
x=0
0 x
0 1 2 3 4
y=0

Figure 1.1 is easy to construct because the pairs (x, y) that satisfy a par-
ticular linear constraint form a “half-plane” whose boundary is the line on
which this constraint holds as an equation. For example:

• The constraint 1xâ•›+â•›2yâ•›≤â•›4 is satisfied as an equation by the pairs (x, y)

on the line 1xâ•›+â•›2yâ•›=â•›4.

• Two points determine a line, and the line 1xâ•›+â•›2yâ•›=â•›4 includes the points
(pairs) (0, 2) and (4, 0).

• Since (0, 0) satisfies the constraint 1xâ•›+â•›2yâ•›≤â•›4 as a strict inequality, this

constraint is satisfied by the half plane in which (0, 0) lies.

• In Figure 1.1, a thick arrow points from the line 1xâ•›+â•›2yâ•›=â•›4 into the half-
plane that satisfies the inequality 1xâ•›+â•›2yâ•›≤â•›4.

The feasible region for Program 1.1 is the intersection of four half-planes,
one per constraint. In Figure 1.1, the feasible region is the area into which the
thick arrows point, and it is shaded.
 Chapter 1: Eric V. Denardo 7

Optimal solutions

Each feasible solution assigns an objective value to the quantity that is

being maximized or minimized. The feasible solution xâ•›=â•›1, yâ•›=â•›0 has 2 as its
objective value, for instance. An optimal solution to a linear program is a
feasible solution whose objective value is largest in the case of a maximization
problem, smallest in the case of a minimization problem. The optimal value
of a linear program is the objective value of an optimal solution to it.

An optimal solution to Program 1.1 is xâ•›=â•›1 and yâ•›=â•›1.5, and its optimal
value z*â•›=â•›2xâ•›+â•›2yâ•›=â•›(2)(1)â•›+â•›(2)(1.5)â•›=â•›5. To convince yourself that this is the
optimal solution to Program 1.1, consider Figure 1.2. It augments Figure 1.1
by including two “iso-profit” lines, each of which is dashed. One of these lines
contains the points (x, y) whose objective value equals 4, the other contains
the pairs (x, y) whose objective value that equals 5. It is clear, visually, that the
unique optimal solution to Program 1.1 has xâ•›=â•›1 and yâ•›=â•›1.5.

Figure 1.2.↜  Feasible region for Program 1.1, with two iso-profit lines.

y
3

2x + 2y = 4
2 (1, 1.5)

2x + 2y = 5
1
feasible region

0 x
0 1 2 3 4

A linear program can have only one optimal value, but it can have more
than one optimal solution. If the objective of Program 1.1 were to maximize
(xâ•›+â•›2y), its optimal value would be 4, and every point on the line segment
connecting (0, 2) and (1, 1.5) would be optimal.
8 Linear Programming and Generalizations

A taxonomy

Linear programs divide themselves into categories. A linear program is

feasible if it has at least one feasible solution, and it is said to be infeasible
if it has no feasible solution. Program 1.1 is feasible, but it would become in-
feasible if the constraint xâ•›+â•›yâ•›≥â•›3 were added to it. Infeasible linear programs
do arise in practice. They model situations that are so tightly restricted as to
have no solution.

A linear program is said to be unbounded if it is feasible and if the objec-

tive value of its feasible solutions can be improved without limit. An example
of an unbounded linear program is:

Max {x}, subject to xâ•›≥â•›2.

An unbounded linear program is almost invariably a signal of an incor-

rect formulation: it is virtually never possible to obtain an infinite amount of
anything that is worthwhile.

A linear program is feasible and bounded if it is feasible and if its objec-

tive cannot be improved without limit. Highlighted below is a property of
linear programs that are feasible and bounded:

Each linear program that is feasible and bounded has at least one opti-
mal solution.

This property is not quite self-evident. It should be proved. The simplex

method will provide a proof.

Each linear program falls into one of these three categories:

• The linear program may be infeasible.

• It may be feasible and bounded.

• It may be unbounded.

To solve a linear program is to determine which of these three categories

it lies in and, if it is feasible and bounded, to find an optimal solution to it.
 Chapter 1: Eric V. Denardo 9

Bounded feasible regions

A linear program is said to have a bounded feasible region if some num-

ber K exists such that each feasible solution equates every decision variable to
a number whose absolute value does not exceed K. Program 1.1 has a bound-
ed feasible region because each feasible solution equates each decision vari-
able to a number between 0 and 2.

If a linear program is unbounded, it must have an unbounded feasible

region. The converse is not true, however. A linear program that has an un-
bounded feasible region can be feasible and bounded. To see that this is so,
consider Program 1.2.

Program 1.2.  z* =â•›Minimize {4uâ•›+ 6v} subject to the constraints

╅╛╛╛╅╇╛1u + 3v ≥ 2,
╛╛2u + 2v ≥ 2,
╇╛╛u â•› ≥ 0,
╅╅╇╛╛v ≥ 0.

Figure 1.3 plots the feasible region for Program 1.2. This feasible region is
clearly unbounded. Program 1.2 is bounded, nonetheless; every feasible solu-
tion has objective value that exceeds 0.

Figure 1.3.↜  Feasible region for Program 1.2.

v
2u + 2v = 2

1
feasible region
1u + 3v = 2

(1/2, 1/2)
0 u
0 1 2

You might suspect that unbounded feasible reasons do not arise in prac-
tice, but that is not quite accurate. In a later chapter, we’ll see that every linear
program is paired with another, which is called its “dual.” We will see that if a
10 Linear Programming and Generalizations

linear program is feasible and bounded, then so is its dual, in which case both
linear programs have the same optimal value, and at least one of them has an
unbounded feasible region. Programs 1.1 and 1.2 are each other’s duals, by
the way. One of their feasible regions is unbounded, as must be.

3.  Generalizations

A linear program is an optimization problem that fits a particular for-

mat: A linear expression is maximized or minimized subject to finitely many
linear constraints. Discussed in this section are the limitations imposed by
this format, along with the parts of this book where most of them are cir-
cumvented.

Constraints that hold strictly

A linear program requires each constraints to take one of three forms; a

linear expression can be “≥” a number, it can be “=” a number, or it can be “≤”
a number. Strict inequalities are not allowed. One reason why is illustrated by
this optimization problem:

Minimize {3y}, subject to yâ•›>â•›2.

This problem does not have an optimal solution. The “infimum” of its
objective equals 6, and setting y slightly above 2 comes “close” to 6, but an
objective value of 6 is not achievable. Ruling out strict inequalities eliminates
this difficulty.

On the other hand, the simplex method can – and will – be used to find
solutions to linear systems that include one or more strict inequalities. To
illustrate, suppose a feasible solution to Program 1.1 is sought for which the
variables x and y are positive. To construct one, use the linear program:

Maximize θ, subject to the constraints of Program 1.1 and

θ ≤ x,

θ ≤ y.

In Chapter 12, strict inequalities emerge in a second way, as a facet of a

subject called “duality.”
 Chapter 1: Eric V. Denardo 11

Integer-valued variables

A linear program lets us impose constraints that require the decision vari-
able x to lie between 0 and 1, inclusive. On the other hand, linear programs
do not allow us to impose the constraint that restrict a decision variable x to
the values 0 and 1. This would seem to be a major restriction. Lots of entities
(people, airplanes, and so forth) are integer-valued.

An integer program is an optimization problem that would become a

linear program if we suppressed the requirement that its decision variables be
integer-valued. The simplex method is so fast that it is used in as a subroutine
in algorithms that solve integer programs. How that occurs is described in
Chapter 13.

In addition, an important class of integer programs can be solved by a

single application of the simplex method. That’s because applying the sim-
plex method to these integer programs can be guaranteed to produce an
optimal solution that is integer-valued. These integer programs are “net-
work flow” models whose data are “integer-valued.” They are studied in
Chapter 9.

Competition

A linear program models a situation in which a single decision maker

strives to select the course of action that maximizes the benefit received. At
first glance, the subject seems to have nothing to do with game theory, that
is, with models of situations in which multiple decision makers can elect to
cooperate or compete. But it does! Chapters 14, 15 and 16 of this book adapt
the ideas and algorithms of linear programming to models of competitive
behavior.

Non-linear functions

Linear programs require that the objective and constraints have a par-
ticular form, that they be linear. A nonlinear program is an optimization
problem whose objective and/or constraints are described by functions
that fail to be linear. The ideas used to solve linear programs generalize
to handle a variety of nonlinear programs. How that occurs is probed in
Chapter 20.
12 Linear Programming and Generalizations

4.  Linearization

Surveyed in this section are some optimization problems that do not

present themselves as linear programs but that can be converted into linear
programs.

A “maximin” objective

Suppose we wish to find a solution to a set of linear constraints that maxi-

mizes the smaller of two measures of benefit, for instance, to solve:

Program 1.3.╇ z*â•›=â•›Maximize the smaller of (2xâ•›+â•›2y) and (xâ•›−â•›3y), subject to

â•…â•… 1x + 2y ≤ 4,
â•…â•… 3x + 2y ≤ 6,
â•…â•… x â•›≥ 0,
╅╅╅╅╇ y ≥ 0.

The object of Program 1.3 is to maximize the smaller of two linear expres-
sions. This is not a linear program because its objective is not a linear expres-
sion. To convert Program 1.3 into an equivalent linear program, we maximize
the quantity t subject to constraints that keep t from exceeding the linear ex-
pressions (2xâ•›+â•›2y) and (xâ•›−â•›3y). In other words, we replace Program 1.3 by

Program 1.3´.╇ z*â•›=â•›Maximize {t}, subject to

t ≤ 2x + 2y,
t ≤ 1x – 3y,
1x + 2y ≤ 4,
3x + 2y ≤ 6,
╇ x ╇╅ ≥ 0,
╅╅╇ y ≥ 0.
Program 1.3´ picks the feasible solution to Program 1.3 that maximizes
the smaller of the linear expressions 2xâ•›+â•›2y and 1xâ•›–â•›3y, exactly as desired.

A “minimax” objective

Suppose we wish to find a solution to a set of linear constraints that mini-

mizes the larger of two linear expressions, e.g., that minimizes the larger of
 Chapter 1: Eric V. Denardo 13

(2xâ•›+â•›2y) and (xâ•›−â•›3y), subject to the constraints of Program 1.3. The same
trick works, as is suggested by:

Program 1.4.╇ Minimize {t}, subject to

t ≥ 2x + 2y,

t ≥ 1x – 3y,

and the constraints of Program 1.3.

Evidently, it is easy to recast a “maximin” or a “minimax” objective in

the format of a linear program. This conversion enhances the utility of linear
programs. Its role in John von Neumann’s celebrated minimax theorem is
discussed in Chapter 14.

“Maximax” and “minimin” objectives?

Suppose we seek to maximize the larger of the linear expressions (2xâ•›+â•›2y)

and (1xâ•›−â•›3y), subject to the constraints of Program 1.3. It does not suffice to
maximize {t}, subject to the original constraints and tâ•›≥â•›2xâ•›+â•›2y and tâ•›≥â•›1xâ•›−â•›3y.
This linear program is unbounded; t can be made arbitrarily large. For the
same reason, we cannot use a linear program to minimize the smaller of two
linear expressions.

The problem of maximizing the larger of two or more linear expressions

can be posed as an integer program, as can the problem of minimizing the
smaller of two or and more linear expressions. How to do this will be illus-
trated in Chapter 7.

Decreasing marginal benefit

A linear program seems to require that the objective vary linearly with
the level of a decision variable. In Program 1.1, the objective is to maximize
the linear expression, 2xâ•›+â•›2y. Let us replace the addend 2y in this objective by
the (nonlinear) function p(y) that is exhibited in Figure 1.4. This function il-
lustrates the case of decreasing marginal benefit, in which the (profit) func-
tion p(y) has slope that decreases as the quantity y increases.
14 Linear Programming and Generalizations

Figure 1.4.   A function p(y) that illustrates decreasing marginal profit.

S\

 VORSHHTXDOV

VORSHHTXDOV

 \
    


Decreasing marginal benefit occurs when production above a certain

level requires extra expense, for instance, by the use of overtime labor. The
profit function p(y) in Figure 1.4 can be accommodated by introducing two
new decision variables, y1 and y2, along with the constraints

y1 ≥ 0, y1 ≤ 0.75, y2 ≥ 0, y = y1 + y2,

and replacing the addend 2y in the objective by (2y1â•›+â•›0.25y2). This results in:

Program 1.5.╇ z*╛=╛Maximize {2x + 2y1 + 0.25y2} subject to the constraints

1x + 2y ≤ 4,

3x + 2y ≤ 6,
y = y1 + y2,
â•›y1 ≤ 0.75,
x ≥ 0, y ≥ 0, y1 ≥ 0, y2 ≥ 0.

To verify that Program 1.5 accounts correctly for the profit function p(y)
in Figure  1.4, we consider two cases. First, if the total quantity y does not
exceed 0.75, it is optimal to set y1â•›=â•›y and y2â•›=â•›0. Second, if the total quantity y
does exceed 0.75, it is optimal to set y1â•›=â•›0.75 and y2â•›=â•›yâ•›−â•›0.75.
 Chapter 1: Eric V. Denardo 15

An unintended option

Program 1.5 is a bit more subtle than it might seem. Its constraints allow
an unintended option, which is to set y2â•›>â•›0 while y1â•›<â•›1. This option is ruled
out by optimization, however. In this case and in general:

Linear models of decreasing marginal benefit introduce unintended

options that are ruled out by optimization.

The point is that a linear program will not engage in a more costly way to
do something if a less expensive method of doing the same thing is available.

Increasing marginal cost

Net profit is the negative of net cost: A net profit of $6.29 is identical to a
net cost of −$6.29, for instance. Maximizing net profit is precisely equivalent
to minimizing net cost. Because of this, the same trick that handles the case
of decreasing marginal profit also handles the case of increasing marginal
cost. One or more unintended options are introduced, but they are ruled out
by optimization. Again, the more costly way of doing something is avoided.

Increasing marginal return?

A profit function exhibits increasing marginal return if its slope increas-

es with quantity. One such function is exhibited in Figure 1.5. Its slope equals
1/2 for quantities below 1 and equals 2 for quantities above 1.

Figure 1.5.↜  A profit function that exhibits increasing marginal return.

T\

VORSH
HTXDOV
 VORSHHTXDOV

 \
    
16 Linear Programming and Generalizations

Let us turn our attention to the variant of Program 1.1 whose object is to
maximize the nonlinear expression, {2xâ•›+â•›q(y)}. Proceeding as before would
lead to:

Program 1.6.╇ z*╛=╛Maximize {2x╛+╛0.5y1╛+╛2y2} subject to the constraints

1x + 2y ≤ 4,
3x + 2y ≤ 6,
╅╅╇ y = y1 + y2,
╅╅╇ y1 ≤ 1,
x ≥ 0, y ≥ 0, y1 ≥ 0, y2 ≥ 0.
Program 1.6 introduces an unintended option, which is to set y2 posi-
tive while y1 is below 1, and this option is selected by optimization. Indeed,
in Program 1.6, it cannot be optimal to set y1 positive. Given the option, the
linear program chooses the more profitable way to do something. In this case
and in general:

Linear models of increasing marginal return introduce unintended

options that are selected by optimization.

Increasing marginal return (equivalently, decreasing marginal cost) can-

not be handled by a linear program. It requires an integer program. Chapter 7
includes a discussion of how to use binary variables (whose values are either
0 or 1) to handle increasing marginal return.
Absolute value in the performance measure
Our attention now turns to an optimization whose constraints are linear
but whose objective weighs the absolute value of one or more linear expres-
sions. To illustrate, let a and b be fixed positive numbers, and consider:
Program 1.7.╇ Minimize {a|x − 1| + b|y − 2|}, subject to the constraints of
Program 1.1.
Program 1.7 is easily converted into an equivalent linear program. To do
so, we introduce two new decision variables, t and u, and consider:
Program 1.7´.╇ Minimize {atâ•›+â•›bu}, subject to the constraints of Program 1.1
and

(1) x – 1 ≤ t, – t ≤ x – 1, y – 2 ≤ u, – u ≤ y – 2.
 Chapter 1: Eric V. Denardo 17

The decision variables t and u appear in no constraints other than (1). To

see what value is taken by the decision variable t, we consider two cases:

• If x exceeds 1, the first two constraints in (1) are satisfied by any value
of t that has tâ•›≥â•›xâ•›−â•›1, and the fact that a is positive guarantees that the
objective is minimized by setting tâ•›=â•›xâ•›−â•›1.

• If 1 exceeds x, the first two constraints in (1) are satisfied by any value
of t that has tâ•›≥â•›1â•›−â•›x, and the fact that a is positive guarantees that the
objective is minimized by setting tâ•›=â•›1â•›−â•›x.

A similar observation applies to y. Programs 1.7 and 1.7´ have the same
optimal value, and the optimal solution to Program 1.7´ specifies values of x
and y that are optimal for Program 1.7.

An alternative to least-squares regression

To illustrate a least-squares regression model, we again let a and b be

fixed positive numbers (data), and consider:

Program 1.8.╇ Minimize {a(x − 1)2 + b(y − 2)2 }, subject to the constraints
in Program 1.1.

By squaring the difference, these models place higher weights on obser-

vations that are further from the norm, e.g., on outliers. Should you wish to
weigh the observations proportionally to their distance from the norm, sub-
stitute the criterion

{a|x − 1| + b|y − 2|},

and convert the model to a linear program, exactly as was done for
Program 1.7.

An alternative to variance minimization

The justly-famous Markowitz model of portfolio theory allocates a bud-

get among investments so as to minimize the variance of the return, subject
to the constraint that the expectation of the return is at least as large as some
preset value. This optimization takes the form of Program 1.8 with a and
b being nonnegative numbers that sum to 1. This model is an easily-solved
nonlinear program.
18 Linear Programming and Generalizations

On the other hand, the variance squares the difference between the out-
come and its expectation, and it weighs upside and downside differences
equally. Substituting a “mean absolute deviation” for the variance produces
a linear program that may make better sense. Also, removing two of the
constraints in (1) minimizes the expected downside variability, which might
make still better sense.

Constraints on ratios

A ratio constraint places an upper bound or a lower bound on the ratio

of two linear expressions. To illustrate, we append to Program 1.1 the ratio
constraint.
x
≤ 0.8.
y
This constraint is not linear, so it cannot be part of a linear program.

But the other constraints in Program 1.1 guarantee yâ•›≥â•›0, and multiply-
ing an inequality by a nonnegative number preserves its sense. In particular,
multiplying the ratio constraint that is displayed above by the nonnegative
number y produces the linear constraint

x ≤ 0.8y.

This conversion must be qualified, slightly, because ratios are not defined
when their denominators equal zero. If the constraint x/yâ•›≤â•›0.8 is intended
to mean that x cannot be positive when yâ•›=â•›0, it is equivalent to xâ•›≤â•›0.8y. In
general:

Multiplying a ratio constraint by its denominator converts it to a linear

constraint if its denominator can be guaranteed to be positive.

If the denominator of a ratio is guaranteed to be nonnegative (rather than

positive), one needs to take care when it equals zero, as is suggested above.

Optimizing a ratio*

The next three subsections concern a linear program whose objective func-
tion is a ratio of linear expressions. These subsections are starred. They cover a
specialized topic that can be skipped or deferred with no loss of continuity. Read-
ers who are facile with matrix notation may wish to read them now, however.
 Chapter 1: Eric V. Denardo 19

Program 1.9, below, maximizes the ratio of two linear expressions, sub-
ject to linear constraints. Its data form the mâ•›×â•›n matrix A, the mâ•›×â•›1 vector b,
the 1â•›×â•›n vector c and the 1â•›×â•›n vector d. Its decision variables are the entries
in the nâ•›×â•›1 vector x.
 cx 
Program 1.9.╇ z* = Maximize , subject to the constraints
dx
(2) â•…â•…â•…â•…â•… â•…â•… Ax = b, x ≥ 0.

Program 1.9 will be analyzed under

Hypothesis A:╇

1.╇ Every vector x that satisfies Axâ•›=â•›b and xâ•›≥â•›0 has dxâ•›>â•›0.

2.╇No vector x satisfies Axâ•›=â•›0, xâ•›≥â•›0 and dxâ•›>â•›0.

It was A. Charnes and W. W. Cooper who showed that Program 1.9,

which they dubbed a linear fractional program, can be converted into an
equivalent linear program1.

Interpretation of Hypothesis A*

Before converting Program 1.9 into an equivalent linear program, we

pause to ask ourselves: How can we tell whether or not a particular model
satisfies Hypothesis A? A characterization of Hypothesis A appears below:

Hypothesis A is satisfied if and only if there exist positive numbers L

and U such that every feasible solution x to Program 1.9 has
Lâ•›≤â•›dxâ•›≤â•›U.

In applications, it is often evident that every vector x that satisfies (2) as-
signs a value to d x that is bounded away from 0 and from +∞.

As a point of logic, we can demonstrate that Program 1.9 is equivalent to

a linear program without verifying the characterization of Hypothesis A that
is highlighted above. And we shall.

A. Charnes and W. W. Cooper, “Programming with linear fractional functionals,”

1╇

Naval Research Logistics Quarterly, V. 9, pp.181-186, 1962.

20 Linear Programming and Generalizations

A change of variables*

A change of variables will convert Program 1.9 into an equivalent lin-

ear program. The decision variables in this equivalent linear program are the
number t and the nâ•›×â•›1 vector x̂ that will be related to x via

1
(3) t= and x̂ = xt.
dx

This change of variables converts the objective of Program 1.9 to

cxt = cx̂. Also, multiplying the constraints in (2) by the positive number t
produces the constraints Ax̂ = bt and x̂ ≥ 0 that appear in:

Program 1.10.╇ z*╛=╛Maximize

cxt = cx̂, subject to

(4) Ax̂ = bt, d x̂ = 1, x̂ ≥ 0, t ≥ 0.

Programs 9 and 10 have the same data, namely, the matrix A and the vec-
tors b, c and d. They have different decision variables. Feasible solutions to
these two optimization problems are related to each other by:

Proposition 1.1.╇ Suppose Hypothesis A is satisfied. Equation (3) relates

each solution x to (2) to a solution (x̂, t) to (4), and conversely, with objective
values

cx
(5) = cx̂.
dx

Proof.╇ First, consider any solution x to (2). Part 1 of Hypothesis A guar-

antees that dx is positive, hence that t, as defined by (3), is positive. Thus,
1 = dxt = d x̂. Also, multiplying (2) by t and using x̂ = xt verifies that Ax̂ = bt
and that x̂ ≥ 0 , so (4) is satisfied. In addition, (cx)/(dx) = (cx)t = c(xt) = cx̂,
so (5) is satisfied.

Next, consider any solution (x̂, t) to (4). Part 2 of Hypothesis A guarantees

tâ•›>â•›0. This allows us to define x by x = x̂/t . Dividing Ax̂ = bt and x̂ ≥ 0 by
the positive number t verifies (2). Also, since x = x̂/t and d x̂ = 1, we have

cx cx cx
= ×t= × t = cx̂,
dx d x̂ 1
which completes the proof. ■
 Chapter 1: Eric V. Denardo 21

Proposition 1.1 shows how every feasible solution to Program 1.9 cor-
responds to a feasible solution to Program 1.10 that has the same objective
value. Thus, rather than solving Program 1.9 (which is nonlinear), we can
solve Program 1.10 (which is linear) and use (3) to construct an optimal solu-
tion to Program 1.9.

5.  Themes

Discussed in this section are several themes that are developed in later
chapters. These themes are:

• The central role played by the simplex pivot.

• The contributions made by linear programming to mathematics.

• The insights provided by linear programming into economics.

• The broad array of situations that can be modeled and solved as linear
programs and their generalizations.

A subsection is devoted to each theme.

Pivoting

At the heart of nearly every software package that solves linear programs
lies the simplex method. The simplex method was devised by George B.
Dantzig in 1947. An enormous number of person-years have been invested in
attempts to improve on the simplex method. Algorithms that compete with it
in specialized situations have been devised, but nothing beats it for general-
purpose use, especially when integer-valued solutions are sought. Dantzig’s
simplex method remains the best general-purpose solver six decades after he
proposed it.

At the core of the simplex method lies the pivot, which plays a central
role in Gauss-Jordan elimination. In Chapter 3, we will see how Gauss-Jordan
elimination pivots in search of a solution to a system of linear equations. In
Chapter 4, we will see that the simplex method keeps on pivoting, in search of
an optimal solution to a linear program. In Chapter 15, we’ll see how a slightly
different pivot rule (called complementary pivoting) finds the solution to a
non-zero sum matrix game. And in Chapter 16, we’ll see how complementary
22 Linear Programming and Generalizations

pivots find an approximation to a Brouwer fixed-point. That’s a remarkable

progression – variants of a simple idea solve a system of linear equations, a
linear program, and a fixed-point equation.

The simplex method presents a dilemma for theoreticians. It is the

best general-purpose solver of linear programs, but its worst-case behavior
is abysmal. It solves practical problems with blazing speed, but there exist
classes of specially-constructed linear programs for which the number of piv-
ots required by the simplex method grows exponentially with the size of the
problem. Many researchers have attempted to explain why these “bad” prob-
lems do not arrive in practice. Chapter 4 includes a thumb-nail discussion of
that issue.

Impact on mathematics

The analysis of linear programs and their generalizations have had a pro-
found impact on mathematics. Three facets of this impact are noted here.

People, commodities and a great many other items exist in nonnegative

quantities. But, prior to the development of linear programming, linear alge-
bra was nearly bereft of results concerning inequalities. The simplex method
changed that. Linear algebra is now rife with results that concern inequalities.
Some of these results appear in Chapter 12. Additionally, the simplex method
is the main technique for solving linear systems some of whose decision vari-
ables are required to be nonnegative.

The simplex method actually solves a pair of linear programs – the one
under attack and its dual. That it does so is an important – and largely un-
anticipated – facet of linear algebra whose implications are probed in Chap-
ter 12. Duality is an important addition to the mathematician’s tool kit; it
facilitates the proof of many theorems, as is evident in nearly every issue of
the journals, Mathematics of Operations Research and Mathematical Program-
ming.

Finally, as noted above, a generalization of the simplex pivot computes

approximate solutions to Brouwer’s fixed-point equation, thereby making a
deep contribution to nonlinear mathematics.

The overarching impact of linear programming on mathematics may

have been to emphasize the value of problem-based research.
 Chapter 1: Eric V. Denardo 23

Economic reasoning

This book includes several insights provided by linear programming and

its generalizations into economic reasoning. Two such insights are noted here.

To prepare for a discussion of one of these insights, it is observed that

nearly every list of the most important concepts in economic reasoning in-
cludes at least two of the following:

• The break-even price (a.k.a. shadow price) of a scarce resource.

• The opportunity cost of engaging in an activity.

• The importance of thinking at the margin, of assessing the incremental

benefit of doing something.

Curiously, throughout much of the economics literature, no clear link is

drawn between these three concepts. It will be seen in Chapter 5 that these
concepts are intimately related if one substitutes for opportunity cost the no-
tion of relative opportunity cost of doing something, this being the reduction
in benefit due to setting aside the resources needed to do that thing.

Within economics, these three concepts are usually described in the con-
text of an optimal allocation of resources. In Chapter 12, however, it will be
seen that these three concepts apply to each step of the simplex method, that
it uses them to pivot from one “basis” to another as it seeks an optimal solu-
tion.

It was mentioned earlier that every linear program is paired with another,
in particular, that Programs 1.1 and 1.2 are each duals. This duality provides
economic insight at several different levels. Three illustrations of its impact
are listed below.

• In Chapter 5, a duality between production quantities and prices is es-

tablished: Specifically, the dual of the problem of producing so as to
make the most profitable use of a bundle of resources is the problem of
setting least costly prices on those resources such that no activity earns
an “excess profit.”

• In Chapter 14, duality is used to construct a general equilibrium for

a stylized model of an economy. One linear program in this pair sets
production and consumption quantities that maximize the consumer’s
24 Linear Programming and Generalizations

well-being while requiring the market for each good to “clear.” The dual
linear program sets prices that maximize the producers’ profits. Their
optimal solutions satisfy the consumer’s budget constraint, thereby
constructing a general equilibrium.

• In Chapter 14, duality is also seen to be a simple and natural way in

which to analyze and solve von Neumann’s celebrated matrix game.

Linear programming also provides a number of insights into financial

economics.

Areas of application

Several chapters of this book are devoted to the situations that can be
modeled as linear programs and their generalizations.

• Models of the allocation of scarce resources are surveyed in Chapter 7.

• Network-based optimization problems are the subject of Chapters 8

and 9.

• Applications that entail strict inequalities are discussed in Chapter 12

• Methods for solving integer programs are included in Chapter 13.

• Models of competitive behavior are studied in Chapters 14-16.

• Optimality conditions for nonlinear programs are presented in Chap-

ter 20.

The applications in the above list are of a linear program, without regard
to its dual. Models of competition can be analyzed by a linear program and
its dual. These include the aforementioned model of an economy in general
equilibrium.

6.  Software

An enormous number of different software packages have been con-

structed that solve linear programs and their generalizations. Many of these
packages are available for classroom use, either at nominal charge or at no
charge. Each package has advantages and disadvantages. Several of them
 Chapter 1: Eric V. Denardo 25

dovetail nicely with spreadsheet computation. You – and your instructor –

have a choice. You may find it convenient to use any of a variety of software
packages.

One choice

To keep the exposition simple, this book is keyed to a pair of software

packages. They two are:

• Solver, which comes with Excel. The original version of Solver was
written by Frontline Systems. Solver is now maintained by Microsoft.

• Premium Solver, which is written and distributed by Frontline Systems.

An educational version of Premium Solver is available, free of charge.

These packages are introduced in Chapter 2, and their uses are elaborated
upon in subsequent chapters. These packages (and many others) have user
interfaces that are amazingly user-friendly.

Large problems

Solver and Premium Solver for Education can handle all of the linear and
nonlinear optimization problems that appear in this text. These codes fail
on problems that are “really big” or “really messy” – those with a great many
variables, with a great many constraints, with a large number of integer-val-
ued variables, or with nonlinear functions that are not differentiable. For big
problems, you will need to switch to one of the many commercially available
packages, and you may need to consult an expert.

7.  The Beginnings

Presented in this section is a brief account of the genesis of linear pro-

gramming. It began just before World War II in the U.S.S.R and just after
World War II in the United States.

Leoinid V. Kantorovich

In Leningrad (now St. Petersburg) in 1939, a gifted mathematician and

economist named L. V. Kantorovich (1912-1986) published a monograph on
26 Linear Programming and Generalizations

the best way to plan for production2. This monograph included a linear pro-
gram, and it recognized the importance of duality, but it seemed to omit a
systematic method of solution. In 1942, Kantorovich published a paper that
included a complete description of a network flow problem, including dual-
ity, again without a systematic solution method3.

For the next twenty years, Kantorovich’s work went unnoticed in the
West. Nor was it applauded within the U. S. S. R., where planning was central-
ized and break-even prices were anathema. It was eventually recognized that
Kantorovich was the first to explore linear programming and that he probed
it deeply. Leonid V. Kantorovich richly deserved his share of the 1975 Nobel
Prize in Economics, awarded for work on the optimal allocation of resources.

George B. Dantzig

George B. Dantzig (1914-2005) spent the years 1941 to 1945 in Washing-

ton, D.C., working on planning problems for the Air Force. To understand
why this might be excellent preparation for the invention of linear program-
ming, contemplate even a simple planning problem, such as organizing the
activities needed to produce parachutes at the rate of 5,000 per month.

After war’s end, Dantzig returned to Berkeley for a few months to com-
plete his Ph. D. degree. By the summer of 1946, Dantzig was back in Washing-
ton as the lead mathematician in a group whose assignment was to mechanize
the planning problems faced by the Air Force. By the spring of 1947, Dantzig
had observed that a variety of Air Force planning problems could be posed
as linear programs. By the summer of 1947 he had developed the simplex
method. These and a string of subsequent accomplishments have cemented
his stature as the preeminent figure in linear programming.

Tjalling C. Koopmans

Tjalling C. Koopmans (1910-1985) developed an interest in economics

while earning a Ph. D. in theoretical physics from the University of Leyden.
In 1940, he immigrated to the United States with his wife and six-week old

2╇
Kantorovich, L. V., The mathematical method of production planning and organiza-
tion, Leningrad University Press, Leningrad, 1939. Translated in Management Sci-
ence, V. 6, pp. 366-422, 1960.
3╇
Kantorovich, L. V., “On the translocation of masses,” Dokl. Akad. SSSR, V. 37,
pp. 227–229.
 Chapter 1: Eric V. Denardo 27

daughter. During the war, while serving as a statistician for the British Mer-
chant Shipping Mission in Washington, D.C., he built a model of optimal
routing of ships, with the attendant shadow costs. Koopmans shared the 1975
Nobel Prize in economics with Kantorovich for his contributions to the opti-
mal allocation of resources.

An historic conference

A conference on activity analysis was held from June 20-24, 1949, at the
Cowles Foundation, then located at the University of Chicago. This confer-
ence was organized by Tjalling Koopmans, who had become very excited
about the potential for linear programming during a visit by Dantzig in the
spring of 1947. The volume that emerged from this conference was the first
published compendium of results related to linear programming4. The par-
ticipants in this conference included six future Nobel Laureates (Kenneth Ar-
row, Robert Dorfman, Tjalling Koopmans, Paul Samuelson, Herbert Simon
and Robert Solow) and five future winners of the von Neumann Theory Prize
in Operations Research (George Dantzig, David Gale, Harold Kuhn, Herbert
Simon and Albert Tucker).

Military applications and the digital computer

Dantzig’s simplex method made possible the solution of a host of in-

dustrial and military planning problems – in theory. Solving these problems
called for vastly more computational power than could be achieved by scores
of operators of desk calculators. It was an impetus for the development of the
digital computer.

With amazing foresight, the Air Force organized Project SCOOP (sci-
entific computation of optimal programs) and funded the development and
acquisition of digital computers that could implement the simplex method.
These computers included:

• The SEAC (short for Standards Eastern Automatic Computer), which,

in 1951, solved a 48-equation 71-variable linear program in 18 hours.

• UNIVAC I, installed in 1952, which solved linear programs as large as

250 equations and 500 variables.

Activity analysis of production and allocation: Proceedings of a conference, Tjalling C.

4╇

Koopmans, ed., John Wiley & Sons, New York, 1951.

28 Linear Programming and Generalizations

It is difficult for a person who is not elderly to appreciate what clunkers

these early computers were – how hard it was to get them to do anything. But
Moore’s law may help: If computer power doubles every two years, accom-
plishing anything was more difficult by a factor of one billion (roughly 260/2)
sixty years ago.

Industrial applications

In a characteristically gracious memoir, William W. Cooper discussed the

atmosphere in the early days5. In the late 1940s at Carnegie Institute of Tech-
nology (now Carnegie Mellon University), a group that he directed wrestled
with the efficient blending of aviation fuels. Cooper describes the extant state
of linear programming as “embryonic … no publications were available.”
He reports that his group’s attempt to adapt activity analysis to the blending
problem was “fraught with difficulties.” He acknowledges failing to recog-
nize fully the significance of Dantzig’s work. His group quickly produced two
seminal papers, one on blending aviation fuels,6 another on the resolution of
degeneracy.7

In the same memoir, Cooper recounted his surprise at the response to

these papers. A large number of firms contacted him to express an eagerness
to learn more about these new methods for planning and control of their op-
erations. Within the oil industry, he received inquiries from the Soviet Bloc.
The oil industry would quickly become a major user of linear programming
and its generalizations.

8.  Review

This chapter is designed to introduce you to linear programming and to

provide you with a feel for what is coming.

5╇
Cooper. W. W., “Abraham Charnes and W. W. Cooper (et al): A brief history of a
long collaboration in developing industrial uses of linear programming,” Operations
Research, V. 50, pp. 35-41.
6╇
Charnes, A., W. W. Cooper and B. Mellon, “Blending aviation gasolines – a study of
programming interdependent activities in an integrated oil company, Econometrica,
V. 20, pp 135-159, 1952.
7╇
Charnes, A., “Optimality and degeneracy in linear programming,” Econometrica, V.
20, pp 160-170, 1952.
 Chapter 1: Eric V. Denardo 29

Terminology

The terminology that appears in Section 2 is used throughout this book,

indeed, throughout the literature on linear programming and its generaliza-
tions. Before proceeding, you should be familiar with each of the terms that
appear in boldface in that section – linear constraint, linear program, feasible
solution, objective value, infeasible linear program, and so forth.

Utility

It is hoped that you now have a feel for the value of studying linear pro-
gramming and its generalizations. Within this chapter, it has been observed
that:

• The basic model is flexible – some optimization problems that appear

to be nonlinear can be converted into equivalent problems that are lin-
ear.

• The methods have broad applicability – they adapt to handle strict in-
equalities, integer-valued variables, nonlinearities, and competition.

• Pivots are potent – with them, we can tackle systems of linear equa-
tions, linear programs, and fixed-point problems.

• Duality is central – it plays key roles in models of competition, in eco-

nomics, and in the mathematics that relates to optimization.

• Applications are ubiquitous – problems from many fields can be for-

mulated as linear programs and their generalizations.

• The subject provides insight into several academic disciplines – these

include computer science, economics, engineering, operations re-
search, and mathematics.

• Modern computer packages are user friendly – they solve a variety of

optimization problems with little effort on the part of the user, and they
are quick.

Its breadth, insight and usefulness may make linear programming the
most important development in applicable mathematics to have occurred
during the last 100 years.
30 Linear Programming and Generalizations

9.  Homework and Discussion Problems

1.╇ (subway cars) The Transit Authority must repair 100 subway cars per
month, and it must refurnish 50 subway cars per month. Both tasks can
be done by the Transit Authority, and both can be contracted to private
shops, but at a higher cost. Private contracting increases the cost by $2000
per car repaired and $2500 per car refurnished.

The Transit Authority repairs and refurnishes subway cars in four

shops. Repairing each car consumes 1/150th of the monthly capacity of its
Evaluation shop, 1/60th of the capacity of its Assembly shop, none of the
capacity of its Paint shop, and 1/60th of the capacity of its Machine shop.
Refurnishing each car requires 1/100 of the monthly capacity if its Evalu-
ation shop, 1/120th of the monthly capacity of its Assembly shop, 1/40th
of the monthly capacity of its Paint shop, and none of the capacity of its
Machine shop.

(a)╇ Formulate the problem of minimizing the monthly expense for private
contracting as a linear program. Solve it graphically.

(b)╇Formulate the problem of maximizing the monthly saving for repair-

ing and refurbishing in the Authority’s own shops as a linear program.
Does this linear program have the same solution as the one in part (a)?
If so, why? If not, why?

2. (A woodworking shop) A woodworking shop makes cabinets and tables.

The profit earned from each cabinet equals $700. The profit earned by
each table equals $500. The company’s carpentry shop has a capacity of
120 hours per week. Its finishing shop has a capacity of 80 hours per week.
Making each cabinet requires 20 hours of carpentry and 15 hours of fin-
ishing. Making each table requires 10 hours of carpentry and 10 hours
of finishing. The company wishes to determine the rates of production
(numbers of cabinets and tables per week) that maximize profit.

(a) Write down a linear program whose optimal solution accomplishes

this.

(b) Solve your linear program graphically.

3. (a fire drill) The principal of a new elementary school seeks an allocation

of students to exit doors that empties the school as quickly as possible
 Chapter 1: Eric V. Denardo 31

in the case of a fire. On a normal school day, there are 450 people in the
building. It has three exterior doors. With a bit of experimentation, she
learned that about 1.5 minutes elapse between the sounding of a fire alarm
and the emergence of people from door A, after which people can emerge
at the rate of 60 per minute. The comparable data for doors B and C are
delay of 1.25 minutes and 1.0 minutes, and rates of 40 per minute and 50
per minute, respectively.

(a) Write a linear program whose optimal solution allocates people to

doors in a way that empties the school as quickly as possible.

(b) Can you “eyeball” the optimal solution to this linear program? Hint:
After the first 1.5 minutes, are people filing out at the rate of 150 per
minute?

4. (deadheading) SW airline uses a single type of aircraft. Its service has been
disrupted by a major winter storm. A total of 20 aircraft, each with its
crew, must be deadheaded (flown without passengers) in order to resume
its normal schedule. To the right of the table that appears below are the
excess supplies at each of three airports. (These total 20). To the bottom
are the excess demands at five other airports. (They also total 20). Within
the table are the deadheading costs. For instance, the airline has 9 aircraft
too many at airport A, it has 5 aircraft too few at airport V, and the cost
of deadheading each aircraft from airport A to airport V is 25 thousand
dollars. The airline wishes to resume its normal schedule with the least
possible expense on deadheading.

(a) Suppose 10 is subtracted from each cost in the right-most column.

Does this subtract $30,000 (which equals 10â•›×â•›3â•›×â•›$1,000) from the
cost of every plan that restores 3 planes to airport Z?

(b) Subtract the smallest cost in each column from every cost in that col-
umn. Did this alter the relative desirability of different plans?

(c) With respect to the costs obtained from part (b), “eyeball” a shipping
plan whose cost is close to zero. How far from optimum can it be?
Have you established a lower bound on the cost of resuming SW air-
line’s normal schedule?
32 Linear Programming and Generalizations

V W X Y Z supply
A 25 10 20 25 20 9
B 5 10 80 20 40 4
C 10 40 75 10 10 7
demand 5 2 4 6 3

5. (linear fractional program)* Suppose Program 1.9 has a bounded feasible

region. Can there be a nonzero solution to the constraints A xâ•›=â•›0 and
xâ•›≥â•›0? Is part 2 of Hypothesis A guaranteed? Explain your answers.

6. (cotton tents) During WW II, Dantzig’s group used mechanical calcula-

tors to help them plan and organize the production of all items as compli-
cated as aircraft. Imagine something relatively simple, specifically, the job
of organizing the production of standard-issue cotton military tents at the
rate of 15,000 per month. Describe a (triangular) “goes-into” matrix whose
entries would determine what goods would need to be produced, each at
a prescribed monthly rate. (You may wish to check the web to see what a
standard-issue military tent might have looked like.) Do production ca-
pacities come into play? Can you conceive of a role for a linear program? If
so, might it be necessary to account for decreasing marginal benefit and/
or ratio constraints? If so, why?
 Chapter 2: Spreadsheet Computation

1.╅ Preview����������������������������������尓������������������������������������尓������������������������ 33
2.╅ The Basics����������������������������������尓������������������������������������尓�������������������� 34
3.╅ Expository Conventions����������������������������������尓���������������������������������� 38
4.╅ The Sumproduct Function����������������������������������尓������������������������������ 40
5.╅ Array Functions and Matrices����������������������������������尓������������������������ 44
6.╅ A Circular Reference ����������������������������������尓������������������������������������尓�� 46
7.╅ Linear Equations ����������������������������������尓������������������������������������尓�������� 47
8.╅ Introducing Solver ����������������������������������尓������������������������������������尓������ 50
9.╅ Introducing Premium Solver����������������������������������尓�������������������������� 56
10.╇ What Solver and Premium Solver Can Do����������������������������������尓���� 60
11.╇ An Important Add-In����������������������������������尓������������������������������������尓�� 62
12.╇ Maxims for Spreadsheet Computation����������������������������������尓���������� 64
13.╇ Review����������������������������������尓������������������������������������尓�������������������������� 65
14.╇ Homework and Discussion Problems����������������������������������尓������������ 65

1.  Preview

Spreadsheets make linear programming easier to learn. This chapter con-

tains the information about spreadsheets that will prove useful. Not all of that
information is required immediately. To prepare for Chapters 3 and 4, you
should understand:

• a bit about Excel functions, especially the sumproduct function;

• what a circular reference is;

E. V. Denardo, Linear Programming and Generalizations, International Series 33

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_2, © Springer Science+Business Media, LLC 2011
34 Linear Programming and Generalizations

• how to download from the Springer website and activate a group of

Excel Add-Ins called OP_TOOLS;

• how to use Solver to find solutions to systems of linear equations.

Excel has evolved, and it continues to evolve. The same is true of Solver.
Several versions of Excel and Solver are currently in use. A goal of this chapter
is to provide you with the information that that is needed to make effective
use of the software with which your computer is equipped.

Excel for PCs

If your computer is a PC, you could be using Excel 2003, 2007 or 2010.
Excel 2003 remains popular. Excel 2007 and Excel 2010 have different file
structures. To ease access, each topic is introduced in the context of Excel
2003 and is adapted to more recent versions of Excel in later subsections.
Needless to say, perhaps, some subsections are more relevant to you than
others.

Excel for Macs

If your computer is a Mac that is equipped with a version of Excel that

is dated prior to 2008, focus on the discussion of Excel 2003, which is quite
similar. If your computer is a Mac that is equipped with Excel 2011, focus on
the discussion of Excel 2010, which is similar.

But if your computer is equipped with Excel 2008 (for Macs only), its
software has a serious limitation. Excel 2008 does not support Visual Basic.
This makes it less than ideal for scientific and business uses. You will not be
able to use your computer to take the grunt-work out of the calculations in
Chapters 3 and 4, for instance. Upgrade to Excel 2011 as soon as possible.
It does support Visual Basic. Alternatively, use a different version of Excel,
either on your computer or on some other.

2.  The Basics

This section contains basic information about Excel. If you are familiar
with Excel, scan it or skip it.
 Chapter 2: Eric V. Denardo 35

At first glance, a spreadsheet is a pretty dull object – a rectangular array of

cells. Into each cell, you can place a number, or some text, or a function. The
function you place in a cell can call upon the values of functions in other cells.
And that makes a spreadsheet a potent programming language, one that has
revolutionized desktop computing.

Cells

Table 2.1 displays the upper left-hand corner of a spreadsheet. In spread-

sheet lingo, each rectangle in a spreadsheet is called a cell. Evidently, the col-
umns are labeled by letters, the rows by numbers. When you refer to a cell,
the column (letter) must come first; cell B5 is in the second column, fifth row.

Table 2.1.↜  A spreadsheet

You select a cell by putting the cursor in that cell and then clicking it.
When you select a cell, it is outlined in heavy lines, and a fill handle appears
in the lower right-hand corner of the outline. In Table 2.1, cell C9 has been
selected. Note the fill handle – it will prove to be very handy.

Entering numbers

Excel allows you to enter about a dozen different types of information

into a cell. Table 2.1 illustrates this capability. To enter a number into a cell,
select that cell, then type the number, and then depress either the Enter key or
any one of the arrow keys. To make cell A2 look as it does, select cell A2, type
0.3 and then hit the Enter key.
36 Linear Programming and Generalizations

Entering functions

In Excel, functions (and only functions) begin with the “=” sign. To enter a
function into a cell, select that cell, depress the “=” key, then type the function,
and then depress the Enter key. The function you enter in a cell will not appear
there. Instead, the cell will display the value that the function has been assigned.

It Table 2.1, cell A3 displays the value 24, but it is clear (from column C)
that cell A3 contains the function 23â•›×â•›3, rather than the number 24. Similarly,
√
cell A5 displays the number 1.414…, which is the value of the function 2,
evaluated to ten significant digits.

Excel includes over 100 functions, many of which are self-explanatory.

We will use only a few of them. To explore its functions, on the Excel Insert
menu, click on Functions.

Entering text

To enter text into a cell, select that cell, then type the text, and then de-
press either the Enter key or any one of the arrow keys. To make cell A6 look
as it does, select cell A6 and type mean. Then hit the Enter key. If the text
you wish to place in a cell could be misinterpreted, begin with an apostrophe,
which will not appear. To make cell A7 appear as it does in Table 2.1, select
cell A7, type ‘= mean, and hit the Enter key. The leading apostrophe tells Excel
that what follows is text, not a function.

Formatting a cell

In Table 2.1, cell A8 displays the fraction 1/3. Making that happen looks
easy. But suppose you select cell A8, type 1/3 and then press the Enter key.
What will appear in cell A8 is “3-Jan.” Excel has decided that you wish to put
a date in cell A8. And Excel will interpret everything that you subsequently
enter into cell A8 as a date. Yuck!

With Excel 2003 and earlier, the way out of this mess is to click on the
Format menu, then click on Cells, then click on the Number tab, and then
select either General format or a Type of Fraction.

Format Cells with Excel 2007

With Excel 2007, the Format menu disappeared. To get to the Format
Cells box, double-click on the Home tab. In the menu that appears, click on
 Chapter 2: Eric V. Denardo 37

the Format icon, and then select Format Cells from the list that appears. From
here on, proceed as in the prior subsection.

Format Cells with Excel 2010

With Excel 2010, the Format Cells box has moved again. To get at it, click
on the Home tab. A horizontal “ribbon” will appear. One block on that ribbon
is labeled Number. The lower-right hand corner of the Number block has a
tiny icon. Click on it. The Format Cells dialog box will appear.

Entering Fractions

How can you get the fraction 1/3 to appear in cell A8 of Table 2.1? Here is
one way. First, enter the function =1/3 in that cell. At this point, 0.333333333
will appear there. Next, with cell A8 still selected, bring the Format Cells box
into view. Click on its Number tab, select Fraction and the Type labeled Up
to one digit. This will round the number 0.333333333 off to the nearest one-
digit fraction and report it in cell A8.

The formula bar

If you select a cell, its content appears in the formula bar, which is the
blank rectangle just above the spreadsheet’s column headings. If you select
cell A5 of Table 2.1, the formula =SQRT(2) will appear in the formula bar, for
instance. What good is the formula bar? It is a nice √ place to edit your func-
tions. If you want to change the number in cell A5 to 3, select cell A5, move
the cursor onto the formula bar, and change the 2 to a 3.

Arrays

In Excel lingo, an array is a rectangular block of cells. Three arrays are

displayed below. The array B3:E3 (note the colon) consists of a row of 4 cells,
which are B3, C3, D3 and E3. The array B3:B7 consists of a column of 5 cells.
The array B3:E7 consists of 20 cells.

B3:E3â•…â•…â•…â•…â•… B3:B7â•…â•…â•…â•…â•… B3:E7

Absolute and relative addresses

Every cell in a spreadsheet can be described in four different ways be-

cause a “$” sign can be included or excluded before its row and/or column.
The came cell is specified by:
38 Linear Programming and Generalizations

B3â•…â•…â•…â•…â•… B$3â•…â•…â•…â•…â•… $B3â•…â•…â•…â•…â•… $B$3

In Excel jargon, a relative reference to a column or row omits the “$” sign,
and an absolute (or fixed) reference to a column or row includes the “$” sign.

Copy and Paste

Absolute and relative addressing is a clever feature of spreadsheet pro-

grams. It lets you repeat a pattern and compute recursively. In this subsection,
you will see what happens when you Copy the content of a cell (or of an array)
onto the Clipboard and then Paste it somewhere else.

With Excel 2003 and earlier, select the cell or array you want to repro-
duce. Then move the cursor to the Copy icon (it is just to the right of the scis-
sors), and then click it. This puts a copy of the cell or array you selected on the
Clipboard. Next, select the cell (or array) in which you want the information
to appear, and click on the Paste icon. What was on the clipboard will appear
where you put it except for any cell addresses in functions that you copied
onto the Clipboard. They will change as follows:

• The relative addresses will shift the number rows and/or columns that
separate the place where you got it and the place where you put it.

• By contrast, the absolute addresses will not shift.

This may seem abstruse, but its uses will soon be evident.

Copy and Paste with Excel 2007 and Excel 2010

With Excel 2007, the Copy and Paste icons have been moved. To make
them appear, double-click on the Home tab. The Copy icon will appear just
below the scissors. The Paste icon appears just to the left of the Copy icon,
and it has the word “Paste” written below it. With Excel 2010, the Copy and
Paste icons are back in view – on the Home tab, at the extreme left.

3.  Expository Conventions

An effort has been made to present material about Excel in a way that is
easy to grasp. As concerns keystroke sequences, from this point on:
 Chapter 2: Eric V. Denardo 39

This text displays each Excel keystroke sequence in boldface type, omit-
ting both:

•â•‡ The Enter keystroke that finishes the keystroke sequence.

•â•‡Any English punctuation that is not part of the keystroke se-

quence.

For instance, cells A3, A4 and A5 of Table 2.1 contain, respectively,

=2^3*3â•…â•…â•…â•… =EXP(1)â•…â•…â•…â•… =SQRT(2)

Punctuation is omitted from keystroke sequences, even when it leaves off

the period at the end of the sentence!

The spreadsheets that appear in this text display the values that have been
assigned to functions, rather than the functions themselves. The convention
that is highlighted below can help you to identify the functions.

When a spreadsheet is displayed in this book:

•â•‡If a cell is outlined in dotted lines, it displays the value of a func-

tion, and that function is displayed in some other cell.

•â•‡The “$” signs in a function’s specification suggest what other cells

contain similar functions.

In Table 2.1, for instance, cells A3, A4 and A5 are outlined in dotted lines,
and column C specifies the functions whose values they contain. Finally:

The Springer website contains two items that are intended for use
with this book. They can be downloaded from http://extras.springer.
com/2011/978-1-4419-6490-8.

One of the items at the Springer website is a folder that is labeled, “Excel
spreadsheets – one per chapter.” You are encouraged to download that folder
now, open its spreadsheet for Chapter 2, note that this spreadsheet contains
sheets labeled Table 2.1, Table 2.2, …, and experiment with these sheets as
you proceed.
40 Linear Programming and Generalizations

4.  The Sumproduct Function

Excel’s SUMPRODUCT function is extremely handy. It will be intro-

duced in the context of

Problem 2.A.╇ For the random variable X that is described in Table 2.2, com-
pute the mean, the variance, the standard deviation, and the mean absolute
deviation.

Table 2.2.   A random variable, X.

The sumproduct function will make short work of Problem 2.A. Before
discussing how, we interject a brief discussion of discrete probability models.
If you are facile with discrete probability, it is safe to skip to the subsection
entitled “Risk and Return.”

A discrete probability model

The random variable X in Table 2.2 is described in the context of a dis-

crete probability model, which consists of “outcomes” and “probabilities:”

• The outcomes are mutually exclusive and collectively exhaustive. Ex-

actly one of the outcomes will occur.

• Each outcome is assigned a nonnegative number, which is interpreted

as the probability that the outcome will occur. The sum of the probabili-
ties of the outcomes must equal 1.0.

A random variable assigns a number to each outcome.

 Chapter 2: Eric V. Denardo 41

The probability model in Table 2.2 has four outcomes, and the sum of
their probabilities does equal 1.0. Outcome b will occur with probability 0.55,
and the random variable X will take the value 3.2 if outcome b occurs.

A measure of the center

The random variable X in Table 2.2 takes values between –6 and +22. The
mean (a.k.a. expectation) of a random variable represents the “center” of its
probability distribution. The mean of a random variable X is denoted as μ or
E(X), and it is found by multiplying the probability of each outcome by the
value that the random variable takes when that outcome occurs and taking
the sum. For the data in Table 2.2, we have

µ = E(X) = (0.30) × (−6) + (0.55) × (3.2) + (0.12) × (10) + (0.03) × (22)

= 1.82.

The mean of a random variable has the same unit of measure as does the
random variable itself. If X is measured in dollars, so is its mean. The mean is
a weighted average; each value that X can take is weighed (multiplied) by its
probability.

Measures of the spread

There are several measures of the spread of a random variable, that is, of
the difference (X – μ) between the random variable X and its mean. The most
famous of these measures of spread is known as the variance. The variance of
a random variable X is denoted as σ 2 or Var(X) and is the expectation of the
square of (X – μ). For the data in Table 2.2, we have

σ 2 = Var(X) = (0.30) × (−6 − 1.82)2 + (0.55) × (3.2 − 1.82)2

+ (0.12) × (10 − 1.82)2 + (0.03) × (22 − 1.82)2 ,
= 39.64.

The unit of measure of the variance is the square of the unit of measure of
the random variable. If X is measured in dollars, Var(X) is measured in (dol-
lars)â•›×â•›(dollars), which is a bit weird.

The standard deviation of a random variable X is denoted as σ 2or StDev(X)

and is the square root of its variance. For the data in Table 2.2,
σ = StDev(X) = 6.296.
42 Linear Programming and Generalizations

The standard deviation of a random variable has the same unit of mea-
sure as does the random variable itself.

A less popular measure of the spread of a random variable is known as its

mean absolute deviation. The mean absolute deviation of a random variable X
is denoted MAD(X) and it is the expectation of the absolute value of (X – μ).
For the data in Table 2.2,

MAD(X) = (0.30) × |−6 − 1.82| + (0.55) × |3.2 − 1.82|

+ (0.12) × |10 − 1.82| + (0.03) × |22 − 1.82|,
= 4.692

Taking the square (in the variance) and then the square root (in the stan-
dard deviation) seems a bit contrived, and it emphasizes values that are far
from the mean. For many purposes, the mean absolute deviation may be a
more natural measure of the spread in a distribution.

Risk and return

Interpret the random variable X as the profit that will be earned from a
portfolio of investments. A tenet of financial economics is that in order to
obtain a higher return one must accept a higher risk. In this context, E(X) is
taken as the measure of return, and StDev(X) as the measure of risk. It can
make sense to substitute MAD(X) as the measure of risk. Also, as suggested in
Chapter 1, a portfolio X that minimizes MAD(X) subject to the requirement
that E(X) be at least as large as a given threshold can be found by solving a
linear program.

Using the sumproduct function

The arguments in the sumproduct function must be arrays that have the
same number of rows and columns. Let us suppose we have two arrays of
the same size. The sumproduct function multiplies each element in one of
these arrays by the corresponding element in the other and takes the sum.
The same is true for three arrays of the same size. That makes it easy to com-
pute the mean, the variance and the standard deviation, as is illustrated in
Table 2.3
 Chapter 2: Eric V. Denardo 43

Table 2.3.↜  A spreadsheet for Problem 2.A.

Note that:

• The function in cell C13 multiplies each entry in the array C5:C8 by
the corresponding entry in the array D5:D8 and takes the sum, thereby
computing μ = E(X).

• The functions in cells E5 through E8 subtract 1.82 from the values in

cells D5 through D8, respectively.

• The function in cell D13 sums the product of corresponding entries

in the three arrays C5:C8 and E5:E8 and E5:E8, thereby computing
Var(X).

The arrays in a sumproduct function must have the same number of rows
and the same number of columns. In particular, a sumproduct function will
not multiply each element in a row by the corresponding element in a column
of the same length.

Dragging

The functions in cells E5 through E8 of Table 2.3 could be entered sepa-

rately, but there is a better way. Suppose we enter just one of these functions,
in particular, that we enter the function =D5 – C$13 in cell E5. To drag this
function downward, proceed as follows:
44 Linear Programming and Generalizations

• Move the cursor to the lower right-hand corner of cell E5. The fill han-
dle (a small rectangle in the lower right-hand corner of cell E5) will
change to a Greek cross (“+” sign).

• While this Greek cross appears, depress the mouse, slide it down to cell
E8 and then release it. The functions =D6 – C$13 through =D8 – C$13
will appear in cells E6 through E8. Nice!

Dragging downward increments the relative row numbers, but not the
fixed row numbers. Similarly, dragging to the right increases the relative col-
umn numbers, but leaves the fixed column numbers unchanged. Dragging is
an especially handy way to repeat a pattern and to execute a recursion.

5.  Array Functions and Matrices

As mentioned earlier, in Excel lingo, an array is any rectangular block of

cells. Similarly, an array function is an Excel function that places values in an
array, rather than in a single cell. To have Excel execute an array function, you
must follow this protocol:

• Select the array (block) of cells whose values this array function will
determine.

• Type the name of the array function, but do not hit the Enter key. In-
stead, hit Ctrl+Shift+Enter (In other words, depress the Ctrl and Shift
keys and, while they are depressed, hit the Enter key).

Matrix multiplication

A matrix is a rectangular array of numbers. Three matrices are exhibited

below, where they have been assigned the names (labels) A, B and C.

3 2
 
0 1 2 4 2
   
A= , B =  2 0 , C= ,
−1 1 −1 1 3
1 1

The product A B of two matrices is defined if – and only if – the number

of columns in A equals the number of rows in B. If A is an mâ•›×â•›n matrix and
B is an nâ•›×â•›p matrix, the matrix product A B is the mâ•›×â•›p matrix whose ijth
 Chapter 2: Eric V. Denardo 45

element is found by multiplying each element in the ith row of A by the cor-
responding element in the jth column of B and taking the sum.

It is easy to check that matrix multiplication is associative, specifically,

that (A B) Câ•›=â•›A (B C) if the number of columns in A equals the number of
rows in B and if the number of columns in B equals the number of rows in C.

A spreadsheet

Doing matrix multiplication by hand is tedious and error-prone. Excel

makes it easy. The matrices A, B and C appear as arrays in Table 2.4. That
table also displays the matrix product A B and the matrix product A B C. To
create the matrix product A B that appears as the array C10:D11 of Table 2.4,
we took these steps:

• Select the array C10:D11.

• Type =mmult(C2:E3, C6:D8)

• Hit Ctrl+Shift+Enter

Table 2.4.↜  Matrix multiplication and matrix inversion.

The matrix product A B C can be computed in either of two ways. One

way is to multiply the array A B in cells C10:D11 by the array C. The other
46 Linear Programming and Generalizations

way is by using the =mmult(array, array) function recursively, as has been

done in Table 2.4. Also computed in Table 2.4 is the inverse of the matrix C.

Quirks

Excel computes array functions with ease, but it has its quirks. One of
them has been mentioned – you need to remember to end each array func-
tion by hitting Ctrl+Shift+Enter rather than by hitting Enter alone.

A second quirk concerns 0’s. With non-array functions, Excel (wisely)

interprets a “blank” as a “0.” When you are using array functions, it does not;
you must enter the 0’s. If your array function refers to a cell containing a blank,
the cells in which the array is to appear will contain an (inscrutable) error
message, such as ##### or #Value.

The third quirk occurs when you decide to alter an array function or to
eliminate an array. To do so, you must begin by selecting all of the cells in
which its output appears. Should you inadvertently attempt to change a por-
tion of the output, Excel will proclaim, “You cannot change part of an Array.”
If you then move the cursor – or do most anything – Excel will repeat its
proclamation. A loop! To get out of this loop, hit the Esc key.

6.  A Circular Reference

An elementary problem in algebra is now used to bring into view an im-

portant limitation of Excel. Let us consider

Problem 2.B.╇ Find values of x and y that satisfy the equations

x = 6 – 0.5y,

y = 2 + 0.5x.

This is easy. Substituting (2â•›+â•›0.5x) for y in the first equation gives xâ•›=â•›4
and hence yâ•›=â•›4.

Let us see what happens when we set this problem up in a naïve way for
solution on a spreadsheet. In Table 2.5, formulas for x and y have been placed
in cells B4 and B5. The formula in each of these cells refers to the value in
 Chapter 2: Eric V. Denardo 47

the other. A loop has been created. Excel insists on being able to evaluate the
functions on a spreadsheet in some sequence. When Excel is presented with
Table 2.5, it issues a circular reference warning.

Table 2.5.   Something to avoid.

You can make a circular reference warning disappear. If you do make it

disappear, your spreadsheet is all but certain to be gibberish. It is emphasized:

Danger: Do not ignore a “circular reference” warning. You can make it

go away. If you do, you will probably wreck your spreadsheet.

This seems ominous. Excel cannot solve a system of equations. But it can,
with a bit of help.

7.  Linear Equations

To see how to get around the circular reference problem, we turn our
attention to an example that is slightly more complicated than Problem 2.B.
This example is

Problem 2.C.╇ Find values of the variables A, B and C that satisfy the equa-
tions

2A + 3B + 4C = 10,
2A – 2B – C = 6,
A + B + C = 1.
48 Linear Programming and Generalizations

You probably recall how to solve Problem 2.C, and you probably recall
that it requires some grunt-work. We will soon see how to do it on a spread-
sheet, without the grunt-work.

An ambiguity

Problem 2.C exhibits an ambiguity. The letters A, B and C are the names
of the variables, and Problem 2.C asks us to find values of the variables A, B
and C that satisfy the three equations. You and I have no trouble with this
ambiguity. Computers do. On a spreadsheet, the name of the variable A will
be placed in one cell, and its value will be placed in another cell.

A spreadsheet for Problem 2.C

Table 2.6 presents the data for Problem 2.C. Cells B2, C2 and D2 contain
the labels of the three decision variables, which are A, B and C. Cells B6, C6
and D6 have been set aside to record the values of the variables A, B and C.
The data in the three constraints appear in rows 3, 4 and 5, respectively.

Table 2.6.   The data for Problem 2.C.

Note that:

• Trial values of the decision variables have been inserted in cells B6, C6
and D6.

• The “=” signs in cells F3, F4 and F5 are memory aides; they remind
us that we want to arrange for the numbers to their left to equal the
numbers to their right, but they have nothing to do with the computa-
tion.
 Chapter 2: Eric V. Denardo 49

• The sumproduct function in E5 multiplies each entry in the array

B$6:D$6 by the corresponding entry in the array B5:D5 and reports
their sum.

• The “$” signs in cell E5 suggest – correctly – that this function has been
dragged upward onto cells E4 and E3. For instance, cell E3 contains the
value assigned to the function

=â•›SUMPRODUCT(B3:D3, B$6:D$6)

and the number 9 appears in cell E3 because Excel assigns this function
the value 9 = 2â•›×â•›1â•›+â•›3â•›×â•›1â•›+â•›4â•›×â•›1.

The standard format

The pattern in Table  2.6 works for any number of linear equations in
any number of variables. This pattern is dubbed the “standard format” for
linear systems, and it will be used throughout this book. A linear system is
expressed in standard format if the columns of its array identify the variables
and the rows identify the equations, like so:

• One row is reserved for the values of the variables (row 6, above).

• The entries in an equation’s row are:

–╇The equation’s coefficient of each variable (as in cells B3:D3, above).

–╇A sumproduct function that multiplies the equation’s coefficient of

each variable by the value of that variable and takes the sum (as in
cell E3).

–╇An “=” sign that serves (only) as a memory aid (as in cell F3).

–╇The equation’s right-hand-side value (as in cell G3).

What is missing?

Our goal is to place numbers in cells B6:D6 for which the values of the
functions in cells E3:E5 equal the numbers in cells G3:G5, respectively. Excel
cannot do that, by itself. We will see how to do it with Solver and then with
Premium Solver for Education.
50 Linear Programming and Generalizations

8.  Introducing Solver

This section is focused on the simplest of Solver’s many uses, which is to

find a solution to a system of linear equations. The details depend, slightly, on
the version of Excel with which your computer is equipped.

A bit of the history

Let us begin with a bit of the history. Solver was written by Frontline
Systems for inclusion in an early version of Excel. Shortly thereafter, Micro-
soft took over the maintenance of Solver, and Frontline Systems introduced
Premium Solver. Over the intervening years, Frontline Systems has improved
its Premium Solver repeatedly. Recently, Microsoft and Frontline Systems
worked together in the design of Excel 2010 (for PCs) and Excel 2011 (for
Macs). As a consequence:

• If your computer is equipped with Excel 2003 or Excel 2007, Solver is per-
fectly adequate, but Premium Solver has added features and fewer bugs.

• If your computer is equipped with Excel 2010 (for PCs) or with Excel
2011 (for Macs), a great many of the features that Frontline Systems
introduced in Premium Solver have been incorporated in Solver itself,
and many bugs have been eliminated.

• If your computer is equipped with Excel 2008 for Macs, it does not sup-
port Visual Basic. Solver is written in Visual Basic. The =pivot(cell, ar-
ray) function, which is used extensively in this book, is also written in
Visual Basic. You will not be able to use Solver or the “pivot” function
until you upgrade to Excel 2011 (for Macs). Until then, use some other
version of Excel as a stopgap.

Preview

This section begins with a discussion of the version of Solver with which
Excel 2000, 2003 and 2007 are equipped. The discussion is then adapted to
Excel 2010 and 2011. Premium Solver is introduced in the next section.

Finding Solver

When you purchased Excel (with the exception of Excel 2008 for Macs),
you got Solver. But Solver is an “Add-In,” which means that it may not be
ready to use. To see whether Solver is up and running, open a spreadsheet.
 Chapter 2: Eric V. Denardo 51

With Excel 2003 or earlier, click on the Tools menu. If Solver appears
there, you are all set; Solver is installed and activated. If Solver does not ap-
pear on the Tools menu, it may have been installed but not activated, and it
may not have been installed. Proceed as follows:

• Click again on the Tools menu, and then click on Add-Ins. If Solver is
listed as an Add-In but is not checked off, check it off. This activates
Solver. The next time you click on the Tools menu, Solver will appear
and will be ready to use.

• If Solver does not appear on the list of Add-Ins, you will need to find
the disc on which Excel came, drag Solver into your Library, and then
activate it.

Finding Solver with Excel 2007

If your computer is equipped with Excel 2007, Solver is not on the Tools
menu. To access Solver, click on the Data tab and then go to the Analysis box.
You will see a button labeled Solver if it is installed and active. If the Solver
button is missing:

• Click on the Office Button that is located at the top left of the spread-
sheet.

• In the bottom right of the window that appears, select the Excel Op-
tions button.

• Next, click on the Add-Ins button on the left and look for Solver Add-
In in the list that appears.

• If it is in the inactive section of this list, then select Manage: Excel Add-
Ins, then click Go…, and then select the box next to Solver Add-in and
click OK.

• If Solver Add-in is not listed in the Add-Ins available box, click Browse
to locate the add-in. If you get prompted that the Solver Add-in is not
currently installed on your computer, click Yes to install it.

Finding Solver with Excel 2010

To find Solver with Excel 2010, click on the Data tab. If Solver appears
(probably at the extreme right), you are all set. If Solver does not appear, you
52 Linear Programming and Generalizations

will need to activate it, and you may need to install it. To do so, open an Excel
spreadsheet and then follow this protocol:

• Click on the File menu, which is located near the top left of the spread-
sheet.

• Click on the Options tab (it is near the bottom of the list) that appeared
when you clicked on the File menu.

• A dialog box named Excel Options will pop up. On the side-bar to its
left, click on Add-Ins. Two lists of Add-Ins will appear – “Active Appli-
cation Add-Ins” and “Inactive Application Add-Ins.”

–╇If Solver is on the “Inactive” list, find the window labeled “Manage:
Excel Add-Ins,” click on it, and then click on the “Go” button to its
right. A small menu entitled Add-Ins will appear. Solver will be on
it, but it will not be checked off. Check it off, and then click on OK.

–╇If Solver is not on the “Inactive” list, click on Browse, and use it to
locate Solver. If you get a prompt that the Solver Add-In is not cur-
rently installed on your computer, click “Yes” to install it. After in-
stalling it, you will need to activate it; see above.

Using Solver with Excel 2007 and earlier

Having located Solver, we return to Problem 2.C. Our goal is to have

Solver find values of the decision variables A, B and C that satisfy the equa-
tions that are represented by Table 2.6. With Excel 2007 and earlier, the first
step is to make the Solver dialog box look like Figure 2.1. (The Solver dialog
box for Excel 2010 differs in ways that are described in the next subsection.)

To make your Solver dialog box look like that in Figure 2.1, proceed as
follows:

• With Excel 2003, on the Tools menu, click on Solver. With Excel 2007,
go to the Analysis box of the Data tab, and click on Solver.

• Leave the Target Cell blank.

• Move the cursor to the By Changing Cells window, then select cells
B6:D6, and then click.

• Next, click on the Add button.

 Chapter 2: Eric V. Denardo 53

Figure 2.1.   A Solver dialog box for Problem 2.C.

• An Add Constraint dialog box will appear. Proceed as follows:

–╇Click on the Cell Reference window, then select cells E3:E5 and click.

–╇Click on the triangular button on the middle window. On the drop-

down menu that appears click on “=”.

–╇Click on the Constraint window. Then select cells G3:G5 and click.
This will cause the Add Constraint dialog box to look like:

–╇Click on OK. This will close the Add Constraint dialog box and re-
turn you to the Solver dialog box, which will now look exactly like
Figure 2.1.

• In the Solver dialog box, do not click on the Solve button. Instead, click
on the Options button and, on the Solver Options menu that appears
54 Linear Programming and Generalizations

(see below) click on the Assume Linear Model window. Then click on
the OK button. And then click on Solve.

In a flash, your spreadsheet will look like that in Table  2.7. Solver has
succeeded; the values it has placed in cells B6:D6 that enforce the constraints
E3:E5â•›=â•›G3:G5; evidently, setting Aâ•›=â•›0.2, Bâ•›=â•›–6.4 and Câ•›=â•›7.2 which solves
Problem 2.C.

Table 2.7.   A solution to Problem 2.C.

Using Solver with Excel 2010

Presented as Figure 2.2 is a Solver dialog box for Excel 2010. It differs from
the dialog box for earlier versions of Excel in the ways that are listed below:
 Chapter 2: Eric V. Denardo 55

• The cell for which the value is to be maximized or minimized in an

optimization problem is labeled Set Objective, rather than Target Cell.

• The method of solution is selected on the main dialog box rather than
on the Options page.

Figure 2.2.↜  An Excel 2010 Solver dialog box.

56 Linear Programming and Generalizations

• The capability to constrain the decision variables to be nonnegative ap-

pears on the main dialog box, rather than on the Options page.

• A description of the “Solving Method” that you have selected appears at

the bottom of the dialog box.

Fill this dialog box out as you would for Excel 2007, but remember to
select the option you want in the “nonnegative variables” box.

9.  Introducing Premium Solver

Frontline Systems has made available for educational use a bundle of soft-
ware called the Risk Solver Platform. This software bundle includes Premium
Solver, which is an enhanced version of Solver. This software bundle also in-
cludes the capability to formulate and run simulations and the capability to
draw and roll back decision trees. Sketched here are the capabilities of Pre-
mium Solver. This sketch is couched in the context of Excel 2010. If you are
using a different version of Excel, your may need to adapt it somewhat.

Note to instructors

If you adopt this book for a course, you can arrange for the participants
in your course (including yourself, of course) to have free access to the edu-
cational version of the Risk Solver Platform. To do so, call Frontline Systems
at 755 831-0300 (country code 01) and press 0 or email them at academics@
solver.com.

Note to students

If you are enrolled in a course that uses this book, you can download
the Risk Solver Platform by clicking on the website http://solver.com/student/
and following instructions. You will need to specify the “Textbook Code,”
which is DLPEPAE, and the “Course code,” which your instructor can pro-
vide.

Using Premium Solver as an Add-In

Premium Solver can be accessed and used in two different ways – as an

Add-In or as part of the Risk Solver Platform. Using it as an Add-In is dis-
 Chapter 2: Eric V. Denardo 57

cussed in this subsection. Using it as part of the Risk Solver Platform is dis-
cussed a bit later.

To illustrate the use of Premium Solver as an Add-In, begin by reproduc-

ing Table 2.6 on a spreadsheet. Then, in Excel 2010, click on the File button.
An Add-Ins button will appear well to the right of the File button. Click on
the Add-Ins button. After you do so, you will see a rectangle at the left with a
light bulb and the phrase “Premium Solver Vxx.x” (currently V11.0). Click on
it. A Solver Parameters dialog box will appear. You will need to make it look
like that in Figure 2.3.

Figure 2.3.↜  A dialog box for using Premium Solver as an Add-In.

58 Linear Programming and Generalizations

Filling in this dialog box is easy:

• In the window to the left of the Options button, click on Standard LP/
Quadratic.

• Next, in the large window, click on Normal Variables. Then click on

the Add button. A dialog box will appear. Use it to identify B6:D6 as
the cells whose values Premium Solver is to determine. Then click on
OK. This returns you to the dialog box in Figure 2.3, with the variables
identified.

• In the large window, click on Normal Constraints. Then click on the

Add button. Use the (familiar) dialog box to insert the constraints
E3:E5 = G3:G5. Then click on OK.

• If the button that makes the variables nonnegative is checked off, click
on it to remove the check mark. Then click on Solve.

In a flash, your spreadsheet will look like that in Table 2.7. It will report
values of 0.2, –6.4 and 7.2 in cells B7, C7, and D7.

When Premium Solver is operated as an Add-In, it is modal, which means

that you cannot do anything outside its dialog box while that dialog box is
open. Should you wish to change a datum on your spreadsheet, you need to
close the dialog box, temporarily, make the change, and then reopen it.

Using Premium Solver from the Risk Solver Platform

But when Premium Solver is operated from the Risk Solver Platform, it is
modeless, which means that you can move back and forth between Premium
Solver and your spreadsheet without closing anything down. The modeless
version can be very advantageous.

To see how to use Premium Solver from the Risk Solver Platform, begin
by reproducing Table 2.6 on a spreadsheet. Then click on the File button. A
Risk Solver Platform button will appear at the far right. Click on it. A menu
will appear. Just below the File button will be a button labeled Model. If that
button is not colored, click on it. A dialog box will appear at the right; in it,
click on the icon labeled Optimization. A dialog box identical to Figure 2.4
will appear, except that neither the variables nor the constraints will be identi-
fied.
 Chapter 2: Eric V. Denardo 59

Figure 2.4.   A Risk Solver Platform dialog box.

Making this dialog box look exactly like Figure 2.4 is not difficult. The
green Plus sign (Greek cross) just below the word “Model” is used to add
information. The red “X” to its right is used to delete information. Proceed
as follows:

• Select cells B6:D6, then click on Normal Variables, and then click on
Plus.

• Click on Normal Constraints and then click on Plus. Use the dialog box
that appears to impose the constraints E3:E5 = G3:G5.

It remains to specify the solution method you will use and to execute the
computation. To accomplish this:

• Click on Engine, which is to the right of the Model button, and select
Standard LP/Quadratic Engine.

• Click on Output, which is to the right of the Engine button. Then click
on the green triangle that points to the right.
60 Linear Programming and Generalizations

In an instant, your spreadsheet will look exactly like Table 2.7. It will ex-
hibit the solution Aâ•›=â•›0.2, Bâ•›=â•›–6.4 and Câ•›=â•›7.2.

10.  What Solver and Premium Solver Can Do

The user interfaces in Solver and in Premium Solver are so “friendly”

that it is hard to appreciate the 800-pound gorillas (software packages) that
lie behind them. The names and capabilities of these software packages have
evolved. Three of these packages are identified below:

1. The package whose name includes “LP” finds solutions to systems of

linear equations, to linear programs, and to integer programs. In newer
versions of Premium Solver, it also finds solutions to certain quadratic
programs.

2. The package whose name includes “GRG” is somewhat slower, but it

can find solutions to systems of nonlinear constraints and to nonlinear
programs, with or without integer-valued variables.

3. The package whose name includes “Evolutionary” is markedly slower,

but it can find solutions to problems that elude the other two.

Premium Solver and the versions of Solver that are in Excel 2010 and Ex-
cel 2011 include all three packages. Earlier editions of Excel include the first
two of these packages. A subsection is devoted to each.

The LP software

When solving linear programs and integer programs, use the LP soft-
ware. It is quickest, and it is guaranteed to work. If you use it with earlier
versions of Solver, remember to shift to the Options sheet and check off As-
sume Linear Model. To use it with Premium Solver as an Add-In, check off
Standard LP/Quadratic in a window on the main dialog box. The advantages
of this package are listed below:

• Its software checks that the system you claim to be linear actually is
linear – and this is a debugging aid. (Excel 2010 is equipped with a ver-
sion of Solver that can tell you what, if anything, violates the linearity
assumptions.)
 Chapter 2: Eric V. Denardo 61

• It uses an algorithm that is virtually foolproof.

• For technical reasons, it is more likely to find an integer-valued optimal

solution if one exists.

The GRG software

When you seek a solution to a system of nonlinear constraints or to an

optimization problem that includes a nonlinear objective and/or nonlinear
constraints, try the GRG (short for generalized reduced gradient) solver. It
may work. Neither it nor any other computer program can be guaranteed to
work in all nonlinear systems. To make good use of the GRG solver, you need
to be aware of an important difference between the it and the LP software:

• When you use the LP software, you can place any values you want in
the changing cells before you click on the Solve button. The values you
have placed in these cells will be ignored.

• On the other hand, when you use the GRG software, the values you
place in the changing cells are important. The software starts with the
values you place in the changing cells and attempts to improve on them.

The closer you start, the more likely the GRG software is to obtain a solu-
tion. It is emphasized:

When using the GRG software, try to “start close” by putting reasonable
numbers in the changing cells.

The multi-start feature

Premium Solver’s GRG code includes (on its options menu) a “multi-
start” feature that is designed to find solutions to problems that are not con-
vex. If you are having trouble with the GRG code, give it a try.

A quirk

The GRG Solver may attempt to evaluate a function outside the range for
which it is defined. It can attempt to evaluate the function =LN(cell) with a
negative number in that cell, for instance. Excel’s =ISERROR(cell) function
can help you to work around this. To see how, please refer to the discussion
on page 643 of Chapter 20.
62 Linear Programming and Generalizations

Numerical differentiation

It is also the case that the GRG Solver differentiates numerically; it ap-
proximates the derivative of a function by evaluating that function at a variety
of points. It is safe to use any function that is differentiable and whose deriva-
tive is continuous. Here are two examples of functions that should be avoided:

• The function =MIN(x, 6) which is not differentiable at xâ•›=â•›6.

• The function =ABS(x) which is not differentiable at xâ•›=â•›0.

If you use a function that is not differentiable, you may get lucky. And you
may not. It is emphasized:

Avoid functions that are not differentiable.

Needless to say, perhaps, it is a very good idea to avoid functions that are
not continuous when you use the GRG Solver.

The Evolutionary software

This software package is markedly slower, but it does solve problems that
elude the simplex method and the generalized reduced gradient method. Use
it when the GRG solver does not work.

The Gurobi and the SOCP software

The Risk Solver Platform includes other optimization packages. The Gu-
robi package solves linear, quadratic, and mixed-integer programs very ef-
fectively. Its name is an amalgam of the last names of the founders of Gurobi
Optimization, who are Robert Bixby, Zonghao Gu, and Edward Rothberg.
The SOCP engine quickly solves a generalization of linear programs whose
constraints are cones.

11.  An Important Add-In

The array function =PIVOT(cell, array) executes pivots. This function is

used again and again, starting in Chapter 3. The function =NL(q, μ, σ) com-
putes the expectation of the amount, if any, by which a normally distributed
 Chapter 2: Eric V. Denardo 63

random variable having μ as its mean and σ as its standard deviation exceeds
the number q. That function sees action in Chapter 7.

Neither of these functions comes with Excel. They are included in an

Add-In called OP_TOOLS. This Add-In is available at the Springer website.
You are urged to download this addend and install it in your Library before
you tackle Chapter 3. This section tells how to do that.

Begin by clicking on the Springer website for this book, which is speci-
fied on page 39. On that website, click on the icon labeled OP_TOOLS, copy
it, and paste it into a convenient folder on your computer, such as My Docu-
ments. Alternatively, drag it onto your Desktop.

What remains is to insert this Add-In in your Library and to activate it.
How to do so depends on which version of Excel you are using.

With Excel 2003

With Excel 2003, the Start button provides a convenient way to find and
open your Library folder (or any other). To accomplish this:

• Click on the Start button. A menu will pop up. On that menu, click on
Search. Then click on For Files and Folders. A window will appear. In
it, type Library. Then click on Search Now.

• After a few seconds, the large window to the right will display an icon
for a folder named Library. Click on that icon. A path to the folder that
contains your Library will appear toward the top of the screen. Click on
that path.

• You will have opened the folder that contains your library. An icon for
your Library is in that folder. Click on the icon for your Library. This
opens your Library.

With your library folder opened, drag OP_TOOLS into it. Finally, acti-
vate OP_TOOLS, as described earlier.

With Excel 2007 and Excel 2010

With Excel 2007 and Excel 2010, clicking on the Start button is not the
best way to locate your Library. Instead, open Excel. If you are using Excel
64 Linear Programming and Generalizations

2007, click on the Microsoft Office button. If you are using Excel 2010, click
on File.

Next, with Excel 2007 or 2010, click on Options. Then click on the Add-
Ins tab. In the Manage drop-down, choose Add-Ins and then click Go. Use
Browse to locate OP_TOOLS and then click on OK. Verify that OP_TOOLS
is on the Active Add-Ins list, and then click on OK at the bottom of the
window.

To make certain that OP_TOOLS is up and running, select a cell, enter

= NL(0, 0, 1) and observe that the number 0.398942 appears in that cell.

12.  Maxims for Spreadsheet Computation

It can be convenient to hide data within functions, as has been done in

Table 2.1 and Table 2.5. This can make the functions easier to read, but it is
dangerous. The functions do not appear on your spreadsheet. If you return
to modify your spreadsheet at a later time, you may not remember where you
put the data. It is emphasized:

Maxim on data: Avoid hiding data within functions. Better practice is

to place each element of data in a cell and refer to that cell.

A useful feature of spreadsheet programming is that the spreadsheet gives

instant feedback. It displays the value taken by a function as soon as you enter
it. Whenever you enter a function, use test values to check that you construct-
ed it properly. This is especially true of functions that get dragged – it is easy
to leave off a “$” sign. It is emphasized:

Maxim on debugging: Test each function as soon as you create it. If

you drag a function, check that you inserted the “$” signs where they
are needed.

The fact that Excel gives instant feedback can help you to “debug as you
go.”
 Chapter 2: Eric V. Denardo 65

13.  Review

All of the information in this chapter will be needed, sooner or later. You
need not master all of it now. You can refer back to this chapter as needed.
Before tackling Chapters 3 and 4, you should be facile with the use of spread-
sheets to solve systems of linear equations via the “standard format.” You
should also prepare to use the software on the Springer website for this book.

A final word about Excel: When you change any cell on a spreadsheet, Ex-
cel automatically re-computes the value of each function on that sheet. This
happens fast – so fast that you may not notice that it has occurred.

14.  Homework and Discussion Problems

1. Use Excel to determine whether or not 989 is a prime number. Do the

same for 991. (↜Hint: Use a “drag” to divide each of these numbers by 1, 3,
5, …, 35.)
2
2. Use Solver to find a number x that satisfies the equation x = e−x .. (↜Hint:
2
With a trial value of x in one cell, place the function e−x in another, and
ask Solver to find the value of x for which the numbers in the two cells are
equal.)

3. (↜the famous birthday problem) Suppose that each child born in 2007 (not
a leap year) was equally likely to be born on any day, independent of the
others. A group of n such children has been assembled. None of these
children are related to each other. Denote as Q(n) the probability that at
least two of these children share a birthday. Find the smallest value of n for
which Q(n)â•›>â•›0.5. Hints: Perhaps the probability P(n) that these n children
were born on n different days be found (on a spreadsheet) from the recur-
sion P(n)â•›=â•›P(n – 1) (365 – n)/365. If so, a “drag” will show how quickly
P(n) decreases as n increases.

4. For the matrices A and B in Table 2.4, compute the matrix product BA.
What happens when you ask Excel to compute (BA)–1? Can you guess
why?
66 Linear Programming and Generalizations

5. Use Solver or Premium Solver to find a solution to the system of three

equations that appears below. Hint: Use 3 changing cells and the Excel
function =LN(cell) that computes the natural logarithm of a number.

3A + 2B + 1C + 5 ln(A) = 6

2A + 3B + 2C + 4 ln(B) = 5

1A + 2B + 3C + 3 ln(C) = 4

6. Recreate Table 2.4. Replace the “0” in matrix A with a blank. What hap-
pens?
7. The spreadsheet that appears below computes 1 + 2n and 2n for various
values of n, takes the difference, and gets 1 for nâ•›≤â•›49 and gets 0 for nâ•›≥â•›50.
Why? Hint: Modern versions of Excel work with 64 bit words.

 Chapter 3: Mathematical Preliminaries

1.╅ Preview����������������������������������尓������������������������������������尓������������������������ 67
2.╅ Gaussian Operations����������������������������������尓������������������������������������尓�� 68
3.╅ A Pivot����������������������������������尓������������������������������������尓�������������������������� 69
4.╅ A Basic Variable����������������������������������尓������������������������������������尓���������� 71
5.╅ Trite and Inconsistent Equations����������������������������������尓�������������������� 72
6.╅ A Basic System����������������������������������尓������������������������������������尓������������ 74
7.╅ Identical Columns����������������������������������尓������������������������������������尓������� 76
8.╅ A Basis and its Basic Solution����������������������������������尓������������������������ 78
9.╅ Pivoting on a Spreadsheet ����������������������������������尓������������������������������ 78
10.╇ Exchange Operations����������������������������������尓������������������������������������尓�� 81
11.╇ Vectors and Convex Sets����������������������������������尓���������������������������������� 82
12.╇ Vector Spaces����������������������������������尓������������������������������������尓���������������� 87
13.╇ Matrix Notation����������������������������������尓������������������������������������尓���������� 89
14.╇ The Row and Column Spaces ����������������������������������尓������������������������ 93
15.╇ Efficient Computation* ����������������������������������尓���������������������������������� 98
16.╇ Review����������������������������������尓������������������������������������尓������������������������ 103
17.╇ Homework and Discussion Problems����������������������������������尓���������� 104

1.  Preview

Presented in this chapter is the mathematics on which an introductory

account of the simplex method rests. This consists principally of:

• A method for solving systems of linear equations that is known as

Gauss-Jordan elimination.

E. V. Denardo, Linear Programming and Generalizations, International Series 67

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_3, © Springer Science+Business Media, LLC 2011
68 Linear Programming and Generalizations

• A discussion of vector spaces and their bases.

• An introduction to terminology that is used throughout this book.

Much of the information in this chapter is familiar. Gauss-Jordan elimina-

tion plays a key role in linear algebra, as do vector spaces. In this chapter, Gauss-
Jordan elimination is described as a sequence of “pivots” that seek a solution to a
system of equations. In Chapter 4, you will see that the simplex method keeps on
pivoting, as it seeks an optimal solution to a linear program. Later in this chapter,
it is shown that Gauss-Jordan elimination constructs a basis for a vector space.

One section of this chapter is starred. That section touches lightly on ef-
ficient numerical computation, an advanced topic on which this book does
not dwell.

2.  Gaussian Operations

Gauss-Jordan elimination wrestles a system of linear equations into a

form for which a solution is obvious. This is accomplished by repeated and
systematic use of two operations that now bear Gauss’s name. These Gauss-
ian operations are:

• To replace an equation by a non-zero constant c times itself.

• To replace an equation by the sum of itself and a constant d times an-

other equation.

To replace an equation by a constant c times itself, multiply each addend

in that equation by the constant c. Suppose, for example, that the equation
2xâ•›−â•›3yâ•›=â•›6 is replaced by the constant −4 times itself. This yields the equa-
tion,â•›−â•›8x + 12yâ•›=â•›−24. Every solution to the former equation is a solution to
the latter, and conversely. In fact, the former equation can be recreated by
replacing the latter by the constant −1/4 times itself.

Both of these Gaussian operations are reversible because their effects

can be undone (reversed). To undo the effect of the first Gaussian operation,
replace the equation that it produced by the constant (1/c) times itself. To
undo the effect of the second Gaussian operation, replace the equation that it
produced by the sum of itself and the constant –d times the other equation.
Because Gaussian operations are reversible, they preserve the set of solutions
to an equation system. It is emphasized:
 Chapter 3: Eric V. Denardo 69

Each Gaussian operation preserves the set of solutions to the equation

system; it creates no new solutions, and it destroys no existing solutions.

Gauss-Jordan elimination will be introduced in the context of system (1),

below. System (1) consists of four linear equations, which have been num-
bered (1.1) through (1.4). These equations have four variables or “unknowns,”
which are x1, x2, x3, and x4. The number p that appears on the right-hand side
of equation (1.3) is a datum, not a decision variable.

(1.1) 2x1 + 4x2 − 1x3 + 8x4 = 4

(1.2) 1x1 + 2x2 + 1x3 + 1x4 = 1

(1.3) 2x3 − 4x4 = p

(1.4) −1x1 + 1x2 − 1x3 + 1x4 = 0

An attempt will be made to solve system (1) for particular values of p.

Pause to ask yourself: How many solutions are there are to system (1)? Has it
no solutions? One? Many? Does the number of solutions depend on p? If so,
how? We will find out.

3.  A Pivot

At the heart of Gauss-Jordan elimination – and at the heart of the simplex

method – lies the “pivot,” which is designed to give a variable a coefficient of
+1 in a particular equation and to give that variable a coefficient of 0 in each
of the other equations. This pivot “eliminates” the variable from all but one of
the equations. To pivot on a nonzero coefficient c of a variable x in equation
(j), execute these Gaussian operations:

• First, replace equation (j) by the constant (1/c) times itself.

• Then, for each k other than j, replace equation (k) by itself minus equa-
tion (j) times the coefficient of x in equation (k).
70 Linear Programming and Generalizations

This definition may seem awkward, but applying it to system (1) will
make everything clear. This will be done twice – first by hand, then on a
spreadsheet.

Let us begin by pivoting on the coefficient of x1 in equation (1.1). This

coefficient equals 2. This pivot executes the following sequence of Gaussian
operations:

• Replace equation (1.1) with the constant (1/2) times itself.

• Replace equation (1.2) with itself minus 1 times equation (1.1).

• Replace equation (1.3) with itself minus 0 times equation (1.1).

• Replace equation (1.4) with itself minus −1 times equation (1.1).

The first of these Gaussian operations changes the coefficient of x1 in

equation (1.1) from 2 to 1. The second of these operations changes the coef-
ficient of x1 in equation (1.2) from 1 to 0. The third operation keeps the coef-
ficient of x1 in equation (1.3) equal to 0. The fourth changes the coefficient of
x1 in equation (1.4) from −1 to 0.

This pivot transforms system (1) into system (2), below. This pivot con-
sists of Gaussian operations, so it preserves the set of solutions to system (1).
In other words, each set of values of the variables x1, x2, x3, and x4 that satis-
fies system (1) also satisfies system (2), and conversely.

(2.1) 1x1 + 2x2 − 0.5x3 + 4x4 = 2

(2.2) 1.5x3 − 3x4 = −1

(2.3) 2x3 − 4x4 = p

(2.4) 3x2 − 1.5x3 + 5x4 = 2

This pivot has eliminated the variable x1 from equations (2.2), (2.3) and
(2.4) because its coefficients in these equations equal zero.
 Chapter 3: Eric V. Denardo 71

4.  A Basic Variable

A variable is said to be basic for an equation if its coefficient in that equa-

tion equals 1 and if its coefficients in the other equations equal zero. The
pivot that has just been executed made x1 basic for equation (2.1), exactly as
planned. It is emphasized:

A pivot on a nonzero coefficient of a variable in an equation makes that

variable basic for that equation.

The next pivot will occur on a nonzero coefficient in equation (2.2). The
variables x3 and x4 have nonzero coefficients in this equation. We could pivot
on either. Let’s pivot on the coefficient of x3 in equation (2.2). This pivot con-
sists of the following sequence of Gaussian operations:

• Replace equation (2.2) by itself divided by 1.5.

• Replace equation (2.1) by itself minus −0.5 times equation (2.2).

• Replace equation (2.3) by itself minus 2 times equation (2.2).

• Replace equation (2.4) by itself minus −1.5 times equation (2.2).

These Gaussian operations transform system (2) into system (3), below.
They create no solutions and destroy none.

(3.1) 1x1 + 2x2 + 3x4 â•›= 5/3

(3.2) + 1x3 – 2x4 = –2/3

(3.3) â•›0x4 = p + 4/3

(3.4) 3x2 â•›+ 2x4 = 1

This pivot made x3 basic for equation (3.2). It kept x1 basic for equation
(3.1). That is no accident. Why? The coefficient of x1 in equation (2.2) had
been set equal to zero, so replacing another equation by itself less some con-
72 Linear Programming and Generalizations

stant times equation (2.2) cannot change its coefficient of x1 . The property
that this illustrates holds in general. It is emphasized:

Pivoting on a nonzero coefficient of a variable x in an equation has these

effects:

•â•‡The variable x becomes basic for the equation that has the coef-
ficient on which the pivot occurred.

•â•‡Any variable that had been basic for another equation remains ba-
sic for that equation.

5.  Trite and Inconsistent Equations

The idea that motivates Gauss-Jordan elimination is to keep pivoting un-

til a basic variable has been created for each equation. There is a complica-
tion, however, and it is now within view. Equation (3.3) is

0x1 + 0x2 + 0x3 + 0x4 = p + 4/3.

Let us recall that p is a datum (number), not a decision variable. It is clear

that equation (3.3) has a solution if p = −4/3 and that it has no solution if
p  = −4/3. This motivates a pair of definitions. The equation

0x1 + 0x2 + · · · + 0xn = d

is said to be trite if d = 0. The same equation is said to be inconsistent if

d  = 0. A trite equation poses no restriction on the values taken by the vari-
ables. An inconsistent equation has no solution.

Gauss-Jordan elimination creates no solutions and destroys none. Thus,

if Gauss-Jordan elimination produces an inconsistent equation, the original
equation system can have no solution. In particular, system (1) has no solu-
tion if p = −4/3.

For the remainder of this section, it is assumed that pâ•›=â•›−4/3. In this case,
equations (3.1) and (3.2) have basic variables, and equation (3.3) is trite.
Gauss-Jordan elimination continues to pivot, aiming for a basic variable for
each non-trite equation. Equation (3.4) lacks a basic variable. The variables x2
 Chapter 3: Eric V. Denardo 73

and x4 have nonzero coefficients in equation (3.4). Either of these variables

could be made basic for that equation. Let’s make x2 basic for equation (3.4).
That is accomplished by executing this sequence of Gaussian operations:

• Replace equation (3.4) by itself divided by 3.

• Replace equation (3.1) by itself minus 2 times equation (3.4).

• Replace equation (3.2) by itself minus 0 times equation (3.4).

• Replace equation (3.3) by itself minus 0 times equation (3.4).

This pivot transforms system (3) into system (4).

(4.1) 1x1 + (5/3)x4 ╛╛= 1

(4.2) + 1x3 – 2x4 = –2/3

(4.3) 0x4 â•›= 0

(4.4) + 1x2 + (2/3)x4 â•›= 1/3

In system (4), each non-trite equation has been given a basic variable. A
solution to system (4) is evident. Equate each basic variable to the right-hand-
side value of the equation for which it is basic, and equate any other variables
to zero. That is, set:

x1 = 1, x3 = –2/3, x2 = 1/3, x4 = 0.

These values of the variables satisfy system (4), hence must satisfy sys-
tem (1).

More can be said. Shifting the non-basic variable x4 to the right-hand

side of system (4) expresses every solution to system (4) as a function of x4 .
Specifically, for each value of x4 , setting

(5.1) x1 = 1 – (5/3)x4,

(5.2) x3 = –2/3 + â•›2x4,

(5.4) x2 = 1/3 – (2/3)x4,

74 Linear Programming and Generalizations

satisfies system (4) and, consequently, satisfies system (1). By the way, the
question posed earlier can now be answered: If p ≠â•›−â•›4/3. system (1) has no
solution, and if pâ•›=â•›−4/3, system (1) has infinitely many solutions, one for
each value of x4 .

The dictionary

System (5) has been written in a format that is dubbed the dictionary
because:

• Each equation has a basic variable, and that basic variable is the sole
item on the left-hand side of the equation for which it is basic.

• The nonbasic variables appear only on the right-hand sides of the equa-
tions.

In Chapter 4, the dictionary will help us to understand the simplex

method.

Consistent equation systems

An equation system is said to be consistent if it has at least one solution

and to be inconsistent if it has no solution. It has been demonstrated that if
an equation system is consistent, Gauss-Jordan elimination constructs a so-
lution. And if an equation system is inconsistent, Gauss-Jordan elimination
constructs an inconsistent equation.

6.  A Basic System

With system (4) in view, a key definition is introduced. A system of lin-

ear equations is said to be basic if each equation either is trite or has a basic
variable. System (4) is basic because equation (4.3) is trite and the remaining
three equations have basic variables.

Basic solution

A basic equation system’s basic solution equates each non-basic variable

to zero and equates each basic variable to the right-hand-side value of the
equation for which it is basic. The basic solution to system (4) is:

x1 = 1, x3 = −2/3, x2 = 1/3, x4 = 0.
 Chapter 3: Eric V. Denardo 75

Recap of Gauss-Jordan elimination

Gauss-Jordan elimination pivots in search of a basic system, like so:

Gauss-Jordan elimination╇ While at least one non-trite equation lacks

a basic variable:

1.╇Select a non-trite equation that lacks a basic variable. Stop if this

equation is inconsistent.

2.╇Else select any variable whose coefficient in this equation is non-

zero, and pivot on it.

When Gauss-Jordan elimination is executed, each pivot creates a basic

variable for an equation that lacked one. If Gauss-Jordan elimination stops at
Step 1, an inconsistent equation has been identified, and the original equation
system can have no solution. Otherwise, Gauss-Jordan elimination constructs
a basic system, one whose basic solution satisfies the original equation system.

A coarse measure of work

A coarse measure of the effort needed to execute an algorithm is the

number of multiplications and divisions that it entails. Let’s count the number
of multiplications and divisions needed to execute Gauss-Jordan elimination
on a system of m linear equations in n decision variables.

Each equation has n + 1 data elements, including its right-hand side val-
ue. The first Gaussian operation in a pivot divides an equation by the coef-
ficient of one of its variables. This requires n divisions (not n + 1) because it
is not necessary to divide a number by itself. Each of the remaining Gaussian
operations in a pivot replaces an equation by itself less a particular constant d
times another equation. This requires n multiplications (not n+1) because it
is not necessary to compute dâ•›−â•›dâ•›=â•›0. We’ve seen that each Gaussian operation
in a pivot requires n multiplications or divisions. Evidently:

• Each Gaussian operation entails n multiplications or divisions.

• Each pivot entails m Gaussian operations, one per equation, for a total
of m n multiplications and divisions per pivot.

• Gauss-Jordan elimination requires as many as m pivots, for a total of

m2 n multiplications and divisions.
76 Linear Programming and Generalizations

In brief:

Worst-case work: Executing Gauss-Jordan elimination on a system of

m linear equations in n unknowns requires as many as m2 n multiplica-
tions and divisions.

Doubling m and n multiplies the work bound m2 n by 23 = 8. Evi-

dently, the worst-case work bound grows as the cube of the problem size. That
is not good news. Fortunately, as linear programs get larger, they tend to get
sparser (have a higher percentage of 0’s), and sparse-matrix techniques help
to make large problems tractable. How that occurs is discussed, briefly, in the
starred section of this chapter.

7.  Identical Columns

A minor complication has been glossed over: a basic system can have
more than one basic solution. To indicate how this can occur, consider system
(6), below. It differs from system (1) in that p equals −4/3 and in that it has a
fifth decision variable, x5 , whose coefficient in each equation equals that of
x2 .

(6.1) 2x1 + 4x2 − 1x3 + 8x4 + 4x5 = 4

(6.2) 1x1 + 2x2 + 1x3 + 1x4 + 2x5 = 1

2x3 − 4x4 = −4/3
(6.3) 2x3 − 4x4 = −4/3

(6.4) −1x1 + 1x2 − 1x3 + 1x4 + 1x5 = 0

From a practical viewpoint, the variables x2 and x5 are indistinguish-

able; either can substitute for the other, and either can be eliminated. But let’s
see what happens if we leave both columns in and pivot as before. The first
pivot makes x1 basic for equation (6.1). This pivot begins by replacing equa-
tion (6.1) by itself times (1/2). Note that the coefficients of x2 and x5 in this
equation remain equal; they started equal, and both were multiplied by (1/2).
The next step in this pivot replaces equation (6.2) by itself less equation (6.1).
 Chapter 3: Eric V. Denardo 77

The coefficients of x2 and x5 in equation (6.2) remain equal. And so forth.

A general principle is evident. It is that:

Identical columns stay identical after executing any number of Gaussian

operations.

As a consequence, applying to system (6) the same sequence of Gaussian

operations that transformed system (1) into system (4) produces system (7),
below. System (7) is identical to system (4), except that the coefficient of x5 in
each equation equals the coefficient of x2 in that equation.

(7.1) 1x1 + (5/3)x4 â•› = 1

(7.2) + 1x3 – 2x4 â•›= – 2/3

(7.3) 0x4 â•›= 0

(7.4) + 1x2 + (2/3)x4 + 1x5 = 1/3

The variables x2 and x5 are basic for equation (7.4). When x2 became
basic, x5 also became basic. Do you see why?

System (7) has two basic solutions. One basic solution corresponds to
selecting x2 as the basic variable for equation (7.4), and it sets

(8) x1 = 1, x2 = 1/3, x3 = −2/3, x4 = 0, x5 = 0.

The other basic solution corresponds to selecting x5 as the basic variable

for equation (7.4), and it sets

(9) x1 = 1, x2 = 0, x3 = −2/3, x4 = 0, x5 = 1/3.

This ambiguity is due to identical columns. Gaussian operations are re-

versible, so columns that are identical after a Gaussian operation occurred
must have been identical before it occurred. Hence, distinct columns stay dis-
tinct. In brief:
78 Linear Programming and Generalizations

If an equation has more than one basic variable, two or more variables
in the original system had identical columns of coefficients, and all of
them became basic for that equation.

The fact that identical columns stay identical is handy – in later chapters,
it will help us to understand the simplex method.

8.  A Basis and its Basic Solution

Consider any basic system. A set of variables is called a basis if this set
consists of one basic variable for each equation that has a basic variable. Sys-
tem (4) has one basis, which is the set {x1 , x3 , x2 } of variables. System (7) has
two bases. One of these bases is the set {x1 , x3 , x2 } of variables. The other
basis is {x1 , x3 , x5 }.

Again, consider any basic system. Each basis for it has a unique basic
solution, namely, the solution to the equation system in which each nonbasic
variable is equated to zero and each basic variable is equated to the right-
hand-side value of the equation for which it is basic. System (7) is basic. It has
two bases and two basic solutions; equation (8) gives the basic solution for the
basis {x1 , x3 , x2 }, and (9) gives the basic solution for the basis {x1 , x3 , x5 }.

The terms “basic variable,” “basis,” and “basic solution” suggest that a ba-
sis for a vector space lurks nearby. That vector space is identified later in this
chapter.

9.  Pivoting on a Spreadsheet

Pivoting by hand gets old fast. Excel can do the job flawlessly and pain-
lessly. This section tells how.

A detached-coefficient tableau

The spreadsheet in Table 3.1 will be used to solve system (1) for the case
in which pâ•›=â•›−4/3. Rows 1 through 5 of Table 3.1 are a detached-coefficient
tableau for system (1). Note that:
 Chapter 3: Eric V. Denardo 79

• Each variable has a column heading, which is recorded in row 1.

• Rows 2 through 5 contain the coefficients of the equations in system

(1), as well as their right-hand-side values.

• The “=” signs have been omitted.

Table 3.1.↜  Detached coefficient tableau for system (1) and the first pivot.

The first pivot

This spreadsheet will be used to execute the same sequence of pivots as

before. The first of these pivots will occur on the coefficient of x1 in equation
(1.1). This coefficient is in cell B2 of Table 3.1. Rows 7 though 11 display the
result of that pivot. Note that:

• Row 7 equals row 2 multiplied by (1/2).

• Row 8 equals row 3 less 1 times row 7.

• Row 9 equals row 4 less 0 times row 7.

• Row 10 equals row 5 less −1 times row 7.

Excel functions could be used to create rows 7-10. For instance, row 7
could be obtained by inserting in cell B7 the function =B2/$B2 and dragging
it across the row. Similarly, row 8 could be obtained by inserting in cell B8
the function =B3â•›−â•›$B3 * B$7 and dragging it across the row. But there is an
easier way.
80 Linear Programming and Generalizations

An Add-In

As Table 3.1 suggests, the array function =pivot(cell, array) executes this

pivot. The easy way to replicate rows 7-10 of Table 3.1 is as follows:

• Select the array B7:F10 (This causes the result of the pivot to appear in
cells B7 through F10.)

• Type =pivot(B2, B2:F5) to identify B2 as the pivot element and B2:F5

as the array of coefficients on which the pivot is to occur.

• Type Ctrl+Shift+Enter to remind Excel that this is an array function.

(It is an array function because it places values in a block (array) of
cells, rather than in a single cell.)

The function =pivot(cell, array) makes short work of pivoting. This

function does not come with Excel, however. It is an Add-In. It is included
in the software that accompanies this text, where it is one of the functions in
Optimization Tools. Before you can use it, you must install it in your Excel
Library and activate it. Chapter 2 tells how to do that.

The second and third pivots

Table 3.2 reports the result of executing two more pivots with the same
array function.

Table 3.2.↜  Two further pivots on system (1).

 Chapter 3: Eric V. Denardo 81

To execute these two pivots:

• Select the block B12:F15 of cells, type =pivot(D8, B7:F10) and then hit
Ctrl+Shift+Enter

• Select the block B17:F20 of cells, type =pivot(C15, B12:F15) and then
hit Ctrl+Shift+Enter

Rows 17-20 report the result of these pivots. The data in rows 17-20 are
identical to those in system (4) with pâ•›=â•›−4/3. In particular:

• The variable x1 has been made basic for equation (4.1).

• The variable x3 has been made basic for equation (4.2).

• Equation (4.3) has become trite.

• The variable x2 has been made basic for equation (4.4).

Pivoting with an Add-In is easy and is error-proof. It has an added advan-

tage – it re-executes the pivot sequence after each change in a datum. The mo-
ment you change a value in cells B2:F5 of the spreadsheet in Table 3.1, Excel
re-executes the pivot sequence, and it does so with blazing speed.

10.  Exchange Operations

Many presentations of Gauss-Jordan elimination include four Gaussian

operations, of which only two have been presented. The other Gaussian op-
erations are called exchange operations, and they appear below:

• Exchange the positions of a pair of equations.

• Exchange the positions of a pair of variables.

Like the others, these exchange operations can be undone. To recover the
original equation system after doing an exchange operation, simply repeat it.

The exchange operations do not help us to construct a basis. They do

serve a “cosmetic” purpose. They let us state results in simple language. For
instance, the exchange operations let us place the basic variables on the diago-
nal and the trite equations at the bottom. To illustrate, reconsider Table 3.2.
82 Linear Programming and Generalizations

Exchanging rows 19 and 20 shifts the trite equation to the bottom. Then, ex-
changing columns C and D puts the basic variables on the diagonal.

In linear algebra, the two Gaussian operations that were introduced ear-
lier and the first of the above two exchange operations are known as elemen-
tary row operations. Most texts on linear algebra begin with a discussion
of elementary row operations and their properties. That’s because Gaussian
operations are fundamental to linear algebra.

11.  Vectors and Convex Sets

Modern computer codes solve linear systems that have of hundreds or

thousands of equations, as does the simplex method. These systems are im-
possible to visualize. Luckily, the intuition obtained from 2-dimensional and
3-dimensional geometry holds up in higher dimensions. It provides insight as
to what’s going on. This section probes the relevant geometry, as it applies to
vectors and convex sets. Much of this section may be familiar, but you might
welcome a review.

Vectors

A linear program has some number n of decision variables, and n may be

large. An ordered set x = (x1 , x2 , . . . , xn ) of values of these decision vari-
ables is called a vector or an n-vector, the latter if we wish to record the num-
ber of entries in it.

Similarly, the symbol n denotes the set of all n-vectors, namely, the set
that consists of each vector x = (x1 , x2 , . . . , xn ) as x1 through xn vary, in-
dependently, over the set  of all real numbers. This set n of all n-vectors is
known as n-dimensional space or, more succinctly, as n-space. The n-vector
xâ•›=â•›(0, 0, …, 0) is called the origin of n .

Relax! There will be no need to visualize higher-dimensional spaces be-

cause we can proceed by analogy with plane and solid geometry. Figure 3.1 is
a two-dimensional example. In it, the ordered pair xâ•›=â•›(5, 1) of real numbers
is located five units to the right of the origin and 1 unit above it. Also, the
ordered pair yâ•›=â•›(−2, 3) is located two units to the left of the origin and three
units above it.
 Chapter 3: Eric V. Denardo 83

Figure 3.1.↜  The vectors xâ•›=â•›(5, 1) and yâ•›=â•›(−2, 3) and their sum.

x + y = (3, 4)
y = (–2, 3)

3 x = (5, 1)

–2 0 3 5

Vector addition

Let x = (x1 , x2 , . . . , xn ) and

x=y =(x(y
1 ,1,xâ•›y22,, . . . , yxnn))be two n-vectors. The
sum, x + y, of the vectors x and y is defined by

(10) x + y = (x1 + y1 , x2 + y2 , . . . , xn + yn ).

Vector addition is no mystery: simply add the components. This is true of

vectors in 2 , in 3 , and in higher-dimensional spaces. Figure 3.1 depicts the
sum x + y of the vectors xâ•›=â•›(5, 1) and yâ•›=â•›(−2, 3). Evidently,

(5, 1) + (−2, 3) = (5 − 2, 1 + 3) = (3, 4).

The gray lines in Figure 3.1 indicate that, graphically, to take the sum of
the vectors (5, 1) and (−2, 3), we can shift the “tail” of either vector to the head
of the other, while preserving the “length” and “direction” of the vector that is
being shifted.

Scalar multiplication

If x = (x1 , x2 , . . . , xn ) is a vector and if c is a real number, the scalar

multiple of x and c is defined by

(11) cxx ==(cx

(x11, cx
x22,, . . . , cx
xnn)).
84 Linear Programming and Generalizations

Evidently, to multiply a vector x by a scalar c is to multiply each compo-

nent of x by  c. This scalar c can be any real number – positive, negative or zero.

What happens when the vector x in Figure 3.1 is multiplied by the scalar
câ•›=â•›0.75? Each entry in x is multiplied by 0.75. This reduces the length of the
vector x without changing the direction in which it points.

What happens when the vector x is multiplied by the scalar câ•›=â•›−1? Each
entry in x is multiplied by −1. This reverses the direction in which x points,
but does not change its length.

With y as a vector, the scalar product (−1) y is abbreviated as – y. With x

and y as two vectors that have the same number n of components, the differ-
ence, xâ•›−â•›y is given by

x – y = x + (–1)y = (x1 x– =
y1,(xx12 ,–xy22,, . . . , xxnn )– yn).

Displayed in Figure 3.2 is the difference xâ•›−â•›y of the vectors xâ•›=â•›(5, 1) and

yâ•›=â•›(−2, 3). These two vectors have xâ•›−â•›yâ•›=â•›(5, 1)â•›−â•›(−2, 3)â•›=â•›(7, −2).

Figure 3.2.↜  The vectors xâ•›=â•›(5, 1) and yâ•›=â•›(−2, 3) and their difference.

y = (–2, 3)

3 x = (5, 1)

–2 0 3 5 7

–2 x – y = (7, –2)

Convex combinations and intervals

Let xâ•›=â•›(x1 , x2 , . . . , xn ) and yâ•›=â•›(y1 , y2 , . . . , yn ) be two n-vectors and

let c be a number that satisfies 0â•›≤â•›câ•›≤â•›1. The vector
 Chapter 3: Eric V. Denardo 85

cx + (1 − c)y

is said to be a convex combination of the vectors x and y. Similarly, the inter-

val between x and y is the set S of n-vectors that is given by

(12) S = {cx + (1 − c)y : 0 ≤ c ≤ 1}.

Here and hereafter, a colon within a mathematical expression is read as

“such that.” Equation (12) defines the interval S as the set of all convex com-
binations of x and y. Figure 3.3 illustrates these definitions.

Figure 3.3.↜  The thick gray line segment is the interval between xâ•›=â•›(5, 1)
and yâ•›=â•›(−2, 3).

c=0
c = 1/4
c = 1/2
3 c = 3/4
c=1

y = (–2, 3) 1
x = (5, 1)

–2 0 5 7

–2 x – y = (7, –2)

Each convex combination of the vectors x and y that are depicted in Fig-
ure 3.3 can be written as

(13) cx + (1 − c)y = cx + y − cy = y + c(x − y),

where c is a number that lies between 0 and 1, inclusive. Evidently, the interval
between x and y consists of each vector y + c(xâ•›−â•›y) obtained by adding y and
the vector c(xâ•›−â•›y) as c varies from 0 to 1. Figure 3.3 depicts y + c(xâ•›−â•›y) for the
values câ•›=â•›0, 1/4, 1/2, 3/4 and 1.
86 Linear Programming and Generalizations

By the way, if x and y are distinct n-vectors, the line that includes x and y
is the set L that is given by

(14) L = {cx + (1 − c)y : c ∈ }.

This line includes x (take câ•›=â•›1) and y (take câ•›=â•›0), it contains the interval
between x and y, and it extends without limit in both directions.

Convex sets

A set C of n-vectors is said to be convex if C contains the interval between

each pair of vectors in C. Figure 3.4 displays eight shaded subsets of 2 (the
plane). The top four are convex. The bottom four are not. Can you see why?

Figure 3.4↜.  Eight subsets of the plane.

Convex sets will play a key role in linear programs and in their general-
izations. A vector x that is a member of a convex set C is said to be an extreme
point of C if x is not a convex combination of two other vectors in C. Read-
ing from left to right, the four convex sets in Figure 3.4 have infinitely many
extreme points, three extreme points, no extreme points, and two extreme
points. Do you see why?

Unions and intersections

Let S and T be subsets of n . The union S ∪ T is the set of n-vectors that

consists of each vector that is in S, or is in T or is in both. The intersection
S ∩ T is the subset of n that consists of each vector that is in S and is in T. It’s
easy to convince oneself visually (and to prove) that:
 Chapter 3: Eric V. Denardo 87

• The union S ∪ T of convex sets need not be convex.

• The intersection S ∩ T of convex sets must be convex.

Linear constraints

Let us recall from Chapter 1 that each constraint in a linear program re-
quires a linear expression to bear one of three relationships to a number, these
three being “=”, “≤”, and “≥.” In other words, with a0 through an as fixed
numbers and x1 through xn as decision variables, each constraint takes one
of these forms:

a1 x1 + a1 x2 + · · · + an xn = a0

a1 x1 + a1 x2 + · · · + an xn ≤ a0

a1 x1 + a1 x2 + · · · + an xn ≥ a0

It’s easy to check that the set of n-vectors xâ•›=â•›(x1, x2, …, xn) that satisfy a
particular linear constraint is convex. As noted above, the intersection of con-
vex sets is convex. Hence, the set of vectors xâ•›=â•› (x1 , x2 , · · · , xn ) that satisfy
all of the constraints of a linear program is convex. It is emphasized:

The set of vectors that satisfy all of the constraints of a linear program
is convex.

Convex sets play a crucial role in linear programs and in nonlinear pro-
grams.

12.  Vector Spaces

The introduction to linear programming in Chapter 4 does not require

an encyclopedic knowledge of vector spaces. It does use the information that
is presented in this section and in the next two. A set V of n-vectors is called
a vector space if:
88 Linear Programming and Generalizations

• V is not empty.

• The sum of any two vectors in V is also in V.

• Each scalar multiple of each vector in V is also in V.

Each vector space V must contain the origin; that is so because V must
contain at least one vector x and because it must also contain the scalar 0
times x, which is the origin. Each vector space is a convex set. Not every con-
vex set is a vector space, however.

Geometric insight

It’s clear, visually, that the subsets V of 2 (the plane) that are vector
spaces come in these three varieties:

• The set V whose only member is the origin is a vector space.

• Any line that passes through the origin is a vector space.

• The plane is itself a vector space.

Ask yourself: Which subsets of 3 are vector spaces?

Linear combinations

Let v1 through vK be n-vectors, and let c1 through cK be scalars (num-

bers); the sum,

(15) c1 v1 + c2 v2 + · · · + cK vK ,

is said to be a linear combination of the vectors v1 through vK . Evidently, a

linear combination of K vectors multiplies each of them by a scalar and takes
the sum.

Linearly independent vectors

The set {v1 , v2 , . . . , vK } of n-vectors are said to be linearly indepen-

dent if the only solution to

(16) 0 = c1 v1 + c2 v2 + · · · + cK vK

is 0 = c1 = c2 = · · · = cK . In other words, the n-vectors v1 through vK are

linearly independent if the only way to obtain the vector 0 as a linear com-
 Chapter 3: Eric V. Denardo 89

bination of these vectors is to multiply each of them by the scalar 0 and then
add them up.

Similarly, the set {v1 , v2 , . . . , vK } of n-vectors is said to be linearly de-

pendent if these vectors are not linearly independent, equivalently, if a solu-
tion to (16) exists in which not all of the scalars equal zero. Convince yourself,
visually, that:

• Any set of n-vectors that includes the origin is linearly dependent.

• Two n-vectors are linearly independent if neither is a scalar multiple of

the other.

• In the plane, 2 , every set of three vectors is linearly dependent.

A set {v1 , v2 , . . . , vK } of vectors in a vector space V is said to span V if

every vector in V is a linear combination of these vectors.

A basis

Similarly, a set {v1 , v2 , . . . , vK } of vectors in a vector space V is said to

be a basis for V if the vectors v1 through vK are linearly independent and if
every element of V is a linear combination of this set {v1 , v2 , . . . , vK } of
vectors.

Trouble?

A basis has just been defined as a set of vectors. Earlier, in our discussion
of Gauss-Jordan elimination, a basis had been defined as a set of decision
variables. That looks to be incongruous, but a correspondence will soon be
established.

13.  Matrix Notation

It will soon be seen that Gauss-Jordan elimination constructs a basis for

the “column space” of a matrix. Before verifying that this is so, we interject a
brief discussion of matrix notation.

In the prior section, the entries in the n-vector xâ•›=â•›(x1, x2, …, xn) could
have been arranged in a row or in a column. When doing matrix arithmetic,
it is necessary to distinguish between rows and columns.
90 Linear Programming and Generalizations

Matrices

A “matrix” is a rectangular array of numbers. Whenever possible, capital

letters are used to represent matrices. Depicted below is an m × n matrix A.
Evidently, the integer m is the number of rows in A, the integer n is the num-
ber of columns, and Aij is the number at the intersection of the ith row and jth
column of A.
 
A11 A12 · · · A1n
 A21 A22 · · · A2n 
 
(17) A= .
 
 .. .. .. 
 . . 

Am1 Am2 · · · Amn

Throughout, when A is an m × n matrix, Aj denotes the jth column of A

and Ai denotes the ith row of A.

A1j
 
 A2j 
Aj =  , Ai = [Ai1 Ai2 · · · Ain ]
 
..
 . 
Amj

Matrix multiplication

This notation helps us to describe the product of two matrices. To see

how, let E be a matrix that has r columns and let F be a matrix that has r rows.
The matrix product E F can be taken, and the ijth element (EF)ij of this matrix
product equals the sum over k of Eik Fkj . In other words,
r
(18) (EF)ij = Eik Fkj = Ei Fj for each i and j.
k=1

Thus, the ijth element of the matrix (EF) equals the product Ei Fj of the ith
row of E and the jth column of F.

Similarly, the ith row (EF)i of EF and jth column (EF)j of EF are given by

(19) (EF)i = Ei F,

(20) (EF)j = EFj .

 Chapter 3: Eric V. Denardo 91

It is emphasized:

The ith row of the matrix product (EF) equals EiF and the jth column of
this matrix product equals EFj

Vectors

In this context, a vector is a matrix that has only one row or only one
column. Whenever possible, lower-case letters are used to represent vectors.
Displayed below are an n × 1 vector x and an m × 1 vector b.

x1 b1
   
x2   b2 
x =  . , b= . 
   
 ..   .. 
xn bm

Evidently, a single subscript identifies an entry in a vector; for instance,

xj is entry in row j of x.

The equation Axâ•›=â•›b

A system of m linear equations in n unknowns is written succinctly as

Axâ•›=â•›b. Here, the decision variables are x1 through xn , the number Aij is the
coefficient of xj in the ith equation, and the number bi is the right-hand side
value of the ith equation.

The matrix equation Axâ•›=â•›b appears repeatedly in this book. As a memory

aide, the following conventions are employed:

• The data in the equation Axâ•›=â•›b are the m × n matrix A and the m × 1
vector b.

• The decision variables (unknowns) in this equation are arrayed into the
n × 1 vector x.

In brief, the integer m is the number of rows in the matrix A, and the
integer n is the number of columns. Put another way, the matrix equation
Axâ•›=â•›b is a system of m equations in n unknowns.
92 Linear Programming and Generalizations

The matrix product Ax

When the equation Axâ•›=â•›b is studied, the matrix product Ax is of particu-

lar importance. Evidently, Ax is an m × 1 vector. Expression (19) with Eâ•›=â•›A
and Fâ•›=â•›x confirms that the ith element of Ax equals Ai x, indeed, that

A11 x1 + A12 x2 + · · · + A1n xn

 
 A21 x1 + A22 x2 + · · · + A2n xn 
(21) Ax =  .
 
.. .. ..
 . . . 
Am1 x1 + Am2 x2 + · · · + Amn xn

Note in expression (21) that the number (scalar) x1 multiplies each entry
in A1 (the 1st column of A), that the scalar x2 multiplies each entry in A2 , and
so forth. In other words,

(22) Ax = A1 x1 + A2 x2 + · · · + An xn .

Equation (22) interprets Ax as a linear combination of the columns of A.

It is emphasized:

The matrix product Ax is a linear combination of the columns of A. In

particular, the scalar xj multiplies Aj.

You may recall that the “column space” of a matrix A is the set of all linear
combinations of the columns of A; we will get to that shortly.

The matrix product yA

Let y be any 1 × m vector. Since A is an m × n matrix, the matrix prod-

uct yA can be taken. Equation (20) shows that the jth entry in yA equals yAj .
In other words,

(23) yâ•›Ax==
(yâ•›A
(x1,1 , yâ•›xA2 ,2 . . . , yâ•›
xAn )n).

Just as Ax is a linear combination of the columns of A, the matrix product

yA is a linear combination of the rows of A, one in which y1 multiplies each
element of A1, in which y2 multiplies each element of A2 , and so forth.

(24) yA = y1 A1 + y2 A2 + · · · + ym Am
 Chapter 3: Eric V. Denardo 93

In brief:

The matrix product yA is a linear combination of the rows of A. In par-

ticular, the scalar yi multiplies Ai.

An ambiguity

When A is a matrix, two subscripts denote an entry, a single subscript

denotes a column, and a single superscript denotes a row. The last of these
conventions must be taken with a grain of salt; “T” abbreviates “transpose,”
and AT denotes the transpose of the matrix A, not its Tth row.

14.  The Row and Column Spaces

Let A be an m × n matrix. For each n × 1 vector x, equation (22) inter-

prets the matrix product Ax as a linear combination of the columns of A. The
set Vc that is specified by the equation,

(25) Vc = {Ax : x ∈ n×1 },

is called the column space of the matrix A. Equation (25) reads, “ Vc equals
the set that contains Ax for every n × 1 vector x.” It is clear from equation
(22) that Vc is the set of all linear combinations of the columns of the matrix
A, moreover, that Vc is a vector space.

With A as an m × n matrix and with y as a 1 × m vector, equation (24)

interprets yA as a linear combination of the rows of A. The set Vr that is
specified by the equation,

(26) Vr = {yA : y ∈ 1 × m },

is called the row space of A. Evidently, Vr is the set of all linear combinations
of the rows of the matrix A, and it too is a vector space.

A basis for the column space

Gauss-Jordan elimination can be used to construct a basis for the col-

umn space of a matrix. In fact, Gauss-Jordan elimination has been used to
94 Linear Programming and Generalizations

construct a basis for the column space of the 4 × 4 matrix A that is given
by

2 4 −1 8
 
 1 2 1 1
(27) A= .
 0 0 2 −4
−1 1 −1 1

Let us see how. With A given by (27) and with x as a 4 × 1 vector, equa-
tion (22) shows the matrix product A x is this linear combination of the col-
umns of A.

2 4 −1 8
       
 1 2   1  1
(28) Ax = 
 0 x1 + 0 x2 +  2 x3 + −4 x4 .
      

−1 1 −1 1

Please observe that (28) is identical to the left-hand side of system (1).

A homogeneous equation

The matrix equation Axâ•›=â•›b is said to be homogeneous if its right-hand-

side vector b consists entirely of 0’s. With Ax given by (28), let us study solu-
tions x to the (homogeneous) equation Axâ•›=â•›0. This equation appears below
as

2 4 −1 8 0
         
 1 2  1  1 0
(29)   x1 +   x2 +   x3 +   x4 =   ,
 0 0  2 −4 0
−1 1 −1 1 0

No new work is needed to identify the solutions to (29). To see why, re-
place the right-hand side values of system (1) by 0’s and repeat the Gaussian
operations that transformed system (1) into system (4), getting:

1 0 0 5/3 0
         
0  0 1  −2  0
(30)   x1 +   x2 +   x3 +   x4 =   .
0  0 0  0  0
0 1 0 2/3 0
 Chapter 3: Eric V. Denardo 95

These Gaussian operations preserve the set of solutions to the equation

system; the scalars x1 through x4 satisfy (29) if and only if they satisfy (30).
From this fact, we conclude that:

• The columns of A are linearly dependent because (30) is satisfied by

equating x4 to any nonzero number and setting

x1 = –(5/3)x4, x3 = 2x4, x2 = –(2/3)x4.

• The columns A1 , A2 , and A3 are linearly independent because

setting x4 = 0 in (29) and (30) shows that the only solution to
A1 x1 + A2 x2 + A3 x3 = 0 is x1 x=2 x=
2 =xx = 0x0 .
3 3=

• The vector A4 aAlinear

1 combination of A1 , A2 ,and A3 because applying
the same sequence of Gaussian operations to the system

2 4 −1 8
       
1 2  1  1
  x1 +   x 2 +   x3 =  
0 0  2 −4
−1 1 −1 1

transforms it into

1 0 0 5/3
       
0  0 1  −2
  x1 +   x2 +   x3 =   ,
0  0 0  0
0 1 0 2/3

which demonstrates that A4 = (5/3)A1 + (2/3)A2 − 2A3 .

These observations imply that the set {A1 , A2 , A3 } of vectors is a basis

for the column space of A. This is so because the vectors A1 , A2 and A3 are
linearly independent and because A4 is a linear combination of them, which
guarantees that every linear combination of A1 through A4 can be expressed
as a linear combination of A1 , A2 and A3 . The same line of reasoning works
for every matrix. It is presented as:

Proposition 3.1 (basis finder).╇ Consider any matrix A. Apply Gauss-

Jordan elimination to the equation Axâ•›=â•›0 and, at termination, denote as
96 Linear Programming and Generalizations

{Aj : j ∈ C} the set of columns on which pivots have occurred. This set
{Aj : j ∈ C} of columns is a basis for the column space of A.

Proof.╇ This application of Gauss-Jordan elimination cannot terminate

with an inconsistent equation because setting xâ•›=â•›0 produces a solution to
Axâ•›=â•›0. It must terminate with a basic solution. Denote as {Aj : j ∈ C} the
set of columns that on which pivots have occurred.

The analog of (30) indicates that the set {Aj : j ∈ C} of columns must
be linearly independent and that each of the remaining columns must be a
linear combination of these columns. Thus, the set {Aj : j ∈ C} of columns
span the column space of A, which completes a proof. 

Reconciliation

Early in this chapter, Gauss-Jordan elimination had been used to trans-

form system (1) into system (4). Let us recall that system (4) is basic, specifi-
cally, that the set {x1 , x2 , x3 } of decision variables is a basis for system (4).

In the current section, the same Gauss-Jordan procedure has been used to
identify the set {A1 , A2 , A3 } of columns as a basis for the column space of A.
These are two different ways of making the same statement. It is emphasized:

The statement that a set of variables is a basis for the equation system
Ax = b means that their columns of coefficients are a basis for the col-
umn space of A and that b lies in the column space of A.

A third way to describe a basis

When the variables in the equation system Axâ•›=â•›b are labeled x1 through
xn , a basis can also be described as a subset β of the first n integers. A subset β
of the first n integers is now said to be a basis if {Aj : j ∈ β} is a basis for the
column space of A . In brief, the same basis for the column space of the 4 × 4
matrix A in equation (27) is identified in these three ways:
• As the set {A1 , A2 , A3 } of columns of A.

• As the set {x1 , x2 , x3 } of decision variables.

• As the set βâ•›=â•›{1, 2, 3} of integers.

Each way in which to describe a basis has its advantages: Describing a

basis as a set of columns is precise. Describing a basis as a set of decision
 Chapter 3: Eric V. Denardo 97

variables will prove to be particularly convenient in the context of a linear

program. Describing a basis as a set of integers is succinct.

What about the row space?

A basis for the row space of a matrix A could be found by applying Gauss-
Jordan elimination to the equation AT x = 0, where AT denotes the trans-
pose of A. A second application of Gauss-Jordan elimination is not necessary,
however.

Three important results

Three key results about vector spaces are stated and illustrated in this
subsection. These three results are highlighted below:

Three results:

•â•‡Every basis for a vector space contains the same number of ele-
ments, and that number is called the rank of the vector space.

•â•‡The row space and the column spaces of a matrix A have the same
rank.

•â•‡If the equation Ax = b has a solution, execution of Gauss–Jordan

elimination constructs a basic system, and the set of rows on which
pivots occur is a basis for the row space of A.

All three of these results are important. Their proofs are postponed, how-
ever, to Chapter 10, which sets the stage for a deeper understanding of linear
programming.

To illustrate these results, we recall that the coefficients of the decision

variables in system (1) array themselves into the 4 × 4 matrix A in equa-
tion (27). A sequence of pivots transformed system (1) into system (4). These
pivots occurred on coefficients in rows 1, 2 and 4, and they produced a basic
tableau whose basis is the set {x1 , x2 , x3 } of decision variables. Proposition
3.1 and the above results show that:

• The set {A1 , A2 , A3 } of columns is a basis for the column space of the
matrix A in (27).

• This matrix A has 3 as the rank of its column space.

98 Linear Programming and Generalizations

• This matrix has 3 as the rank of its row space.

• The set {A1 , A2 , A4 } of rows is a basis for the row space of A.

The rank of a vector space is also called its dimension; these terms are
synonyms. “Dimension” jives better with our intuition. In 3-space, every
plane through the origin has 2 as its dimension (or rank), for instance.

15.  Efficient Computation*

Efficient computation is vital to codes that solve large linear programs,

e.g., those having thousands of decision variables. Efficient computation is
not essential to a basic grasp of linear programming, however. For that rea-
son, it is touched upon lightly in this starred section.

Pivots make the simplex method easy to understand, but they are rela-
tively inefficient. Gaussian elimination substitutes “lower pivots” for pivots. It
solves an equation system with roughly half the work. Or less.

Lower pivots

To describe lower pivots, we identify the set S of equations on which

lower pivots have not yet occurred. Initially, S consists of all of the equations
in the system that is being solved. Each lower pivot selects an equation in S,
removes it, and executes certain Gaussian operations on the equations that
remain in S. Specifically, each lower pivot consists of these steps:

• Select an equation (j) in S and a variable x whose coefficient in equation

(j) is not zero.

• Remove equation (j) from S.

• For each equation (k) that remains in S, replace equation (k) by itself
less the multiple of equation (j) that equates the coefficient of x in equa-
tion (k) to zero.

This verbal description of lower pivots is cumbersome. But, as was the

case for full pivots, an example will make everything clear.
 Chapter 3: Eric V. Denardo 99

A familiar example

To illustrate lower pivots, we return to system (1). This system will be

solved a second time, with each “full” pivot replaced by the comparable lower
pivot. For convenient reference, system (1) is reproduced here as system (31).

(31.1) 2x1 + 4x2 − 1x3 + 8x4 = 4

(31.2) 1x1 + 2x2 + 1x3 + 1x4 = 1

(31.3) 2x3 − 4x4 = p

(31.4) −1x1 + 1x2 − 1x3 + 1x4 = 0

Initially, before any lower pivots have occurred, the set S consists of equa-
tions (31.1) through (31.4).

The first lower pivot

In this illustration, the same pivot elements will be selected as before.

The first lower pivot will occur on the coefficient of x1 in equation (31.1).
This lower pivot eliminates (drives to zero) the coefficient of x1 in equations
(31.2), (31.3) and (31.4). This lower pivot is executed by removing equation
(31.1) from S and then:

• Replacing equation (31.2) by itself minus (1/2) times equation (31.1).

• Replacing equation (31.3) by itself minus (0/2) times equation (31.1).

• Replacing equation (31.4) by itself minus (−1/2) times equation (31.1).

The three equations that remain in S become:

(32.2) 1.5x3 − 3x4 = −1

(32.3) 2x3 − 4x4 = p

(32.4) 3x2 − 1.5x3 + 5x4 = 2

100 Linear Programming and Generalizations

The variable x1 does not appear in equations (32.2), (32.3) and (32.4).
These three equations are identical to equations (2.2), (2.3) and (2.4), as
must be.

Equation (31.1) has been set aside, temporarily. After equations (32.2)
through (32.4) have been solved for values of the variables x2 , x3 and x4 ,
equation (31.1) will be solved for the value of x1 that is prescribed by these
values of x2 , x3 and x4 .

The second lower pivot

As was the case in the initial presentation of Gauss-Jordan elimination,

the second pivot element will be the coefficient of x3 in equation (32.2). A
lower pivot on this coefficient will drive to zero the coefficient of x3 in equa-
tions (32.3) and (32.4). This lower pivot is executed by removing equation
(32.2) from S and then:

• Replacing equation (32.3) by itself minus (2/1.5) times equation (32.2)

• Replacing equation (32.4) by itself minus (−1.5/1.5) times equation

(32.2).

This lower pivot replaces (32.3) and (32.4) by equations (33.3) and (33.4).

(33.3) 0x4 = p + 4/3

(33.4) 3x2 + 2x4 = 1

The variable x3 has been eliminated from equations (33.3) and (33.4).
These two equations are identical to equations (3.3) and (3.4), exactly as in
the case for the first lower pivot.

Equation (32.2), on which this pivot occurred, is set aside. After solving
equations (33.3) and (33.4) for values of the variables x2 and x4 , equation
(32.2) will be solved for the variable x3 on which the lower pivot has occurred.

The next lower pivot is slated to occur on equation (33.3). Again, there
are two cases to consider. If p is unequal to −4/3, equation (33.3) is inconsis-
tent, so no solution can exist to the original equation system. Alternatively, if
pâ•›=â•›−4/3, equation (33.3) is trite, and it has nothing to pivot upon.
 Chapter 3: Eric V. Denardo 101

Let us proceed on the assumption that pâ•›=â•›−4/3. In this case, equation

(33.3) is trite, so it is removed from S, which reduces to S to equation (34.4),
below.

(34.4) 3x2 + 2x4 = 1

The final lower pivot

Only equation (34.4) remains in S. The next step calls for a lower pivot on
equation (34.4). The variables x2 and x4 have nonzero coefficients in equa-
tion (34.4), so a lower pivot could occur on either of them. As before, we pivot
on the coefficient of x2 in this equation. But no equations remain in S after
equation (34.4) is removed. Hence, this lower pivot entails no arithmetic. As
concerns lower pivots, we are finished.

Back-substitution

It remains to construct a solution to system (31). This is accomplished

by equating to zero each variable on which no lower pivot has occurred and
then solving the equations on which lower pivots have occurred in “reverse”
order. In our example, no lower pivot has occurred on a the variable x4 . With
x4 = 0, the three equations on which lower pivots have occurred are:

2x1 + 4x2 − 1x3 = 4

1.5x3 = −1

3x2 =1

The first lower pivot eliminated x1 from the bottom two equations. The
second lower pivot eliminated x3 from the bottom equation. Thus, these
equations can be solved for the variables on which their lower pivots have
occurred by working from the bottom up. This process is aptly called back-
substitution. For our example, back-substitution first solves the bottom
equation for x2 , then solves the middle equation for x3 , and then solves the
top equation for x1 . This computation gives x2 = 1/3 and x3 = −2/3 and
x1 = 1, exactly as before.
102 Linear Programming and Generalizations

Solving an equation system by lower pivots and back-substitution is

known as Gaussian elimination and by the fancier label, L-U decomposi-
tion. By either name, it requires roughly half as many multiplications and
divisions as does Gauss-Jordan elimination. This suggests that lower pivots
are twice as good. Actually, lower pivots are a bit better; they allow us to take
better advantage of “sparsity” and help us to control “round-off ” error.

Sparsity and fill-in

Typically, a large system of linear equations is sparse, which means that

all but a tiny fraction of its coefficients are zeros. As pivoting proceeds, a
sparse equation system tends to “fill in” as nonzero entries replace zeros.

An adroit sequence of pivots can reduce the rate at which fill-in occurs.
A simple method for retarding fill-in counts the number of nonzero elements
that might be created by each pivot and select a pivot element that minimizes
this number. This method works with full pivots, and it works a bit better
with lower pivots, for which it is now described. Specifically:

• Keep track of the set R of rows on which lower pivots have not yet oc-
curred and the set C of columns of for which variables have not yet
been made basic.

• For each k ∈ C, denote as ck the number of equations in R in which xk

has a nonzero coefficient.

• For each j ∈ R, denote as rj the number of variables whose coeffi-

cients in equation (j) are nonzero.

Take a moment to convince yourself that a lower pivot on the coefficient

of the variable xk in equation (j) will fill in (render non-zero) at most (rjâ•›−â•›1)
(ck− 1) zeros. This motivates the rule that’s displayed below.

Myopic pivoter (initialized as indicated above).

While R is nonempty:

Among the pairs (j, k) with j ∈ R and k ∈ C for which the coefficient
of xk in row j of the current tableau is nonzero, pick a pair that mini-
mizes (rj − 1)(ck − 1).

Execute a lower pivot on the coefficient of xk in row j of the current

tableau.
 Chapter 3: Eric V. Denardo 103

Remove k from C and j from R.

Update rj for each equation j ∈ R, and update ck for each k ∈ C, .

This rule is myopic (near-sighted) in the sense that it aims to minimize

the amount of fill–in at the moment, without looking ahead.

Gaussian elimination with back-substitution requires roughly half as

many multiplications and divisions, but the worst-case work count still grows
as the cube of the problem size. As the problem size increases, the coefficient
matrix tends to become increasingly sparse (have a larger fraction of zeros),
and the work bound grows less rapidly if care is taken to pivot in a way that
retards fill-in.

Pivoting on very small numbers

Modern implementations of Excel do floating-point arithmetic with a 64-

bit word length. This allows about 16 digits of accuracy. In small or moderate-
sized problems, round-off error is not a problem, provided we avoid pivoting
on very small numbers.

To see what can go awry, consider a matrix (array) whose nonzero entries
are between 1 and 100, except for a few that are approximately 10−6 . Pivoting
on one of these tiny entries multiplies everything in its row by 106 and shifts
the information in some of the other rows about 6 digits to the right. Doing
that once may be OK. Doing two or three times can bury the information in
the other rows. And that’s without worrying about the round-off error in the
pivot element. In brief:

When executing Gauss-Jordan elimination, try not to pivot on coeffi-

cients that are several orders of magnitude below the norm.

16.  Review

Gauss-Jordan elimination makes repeated and systematic use of two

Gaussian operations. These operations are organized into pivots. Each pivot
creates a basic variable for an equation that lacked one. Each pivot keeps the
variables that had been basic for the other equations basic for those equa-
tions. Gauss-Jordan elimination keeps pivoting until:
104 Linear Programming and Generalizations

• Either it constructs an inconsistent equation.

• Or it creates a basic system, specifically, a basic variable for each non-

trite equation.

If Gauss-Jordan elimination constructs an inconsistent equation, the orig-

inal equation system can have no solution. If Gauss-Jordan elimination con-
structs a basic system, its basic solution satisfies the original equation system.
This basic solution equates each non-basic variable to zero, and it equates each
basic variable to the right-hand-side value of the equation for which it is basic.

Pivoting lies at the core of an introductory account of the simplex method.

In Chapter 4, it will be seen that:

• The simplex method executes Gauss-Jordan elimination and then keeps

on pivoting in search of an optimal solution to the linear program.

• The terminology introduced here is used to describe the simplex meth-

od. These terms include pivot, basic variable, basic system, basic solu-
tion, basis, and basic solution.

• Geometry will help us to visualize the simplex method and to relate it

to fundamental ideas in linear algebra.

In a starred section, it was observed, that “lower” pivots and back-sub-

stitution are preferable to the “full” pivots of Gauss-Jordan elimination. Low-
er pivots are faster. They reduce the rate of fill-in and the accumulation of
round-off error. When lower pivots are used in conjunction with the simplex
method, the notation becomes rather involved, and the subject matter shifts
the tenor of the discussion from linear algebra to numerical analysis, which
we eschew.

17.  Homework and Discussion Problems

1. To solve the following system of linear equations, implement Gauss-Jor-

dan elimination on a spreadsheet. Turn your spreadsheet in, and indicate
the functions that you have used in your computation.

1A − 1B + 2C = 10
−2A + 4B − 2C = 0
0.5A − 1B − 1C = 6
 Chapter 3: Eric V. Denardo 105

2. Consider the following system of three equations in three unknowns.

2A + 3B − 1C = 12
−2A + 2B − 9C = 3
4A + 5B = 21
(a) Use Gauss-Jordan elimination to find a solution to this equation sys-
tem.

(b) Plot those solutions to this equation system in which each variable
is nonnegative. Complete this sentence: The solutions that have been
plotted form a ________________.

(c) What would have happened if one of the right-hand-side values had
been different from what it is? Why?

3. Use a spreadsheet to find all solutions to the system of linear equations

that appears below. (↜Hint: construct a dictionary.)

2x1 + 4x2 – 1x3 + 8x4 + â•›10x5 = 4

1x1 + 2x2 + 1x3 + 1x4 + â•› 2x5 = 1

╇╛╛2x3 – 4x4â•› – 4x5 = –4/3

–1x1 + 1x2 – 1x3 + 1x4 â•›– 1x5 = 0

4. Redo the spreadsheet computation in Tables 1-4 using lower pivots in

place of (full) pivots. Turn in your spreadsheet. On it, indicate the func-
tions that you used.

5. Consider system (1) with pâ•›=â•›−4/3. Alter any single coefficient of x1 in

equation (1.1) or (1.2) or (1.3) and then re-execute the pivots that pro-
duced system (4). Remark: No grunt-work is needed if you use spread-
sheets.

(a) What happens?

(b) Can you continue in a way that produces a basic solution? If so, do so.

6. The matrix A given by (27) consists of the coefficients of the decision vari-
ables in system (1). For this matrix A:
106 Linear Programming and Generalizations

(a) Use Gauss-Jordan elimination to show that A3 is a linear combination

of A1 and A2 . Remark: This can be done without grunt-work if you
apply the pivot function to the homogeneous equation AT y = 0.

(b) Determine whether or not A4 is a linear combination of A1 , A2 and

A3 .

(c) Which subsets of {A1 , A2 , A3 , A4 } are a basis for the row space of A?
Why?

7. Tables 1-4 showed how to execute Gauss-Jordan elimination on a spread-

sheet for the special case in which the datum p equals −4/3. Re-do this
spreadsheet for the general case in which the datum p can be any number.
Hint: Append to Table 3.1 a column whose heading (in row 1) is p and
whose coefficients in rows 2, 3, 4 and 5 are 0, 0, 1, and 0, respectively.

8. (a basis) This problem concerns the four vectors that are listed below.
Solve parts (a), (b) and (c) without doing any numerical computation.

2 4 −1 8
       
 1 2  1  1
 ,  ,  ,  .
 0 0  2 −4
−1 1 −1 1

(a) Show that the left-most three of these vectors are linearly indepen-
dent.

(b) Show that the left-most three of these vectors span the other one.

(c) Show that the left-most three of these vectors are a basis for the vector
space that consists of all linear combinations of these four vectors.

9. (Opposite columns) In the equation Axâ•›=â•›b, columns j and k are said to be

opposite if Aj = −Ak . Suppose columns 5 and 12 are opposite.

(a) After one Gaussian operation, columns 5 and 12 ___________.

(b) After any number of Gaussian operations, columns 5 and 12

___________.

(c) If a pivot makes x5 basic for some equation, then x12 ____________.
 Chapter 3: Eric V. Denardo 107

10. (Homogeneous systems) True or false?

(a) When Gauss-Jordan elimination is applied to a homogeneous system,

it can produce an inconsistent equation.

(b) Every (homogeneous) system Axâ•›=â•›0 has at least one non-trivial solu-
tion, that is, one solution that has x ≠ 0

(c) Application of Gauss-Jordan elimination to a homogeneous system

constructs a non-trivial solution if one exists.

(d) Every homogeneous system of four equations in five variables has at

least one non-trivial solution.

11. Let A be an m × n matrix with m < n.

(a) Show that the columns of A are linearly dependent.

(b) Prove or disprove: There exists a nonzero vector x such that Axâ•›=â•›0.

12. True or false? Each subset V of n that is a vector space has a basis. Hint:
take care.

13. This problem concerns the matrix equation Axâ•›=â•›b. Describe the condi-
tions on A and b under which this equation has:

(a) No solutions.

(b) Multiple solutions.

(c) Exactly one solution.

14. Prove that a non-empty set {v1 , v2 , . . . , vK } of n-vectors is linearly in-

dependent if and only if none of these vectors is a linear combination of
the others.

15. Prove that a set V of n-vectors that includes the origin is a vector space if
and only if V contains the vector [(1 − α)u + αv] for every pair u and v
of elements of V and for every real number α.

16. A set W of n-vectors is called an affine space if W is not empty and if W

contains the vector [(1 − α)u + αv] for every pair u and v of elements of
V and for every real number α.
108 Linear Programming and Generalizations

(a) If an affine space W contains the origin, is it a vector space?

(b) For the case nâ•›=â•›2, describe three types of affine space, and guess the
“dimension” of each.

17. Designate as X the set consisting of each vector x that satisfies the matrix
equation Axâ•›=â•›b. Suppose X is not empty. Is X a vector space? Is X an affine
space? Support your answers.

18. Verify that equations (19) and (20) are correct. Hint: Equation (18) might
help.

19. (↜Small pivot elements) You are to solve following system twice, each time
by Gauss-Jordan elimination. Throughout each computation, you are to
approximate each coefficient by three significant digits; this would round
the number 0.01236 to 0.0124, for instance.

0.001A + 1B = 10
1A − 1B = 0

(a) For the first execution, begin with a pivot on the coefficient of A in
the topmost equation.

(b) For the second execution, begin with a pivot on the coefficient of B in
the topmost equation.

(c) Compare your solutions. What happens? Why?

Remark: The final two problems (below) refer to the starred section on
efficient computation.

20. (Work for lower pivots and back-substitution) Imagine that a system of m
equations in n unknowns is solved by lower pivots and back-substitution
and that no trite or inconsistent equations have been encountered.

(a) Show that the number of multiplications and divisions required by

back-substitution equals (1 + 2 + · · · + m) = (m)(m + 1)/2.

(b) For each j < m, show that the j-th lower pivot requires (m + 1â•›−â•›j) (n)
multiplications and divisions.
 Chapter 3: Eric V. Denardo 109

(c) How many multiplications and divisions are needed to execute Gauss-
Jordan elimination with lower pivots and back-substitution? Hint: sum-
ming part (b) gives (2 + 3 + · · · + m)(n) = (n)(m)(m + 1)/2 − n.

21. (Sparseness) In the detached-coefficient tableau that follows, each non-

zero number is represented by an asterisk (*). Specify a sequence of lower
pivots that implements the myopic rule, with ck equal to the number
of non-zero coefficients of xk in rows on which pivots have not yet oc-
curred. How many Gaussian operations does this implementation re-
quire? How many multiplications and divisions does it require, assuming
that you omit multiplication by zero?

Equation x1 x2 x3 x4 x5 RHS
(1) * * * * * *
(2) * * *
(3) * * *
(4) * * * *
(5) * * *

 Part II–The Basics

This section introduces you to the simplex method and prepares you to
make intelligent use of the computer codes that implement it.

Chapter 4. The Simplex Method, Part 1

In Chapter 3, you saw that Gauss-Jordan elimination pivots until it finds

a basic solution to an equation system. In Chapter 4, you will see that the
simplex method keeps on pivoting – it aims to improve the basic solution’s
objective value with each pivot, and it stops when no further improvement
is possible.

Chapter 5. Analyzing Linear Programs

In this chapter, you will learn how to formulate linear programs for solu-
tion by Solver and by Premium Solver for Education. You will also learn how
to interpret the output that these software packages provide. A linear pro-
gram is seen to be the ideal environment in which to relate three important
economic concepts – shadow price, “relative” opportunity cost, and marginal
benefit. This chapter includes a “Perturbation Theorem” that can help you
to grapple with the fact that a linear program is a model, an approximation.

Chapter 6. The Simplex Method, Part 2

This chapter plays a “mop up” role. If care is not taken, the simplex meth-
od can pivot forever. In Chapter 6, you will see how to keep that from oc-
curring. The simplex method, as presented in Chapter 4, is initiated with a
feasible solution. In Chapter 6, you will see how to adapt the simplex method
to determine whether a linear program has a feasible solution and, if so, to
find one.
 Chapter 4: The Simplex Method, Part 1

1.╅ Preview����������������������������������尓������������������������������������尓���������������������� 113

2.╅ Graphical Solution����������������������������������尓������������������������������������尓���� 114
3.╅ A Format that Facilitates Pivoting ����������������������������������尓�������������� 119
4.╅ First View of the Simplex Method����������������������������������尓���������������� 123
5.╅ Degeneracy����������������������������������尓������������������������������������尓���������������� 132
6.╅ Detecting an Unbounded Linear Program����������������������������������尓�� 134
7.╅ Shadow Prices����������������������������������尓������������������������������������尓������������ 136
8.╅ Review����������������������������������尓������������������������������������尓������������������������ 144
9.╅ Homework and Discussion Problems����������������������������������尓���������� 147

1.  Preview

The simplex method is the principal tool for computing solutions to lin-
ear programs. Computer codes that execute the simplex method are widely
available, and they run on nearly every computer. You can solve linear pro-
grams without knowing how the simplex method works. Why should you
learn it? Three reasons are listed below:

• Understanding the simplex method helps you make good use of the
output that computer codes provide.

• The “feasible pivot” that lies at the heart of the simplex method is cen-
tral to constrained optimization, much as Gauss-Jordan elimination is
fundamental to linear algebra. In later chapters, feasible pivots will be
adapted to solve optimization problems that are far from linear.

• The simplex method has a lovely economic interpretation. It will be

seen that each basis is accompanied by a set of “shadow prices” whose

E. V. Denardo, Linear Programming and Generalizations, International Series 113

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_4, © Springer Science+Business Media, LLC 2011
114 Linear Programming and Generalizations

values determine the benefit of altering the basic solution by engaging

in any activity that is currently excluded from the basis.

The simplex method also has a surprise to offer. It actually solves a pair
of optimization problems, the one under attack and its “dual.” That fact may
seem esoteric, but it will be used in Chapter 14 to formulate competitive situ-
ations for solution by linear programming and its generalizations.

2.  Graphical Solution

The simplex method will be introduced in the context of a linear program

that is simple enough to solve visually. This example is

Problem A.╇ Maximize {2x╛+ ╛3y} subject to the constraints

x ≤ 6,
x + y ≤ 7,
â•… ╛╛2y ≤ 9,
– x + 3y ≤ 9,
x ≥ 0,
y ≥ 0.

Before the simplex method is introduced, Problem A is used to review

some terminology that was introduced in Chapter 1.

Feasible solutions

A feasible solution to a linear program is an assignment of values to its

decision variables that satisfy all of its constraints. Problem A has many fea-
sible solutions, one of which is the pair (x, y)â•›=â•›(1, 0) in which xâ•›=â•›1 and yâ•›=â•›0.
Because Problem A has only two decision variables, its feasible solutions can
be depicted on the plane. Figure 4.1 does so. In it, each constraint in Problem
A is represented as a line on which that constraint holds as an equation, ac-
companied by an arrow pointing into the half-space that satisfies it strictly.
For instance, the pairs (x,  y) that satisfy the constraint −x −xâ•›++â•›3y ≤ 9 as an
equation form the line through (0, 3) and (6, 5), and an arrow points from
that line into the region that satisfies the constraint as a strict inequality.
 Chapter 4: Eric V. Denardo 115

Figure 4.1.↜  The feasible solutions to Problem A.

7
2y ≤ 9
6 –x + 3y ≤ 9

4
x+y≤7
3

2
feasible region x≤6
x≥0 1

0 x
y≥0
0 1 2 3 4 5 6 7

In Problem A and in general, the feasible region is the set of values of the
decision variables that satisfy all of the constraints of the linear program. In
Figure 4.1, the feasible region is shaded. Let us recall from Chapter 3 that the
feasible region of a linear program is a convex set because it contains the in-
terval (line segment) between each pair of points in it. A constraint in a linear
program is said to be redundant if its removal does not change the feasible
region. Figure 4.1 makes it clear that the constraint 2y ≤ 9 is redundant.

Iso-profit lines

Figure 4.1 omits any information about the objective function. Each fea-
sible solution assigns a value to the objective in the natural way; for instance,
feasible solution (5, 1) has objective value 2xâ•›+â•›3yâ•›=â•›(2)(5)â•›+â•›(3)(1)â•›=â•›13.

An iso-profit line is a line on which profit is constant. Figure 4.2 displays

the feasible region for Problem A and four iso-profit lines. Its objective, 2xâ•›+â•›3y,
equals 6 on the iso-profit line that contains the points (3, 0) and (0, 2). Simi-
larly, the iso-profit line on which 2xâ•›+â•›3yâ•›=â•›12 contains the points (6, 0) and
(0, 4). In this case and in general, the iso-profit lines of a linear program are
116 Linear Programming and Generalizations

parallel to each other. Notice in Figure 4.2 that the point (3, 4) has a profit of
18 and that no other feasible solution has a profit as large as 18. Thus, xâ•›=â•›3 and
yâ•›=â•›4 is the unique optimal solution to Problem A, and 18 is its optimal value.

Figure 4.2.↜  Feasible region for Problem A, with iso-profit lines and
objective vector (2, 3).

7 objective vector
3 equals (2, 3)
6
2x + 3y = 18 2
5

4 (3, 4)
2x + 3y = 12
3

2
2x + 3y = 6 feasible region
1 (6, 1)

0 x

2x + 3y = 0 0 1 2 3 4 5 6 7

Each feasible solution to a linear program assigns a value to its objective

function. An optimal solution to a linear program is a feasible solution whose
objective value is largest in the case of a maximization problem, smallest in
the case of a minimization problem. The optimal value of a linear program is
the objective value of an optimal solution to it. It’s clear from Figure 4.2 that
(3, 4) is an optimal solution to Problem A and that 18 is its optimal value.

The objective vector

There is a second way in which to identify the optimal solution or solu-

tions to a linear program. The object of Problem A is to maximize the expres-
sion (2xâ•›+â•›3y). The coefficients of x and y in this expression form the objective
vector (2, 3). A vector connotes motion. We think of the vector (2, 3) as mov-
 Chapter 4: Eric V. Denardo 117

ing 2 units toward the right of the page and 3 units toward the top. In Fig-
ure 4.2, the objective vector shown touching the iso-profit line 2xâ•›+â•›3yâ•›=â•›18.

The objective vector can have its tail “rooted” anywhere in the plane. In
Figure  4.2 and in general, the objective vector is perpendicular to the iso-
profit lines. It’s the direction in which the objective vector points that matters.
In a maximization problem, we seek a feasible solution that lies farthest in
the direction of the objective vector. Similarly, in a minimization problem, we
seek a feasible solution that lies farthest in the direction that is opposite to the
objective vector. It is emphasized:

The objective vector points “uphill” – in the direction of increase of the

objective.

In Figure 4.2, for instance, the optimal solution is (3, 4) because, among

feasible solutions, it lies farthest in the direction of the objective vector.

Extreme points

It is no surprise that the feasible region in Figure 4.2 is a convex set. In

Chapter 3, it was observed that the feasible region of every linear program is
a convex set. It is recalled that an element of a convex set is an extreme point
of that set if it is not a convex combination of two other points in that set. The
feasible region in Figure 4.2 has five extreme points (corners). The optimal
solution lies at the extreme point (3, 4). The other four extreme points are
(0, 0), (6, 0), (6, 1) and (0, 3).

Edges

The mathematical definition of an “edge” is a bit involved. But it is clear,

visually, that the feasible region in Figure 4.2 has five edges. Each of these
edges is a line segment that connects two extreme points. The line segment
connecting extreme points (0, 0) and (6, 0) is an edge, for instance. Not every
line segment that connects two extreme points is an edge. The line segment
connecting extreme points (0, 0) and (3, 4) is not an edge (because it inter-
sects the “interior” of the feasible region).

Optimality of an extreme point

In Figure 4.2, suppose the objective vector pointed in some other direc-

tion. Would an extreme point still be optimal? Yes, it would, but it could be
118 Linear Programming and Generalizations

a different extreme point. Suppose, for instance, that the objective vector is
(3, 3). In this case, the objective vector has rotated clockwise, and extreme
points (3, 4) and (6, 1) are both optimal, as is each point in the edge con-
necting them. If the objective vector is (4, 3), the objective vector has rotated
farther clockwise, and the unique optimal solution is the extreme point (6, 1).

Adjacent extreme points

Two extreme points are said to be adjacent if the interval between them
is an edge. In Figure 4.2, extreme points (0, 0) and (0, 3) are adjacent. Extreme
points (0, 0) and (3, 4) are not adjacent.

Simplex pivots

“Degeneracy” is discussed later in this chapter. If a simplex pivot is de-

generate, the extreme point does not change. If a simplex pivot is nondegen-
erate, it occurs to an adjacent extreme point, and each such pivot improves
the objective value. The simplex method stops pivoting when it discovers that
the current extreme point has the best objective value.

When the simplex method is applied to Problem A, the first pivot will
occur from extreme point (0, 0) to extreme point (0, 3), and the second pivot
will occur to extreme point (3, 4), which will be identified as optimal.

Bounded feasible region

A linear program is said to have a bounded feasible region if at least one

feasible solution exists and if there exists a positive number K such that no
feasible solution assigns any variable a value below –K or above +K. The fea-
sible region in Figure 4.2 is bounded; no feasible solution has |x| > 6 or |y| > 6.
A feasible region is said to be unbounded if it is not bounded.

Bounded linear programs

A linear program is said to be feasible and bounded if it has at least one fea-
sible solution and if its objective cannot be improved without limit. Problem
A is feasible and bounded. It would not be bounded if the constraints xâ•›+â•›y ≤ 7
and x ≤ 6 were removed. It is easy to convince oneself, visually, of the following:

If a linear program whose variables are constrained to be nonnegative is

feasible and bounded, at least one of its extreme points is optimal.
 Chapter 4: Eric V. Denardo 119

A linear program can be feasible and bounded even if its feasible region is
unbounded. An example is: Minimize {x}, subject to x ≥ 0.

Unbounded linear programs

A maximization problem is said to be unbounded if it is feasible and if no

upper bound exists on the objective value of its feasible solutions. Similarly,
a minimization problem is unbounded if it is feasible and if no lower bound
exists on the objective value of its feasible solutions. Unbounded linear pro-
grams are unlikely to occur in practice because they describe situations in
which one can do infinitely well. They do occur from inaccurate formulations
of bounded linear programs.

3.  A Format that Facilitates Pivoting

The simplex method consists of a deft sequence of pivots. Pivots occur on

systems of equality constraints. To prepare Problem A for pivoting, it is first
placed in the format called Form 1, namely, as a liner program having these
properties:

• The object is to maximize or minimize the quantity z.

• Each decision variable other than z is constrained to be nonnegative.

• All of the other constraints are linear equations.

Form 1 introduces z as the quantity that we wish to make largest in a

maximization problem, smallest in a minimization problem. Form 1 requires
each decision variable other than z to be nonnegative, and it gets rid of the
inequality constraints, except for those on the decision variables.

A canonical form?

The simplex method will be used to solve every linear program that has
been cast in Form 1. Can every linear program be cast in Form 1? Yes. To
verify that this is so, observe that:

• Form 1 encompasses maximization problems and minimization prob-

lems.

• An equation can be included that equates z to the value of the objective.

120 Linear Programming and Generalizations

• Each inequality constraint can be converted into an equation by inser-

tion of a nonnegative (slack or surplus) variable.

• Each variable that is unconstrained in sign can be replaced by the dif-

ference of two nonnegative variables.

A canonical form for linear programs is any format into which every lin-
ear program can be cast. Form 1 is canonical form. Since Form 1 is canonical,
describing the simplex method for Form 1 shows how to solve every linear
program. It goes without saying, perhaps, that it would be foolish to describe
the simplex method for linear programs that have not been cast in a canoni-
cal form.

Recasting Problem A

Let us cast Problem A in Form 1. The quantity z that is to be maximized

is established by appending to Problem A the “counting” constraint,

2x + 3y = z,

which equates z to the value of the objective function. Problem A has four
“ ≤ ” constraints, other than those on its decision variables. Each of these in-
equality constraints is converted into an equation by inserting a slack vari-
able on its left-hand side. This re-writes Problem A as

Problem A’.╇ Maximize {z}, subject to the constraints

(1.0) 2x + 3y – z = 0,

(1.1) 1x + s1 = 6,

(1.2) 1x + 1y + s2 = 7,

(1.3) â•…â•…â•›2y + s3 = 9,

(1.4) – 1x + 3y + s4 = 9,

x ≥ 0, y ≥ 0, â•›s1 ≥ 0 for i = 1, 2, 3, 4.
 Chapter 4: Eric V. Denardo 121

Problem A’ is written in Form 1. It has seven decision variables and five

equality constraints. Each decision variable other than z is constrained to be
nonnegative. The variable z has been shifted to the left-hand side of equation
(1.0) because we want all of the decision variables to be on the left-hand sides
of the constraints.

To see where the “slack variables” get their name, consider the constraint
xâ•›+â•›y ≤ 7. In the constraint xâ•›+â•›yâ•›+â•› s2 = 7, the variable s2 is positive if xâ•›+â•›y < 7
and s2 is zero if xâ•›+â•›y = 7. Evidently, s2 “takes up the slack” in the constraint
xâ•›+â•›y ≤ 7.

The variable –z

In Form 1, the variable z plays a special role because it measures the ob-
jective. We elect to think of –z as a decision variable. In Problem A’, the vari-
able –z is basic for equation (1.0) because –z has a coefficient of +1 in equa-
tion (1.0) and has coefficients of 0 in all other equations. During the entire
course of the simplex method, no pivot will ever occur on any coefficient in
the equation for which –z is basic. Consequently, –z will stay basic for this
equation.

Reduced cost

The equation for which –z is basic plays a guiding role in the simplex
method, and its coefficients have been given names. The coefficient of each
variable in this equation is known as that variable’s reduced cost. In equation
(1.0), the reduced cost of x equals 2, the reduced cost of y equals 3, and the
reduced cost of each slack variable equals 0. The term “reduced cost” is firmly
established in the literature, and we will use it. But it will soon be clear that
“marginal profit” would have been more descriptive.

The feasible region for Problem A’

Problem A’ has seven decision variables. It might seem that the feasible
region for Problem A’ can only be “visualized” in seven-dimensional space.
Figure 4.3 shows that a 2-dimensional picture will do. In Figure 4.3, each line
in Figure 4.1 has been labeled with the variable in Problem A’ that equals zero
on it. For instance, the line on which the inequality xâ•›+â•›y ≤ 7 holds as an equa-
tion is relabeled s2 = 0 because s2 is the slack variable for the constraint
xâ•›+â•›yâ•›+â•› s2 = 7.
122 Linear Programming and Generalizations

Figure 4.3.↜  The feasible region for Problem A’â•›.

y
7

6
s4 = 0 s3 = 0
5

3 s2 = 0

2
x=0 feasible region
1 s1 = 0
0 x
y=0 0 1 2 3 4 5 6 7

Bases and extreme points

Figure 4.3 also enables us to identify the extreme points with basic solu-
tions to system (1). Note that each extreme point in Figure 4.3 lies at the inter-
section of two lines. For instance, the extreme point (0, 3) is the intersection
of the lines xâ•›=â•›0 and s4 = 0. The extreme point (0, 3) will soon be associated
with the basis that excludes the variables that x and s4 .

System (1) has five equations and seven variables. The variables –z and
s1 through s4 form a basis for system (1). This basis consists of five variables,
one per equation. A fundamental result in linear algebra (see Proposition 10.2
on page 334 for a proof) is that every basis for a system of linear equations
has the same number of variables. Thus, each basis for system (1) contains
exactly five variables, one per equation. In other words, each basis excludes
two of the seven decision variables. Each basis for system (1) has a basic solu-
tion, and that basic solution equates its two nonbasic variables to zero. This
identifies each extreme point in Figure 4.3 with a basis. Extreme point (0, 3)
corresponds to the basis that excludes x and s4 because (0, 3) is the intersec-
tion of the lines xâ•›=â•›0 and s4 = 0. Similarly, extreme point (3, 4) corresponds to
 Chapter 4: Eric V. Denardo 123

the basis that excludes s2 and s4 because (3, 4) is the intersection of the lines
s2 = 0.and s4 = 0.

4.  First View of the Simplex Method

Problem A’ will now be used to introduce the simplex method, and Fig-
ure 4.3 will be used to track its progress. System (1) is basic because each of its
equations has a basic variable. The basis for system (1) consists of –z and the
slack variables. This basis excludes x and y. Its basic solution equates to zero
its nonbasic variables (which are x and y) and is

x = 0, y = 0, −z = 0, s1 = 6, s2 = 7, s3 = 9, s4 = 9.

A feasible basis

A basis for Form 1 is now said to be feasible if its basic solution is feasible,
that is, if the values of the basic variables are nonnegative, with the possible
exception of –z. Evidently, the basis {–z, s1, s2, s3, s4â•›} is feasible.

Phases I and II

For Problem A, a feasible basis sprang immediately into view. That is not
typical. Casting a linear program in Form 1 does not automatically produce
a basis, let alone a feasible basis. Normally, a feasible basis must be wrung
out of the linear program by a procedure that is known as Phase I of the
simplex method. Using Problem A to introduce the simplex method begins
with “Phase II” of the simplex method. Phase I has been deferred to Chapter
6 because it turns out to be a minor adaptation of Phase II.

Phase II of the simplex method begins with a feasible basis and with –z
basic for one of its equations. Phase II executes a series of pivots. None of
these pivots occurs on any coefficient in the equation for which –z is basic.
Each of these pivots:

• keeps –z basic;

• changes the basis, but keeps the basic solution feasible;

• improves the basic solution’s objective value or, barring an improve-

ment, keeps it from worsening.
124 Linear Programming and Generalizations

Phase II stops pivoting when it discerns that the basic solution’s objective
value cannot be improved. How this occurs will soon be explained.

A simplex tableau

A basic system for Form 1 is said to be a simplex tableau if –z is ba-

sic for the top-most equation and if the right-hand-side values of the other
equations are nonnegative. This guarantees that the basic solution is feasible
(equates all variables other than –z to nonnegative values) and that it equates
z to the basic solution’s objective value. A simplex tableau is also called a basic
feasible tableau; these terms are synonyms.

The dictionary

We wish to pivot from simplex tableau to simplex tableau, improving – or

at least not worsening – the objective with each pivot. It is easy to see which
pivots do the trick if system (1) is cast in a format that has been dubbed a
dictionary.1 System (1) is placed in this format by executing these two steps.

• Shift the non-basic variables x and y to the right-hand sides of the con-
straints.

• Multiply equation (1.0) by –1, so that z (and not –z) appears on its left-
hand side.

Writing system (1) in the format of a dictionary produces system (2), below.

(2.0) z = 0 + 2x + 3y

(2.1) s1 = 6 − 1x + 0y

(2.2) s2 = 7 − 1x + 1y,

(2.3) s3 = 9 − 0x − 2y

(2.4) s4 = 9 + 1x − 3y

The term “dictionary” is widely attributed to Vašek Chvátal, who popularized it in

1╇

his lovely book, Linear Programming, published in 1983 by W. H. Freedman and Co.,
New York. In that book, Chvátal attributes the term to J. E. Strum’s, Introduction to
Linear Programming, published in 1972 by Holden-Day, San Francisco.
 Chapter 4: Eric V. Denardo 125

In system (2), the variable z (rather than –z) is basic for the topmost equa-
tion, and the slack variables are basic for the remaining equations. The basic
solution to system (2) equates each non-basic variable to zero and, conse-
quently, equates each basic variable to the number on the right-hand-side
value of the equation for which it is basic.

Perturbing a basic solution

The dictionary indicates what happens if the basic solution is perturbed

by setting one or more of the nonbasic variables positive and adjusting the
values of the basic variables so as to preserve a solution to the equation sys-
tem. Equation (2.0) shows that the objective value is increased by setting x
positive and by setting y positive.

Reduced cost and marginal profit

The coefficients of x and y in equation (2.0) equal their reduced costs,

namely, their coefficients in equation (1.0). To see why this occurs, note that
the reduced costs have been multiplied by –1 twice, once when equation (1.0)
was multiplied by –1 and again when the nonbasic variables were transferred
to its right-hand side.

In a simplex tableau for a maximization problem, the marginal profit

of each nonbasic variable equals the change that occurs in its objective value
when the basic solution is perturbed by setting that variable equal to 1 and
keeping all other nonbasic variables equal to zero. The dictionary in system
(2) makes the marginal profits easy to see. Its basic solution equates the non-
basic variables x and y to 0. Equation (2.0) shows that the marginal profit of x
equals 2 and that the marginal profit of y equals 3. The marginal profit of each
nonbasic variable is its so-called “reduced cost.” It is emphasized:

In each simplex tableau for a maximization problem, the “reduced cost”

of each nonbasic variable equals the marginal profit for perturbing the
tableau’s basic solution by equating that variable to 1, keeping the other
nonbasic variables equal to zero, and adjusting the values of the basic
variables so as to satisfy the LP’s equations.

Similarly, in a minimization problem, the “reduced cost” of each non-

basic variable equals the marginal cost of perturbing the basic solution by
126 Linear Programming and Generalizations

setting that nonbasic variable equal to 1 and adjusting the values of the basic
variables accordingly.

As mentioned earlier, we cleave to tradition and call the coefficient of

each variable in the equation for which –z is basic its reduced cost. Please
interpret the “reduced cost” of each nonbasic variable as marginal profit in a
maximization problem and as marginal cost in a minimization problem.

A pivot

Our goal is to pivot in a way that improves the basic solution’s objec-
tive value. Each pivot on a simplex tableau causes one variable that had been
nonbasic to become basic and causes one basic variable to become nonbasic.
Equation (2.0) shows that the objective function improves if the basic solu-
tion is perturbed by setting x positive or by setting y positive. We could pivot
in a way that makes x basic or in a way that makes y basic.

Perturbing system (2) by keeping xâ•›=â•›0 and setting y > 0 produces:

(3.0) z = 0 + 3y;

(3.1) s1 = 6 so s1 is positive for all values of y;

(3.2) s2 = 7 – 1y, so s2 decreases to zero when y = 7/1 = 7;

(3.3) s3 = 9 – 2y, so s3 decreases to zero when y = 9/2 = 4.5;

(3.4) s4 = 9 – 3y, so s4 decreases to zero when y = 9/3 = 3.

Evidently, the largest value of y that keeps the perturbed solution feasible
is y = 3. If y exceeds 3, the perturbed solution has s4 < 0.

Graphical interpretation

Figure 4.3 is now used to interpret the ratios in system (3). The initial ba-
sis excludes x and y, and so the initial basic solution lies at the intersection of
the lines x = 0 and yâ•›=â•›0, which is the point (0, 0). The perturbation in system
(3) keeps xâ•›=â•›0 and allows y to become positive, thereby moving upward on
the line xâ•›=â•›0. Each “ratio” in system (3) is a value of y for which (0, y) inter-
sects a constraint. No ratio is computed for constraint (3.1) because the lines
(0, y) and s1 = 0 do not intersect. The smallest ratio is the largest value of y
for which the perturbed solution stays feasible.
 Chapter 4: Eric V. Denardo 127

Feasible pivots

Rather than proceeding directly with the simplex method, we pause to

describe a class of pivots that keeps the basic solution feasible. Specifically,
starting with a basic feasible solution for Form 1, we select any nonbasic vari-
able and call it the entering variable. In system (1), we take y as the entering
variable. The goal is to pivot on a coefficient of y that keeps the basic solution
feasible and keeps –z basic for the top-most equation. In a basic tableau for
Form 1, which coefficient of the entering variable shall we pivot upon? Well:

• No coefficient of the equation for which –z is basic is pivoted upon in

order to keep –z basic for the equation. For this reason, no “ratio” is
ever computed for this equation.

• No coefficient that is negative is pivoted upon.

• Excluding the equation for which –z is basic, each equation whose co-
efficient of the entering variable is positive has a ratio that equals this
equation’s right-hand side value divided by its coefficient of the enter-
ing variable.

• The pivot occurs on the coefficient of the entering variable in an equa-

tion whose ratio is smallest.

System (1) is now used to illustrate feasible pivots. In this system, let y
be the entering variable. No ratio is computed for equation (1.0) because
–z stays basic for that equation. No ratio is computed for equation (1.1)
because the coefficient of y in this equation is not positive. Ratios are com-
puted for equations (1.2), (1.3) and (1.4), and these ratios equal 7, 4.5 and
3, respectively. The pivot occurs on the coefficient of y in equation (1.4) be-
cause that equation’s ratio is smallest. Note that this pivot results in a basic
tableau for which y becomes basic and the variable s4 that had been basic
for equation (1.4) becomes nonbasic. Equation (3.4) with s4 = 0 shows that
yâ•›=â•›3, hence that this pivot keeps the basic solution feasible. In this case and
in general:

In a feasible tableau for Form 1, pivoting on the coefficient of the enter-

ing variable in a row whose ratio is smallest amongst those rows whose
coefficients of the entering variable are positive keeps the basic solution
feasible.
128 Linear Programming and Generalizations

A pivot is said to be feasible if it occurs on the coefficient of the enter-

ing variable in the “pivot row,” where the pivot row has a positive coefficient
of the entering variable and, among all rows having positive coefficients of
the entering variable, the pivot row has the smallest ratio of RHS value to
coefficient of the entering variable. The variable that had been basic for the
pivot row is called the leaving variable. Thus, each feasible pivot causes
the “entering variable” to join the basis and causes the “leaving variable” to
depart.

With x (and not y) as the entering variable in system (1), ratios would be
computed form equations (1.1) and (1.2), these ratios would equal 6/1â•›=â•›6 and
7/1â•›=â•›7, respectively, and a feasible pivot would occur on the coefficient of x
in equation (1.1). This pivot causes s1 to leave the basis, resulting in a basic
tableau whose basic solution has xâ•›=â•›6 and remains feasible. By the way, the
coefficient of x in equation (1.4) equals –1, which is negative, and a pivot on
this coefficient would produce a basic solution having xâ•›=â•›9/(–1)â•›=â•›–9, which
would not be feasible.

A simplex pivot

In a maximization problem, a simplex pivot is a feasible pivot for which

the reduced cost (marginal profit) of the entering variable is positive. Com-
pare equations (1.0) and (3.0) to see that the entering variable for a simplex
pivot can be x or y. As noted previously, setting either of these variables posi-
tive improves the objective.

Is the simplex pivot unambiguous? No, it is not. More than one nonbasic
variable can have marginal profit that is positive. Also, two or more rows can
tie for the smallest ratio.

Rule #1

When illustrating the simplex method, some of the ambiguity in choice

of pivot element is removed by employing Rule #1, which takes the enter-
ing variable as a nonbasic variable whose reduced cost is most positive in
the case of a maximization problem, most negative in the case of a mini-
mization problem. Rule #1 is not unambiguous. More than one nonbasic
variable can have the most positive (negative) reduced cost in a maximiza-
tion (minimization) problem, and two or more rows can tie for the smallest
ratio.
 Chapter 4: Eric V. Denardo 129

The first simplex pivot

Table 4.1 shows how to execute a simplex pivot on a spreadsheet. In this

table, the variable y has been selected as the entering variable (it has the larg-
est reduced cost, and we are invoking Rule #1). The cell containing the label y
has been shaded. The “IF” statements in column J of Table 4.1 compute ratios
for the equations whose coefficients of y are positive. The smallest of these
ratios equals 3 (which is no surprise), and the cell in which it appears is also
shaded. The pivot element lies at the intersection of the shaded column and
row, and it too is shaded.

To execute this pivot, select the block B12:I16, type the function
=pivot(C7, B3:I7) and then hit Ctrl+Shift+Enter to remind Excel that this
is an array function (because it sets values in an array of cells, rather than in
a single cell).

Table 4.1.↜  The first simplex pivot.

The pivot in Table 4.1 causes y to enter the basis and s4 to depart. The
basic solution that results from this pivot remains feasible because it equates
each basic variable other than –z to a nonnegative value.
130 Linear Programming and Generalizations

The change in objective value

This pivot improves z by 9, which equals the product of the reduced cost
(marginal profit) of y and the ratio for its pivot row. This reflects a property
that holds in general and is highlighted below:

In each feasible pivot, the change in the basic solution’s objective value
equals the product of the reduced cost of the entering variable and the
ratio for its pivot row.

This observation is important enough to be recorded as the equation,

(4)
change in the basic solution’s reduced cost of the ratio for its
     
= × .
objective value entering variable pivot row

In Problem A, each pivot will improve the basic solution’s objective value.
That does not always occur, however. The RHS value of the pivot row can
equal 0. If it does equal 0, equation (4) shows that no change occurs in the
basic solution’s objective value. That situation is known as “degeneracy,” and
it is discussed in the next section.

The second simplex pivot

Let us resume the simplex method. For the tableau in rows 12-16 of Table
4.1, x is the only nonbasic variable whose marginal profit is positive; its re-
duced cost equals 3. So x will be the entering variable for the next simplex
pivot. The spreadsheet in Table 4.2 identifies that 3 is the smallest ratio and
displays the tableau that results from a pivot on the coefficient of x in this
row. Equation (4) shows that this pivot will improve the basic solution’s objec-
tive value by 9 = 3 × 3. This pivot causes x to become basic and causes s2
(which had been basic for the pivot row) to become nonbasic. Rows 21-25 of
Table 4.2 exhibit the result of this pivot.

The basic solution to the tableau in rows 21-25 of Table 4.2 has xâ•›=â•›3,
yâ•›=â•›4 and zâ•›=â•›18. The nonbasic variables in this tableau are s2 and s4 . In Fig-
ure  4.3, this basic solution lies at the intersection of the lines s2 = 0 and
s4 = 0. Visually, it is optimal.
 Chapter 4: Eric V. Denardo 131

Table 4.2↜.  The second simplex pivot.

An optimality condition

To confirm, algebraically, that this basic solution is optimal, we write the

equation system depicted in rows 20-25 in the format of a dictionary, that is,
with the nonbasic variables on the right-hand side and with z (rather than –z)
on the left-hand side of the topmost equation.

(5.0) z = 18 − 2.25s2 − 0.25s4 ,

(5.1) s1 = 3 + 0.75s2 − 0.25s4 ,

(5.2) x = 3 − 0.75s2 + 0.25s4 ,

(5.3) s3 = 1 + 0.50s2 + 0.50s4 ,

(5.4) y = 4 − 0.25s2 − 0.25s4 .

In system (5), the variables s2 and s4 are nonbasic. The basic solution
to system (5) is the unique solution to system (5) in which the nonbasic vari-
ables s2 and s4 are equated to zero. This basic solution has zâ•›=â•›18. Since the
132 Linear Programming and Generalizations

coefficients of s2 and s4 in equation (5.0) are negative, any solution that sets
either s2 and s4 to a positive value has z < 18. In brief, the basic solution to
system (5) is the unique optimal solution to Problem A’.

Test for optimality. The basic solution to a basic feasible system for
Form 1 is optimal if the reduced costs of the nonbasic variables are:

•â•‡nonpositive in the case of a maximization problem;

•â•‡nonnegative in the case of a minimization problem.

Recap

Our introduction to Phase II of the simplex method is nearly complete.

For a linear program that is written in Form 1, we have seen how to:

• Execute feasible pivots on a spreadsheet.

• Execute simplex pivots on a spreadsheet.

• Identify the optimal solution.

From the dictionary, we have seen that:

• The reduced cost of each nonbasic variable equals the change that oc-
curs in the objective value if the basic solution is perturbed by setting
that nonbasic variable equal to 1.

• If an equation has a ratio, this ratio equals the value of the entering
variable for which the perturbed solution reduces the equation’s basic
variable to zero.

• The smallest of these ratios equals the largest value of the entering vari-
able that keeps the perturbed solution feasible.

It would be hard to overstate the usefulness of the dictionary.

5.  Degeneracy

In a feasible pivot, the RHS value of the pivot row must be nonnegative. A
feasible pivot is said to be nondegenerate if the right-hand-side value of the
 Chapter 4: Eric V. Denardo 133

pivot row is positive. Similarly, a feasible pivot is said to be degenerate if the

RHS value of the pivot row equals 0.

Nondegenerate pivots

Equation (4) holds for every pivot that occurs on a basic tableau. If a pivot
is nondegenerate:

• The RHS value of the pivot row is positive.

• The coefficient of the entering variable in the pivot row must be posi-
tive, so the ratio for the pivot row must be positive.

• Hence, equation (4) shows that each nondegenerate simplex pivot im-
proves the basic solution’s objective value.

It is emphasized:

Nondegenerate pivots: If a simplex pivot is nondegenerate, the basis

changes and objective value of the basic solution improves.

Degenerate pivots

Let us now interpret equation (4) for the case of a feasible pivot that is
degenerate. In this case:

• The RHS value of the pivot row equals 0.

• This pivot (like any other) multiplies the pivot row by a constant, and
it replaces the other rows by themselves less constants times the pivot
row. Since the pivot is degenerate, the RHS value of the pivot row equals
0, so the pivot changes no RHS values.

• The variables that had been basic for rows other than the pivot row
remain basic for those rows; their values in the basic solution remain as
they were because the RHS values do not change.

• The variable that departs from the basis had equaled zero, and the vari-
able that enters the basis will equal zero.
134 Linear Programming and Generalizations

In brief:

Degenerate pivots: If a feasible pivot is degenerate, the basis changes,

but no change occurs in the basic solution or in its objective.

Cycling

Each nondegenerate simplex pivot improves the basic solution’s objective

value. Each degenerate simplex pivot preserves the basic solution’s objective
value. Hence, each nondegenerate simplex pivot results in a basis whose ob-
jective value improves on any seen previously. There are only finitely many
bases because each basis is a subset of the variables and there are finitely many
subsets. Thus, the simplex method can execute only finitely many nondegen-
erate simplex pivots before it terminates.

On the other hand, each degenerate pivot changes the basis without
changing the basic solution. The simplex method is said to cycle if a sequence
of simplex pivots leads to a basis visited previously. If a cycle occurs, it must
consist exclusively of degenerate pivots.

The simplex method can cycle! In Chapter 6, an example will be exhibited

in which Rule #1 does cycle. In that chapter, the ambiguity in Rule #1 will be
resolved in a way that precludes cycling, thereby assuring finite termination.

In discussions of the simplex method, it is convenient to apply the terms

“degenerate” and “nondegenerate” to basic solutions as well as to pivots. A
basic solution is said to be nondegenerate if it equates every decision vari-
able, with the possible exception of –z, to a nonzero value. Similarly, a basic
solution is said to be degenerate if it equates to zero at least one basic variable,
other than –z.

6.  Detecting an Unbounded Linear Program

Let us recall that a linear program is unbounded if it is feasible and if the

objective value of its feasible solutions can be improved without limit. What
happens if Phase II of the simplex method is applied to an unbounded linear
program? Phase II cannot find an optimal solution because none exists. To
explore this issue, we introduce
 Chapter 4: Eric V. Denardo 135

Program B.╇ Maximize {0x╛╛+╛3y}, subject to the constraints

−x + y ≤ 2,

╇╛x ≥ 0, y ≥ 0.

Please sketch the feasible region of Problem B. Note that its constraints
are satisfied by each pair (x, y) having y ≥ 2 and xâ•›=â•›y – 2; moreover, that each
such pair has objective value of 0xâ•›+â•›3yâ•›=â•›3y, which becomes arbitrarily large
as y increases. To see what happens when the simplex method is applied to
Problem B, we first place it in Form 1, as

Program B’.╇ Maximize {z} subject to the constraints

(6.0) 0x+ 3y – z = 0,

(6.1) –x+ y + s1 = 2,

x ≥ 0 ,â•…â•›y ≥ 0 ,â•…â•›s1 ≥ 0.

Table 4.3 shows what happens when the simplex method is applied to
Problem B’. The first simplex pivot occurs on the coefficient of y in equa-
tion (6.1), producing a basic feasible tableau whose basis excludes x and s1
and is

x = s1 = 0, y = 2, −z = −6.

Table 4.3.↜  Application of the simplex method to Problem B.

136 Linear Programming and Generalizations

Writing rows 7 and 8 in the format of the dictionary produces

(7.0) z = 6 + 3x − 3s1 ,

(7.1) y = 2 + 1x − 1s1 .

Perturbing the basic solution to system (7) by making x positive improves

the objective and increases y. No basic variable decreases. No equation has a
ratio. And the objective improves without limit as x is increased. In brief:

Test for unboundedness. A linear program in Form 1 is unbounded if

an entering variable for a simplex pivot has nonpositive coefficients in
each equation other than the one for which −z is basic.

A maximization problem is unbounded if the marginal profit (reduced

cost) of a nonbasic variable is positive and if perturbing the basic solution by
setting that variable positive causes no basic variable to decrease. The per-
turbed solution remains feasible no matter how large that nonbasic variable
becomes, and its objective value becomes arbitrarily large.

7.  Shadow Prices

A “shadow price” measures the marginal value of a change in a RHS

(right-hand-side) value. Computer codes that implement the simplex method
report the shadow prices for the basis with which the simplex method termi-
nates. These shadow prices can be just as important as the optimal solution.
In Chapter 5, we will see why this is so.

Shadow prices are present not just for the final basis, but at every step
along the way. They guide the simplex method. In Chapter 11, we will see
how they do that.

The Full Rank proviso

It will be demonstrated in Proposition 10.2 (on page 334) that every basis
for the column space of a matrix has the same number of columns. Thus,
every basic tableau for a linear program has the same number (possibly zero)
of trite rows. A linear program is said to satisfy the Full Rank proviso if any
 Chapter 4: Eric V. Denardo 137

basic tableau for its Form 1 representation has a basic variable for each row.
Proposition 10.2 implies that the Full Rank proviso is satisfied if and only if
every basic tableau has one basic variable for each row.

System (1) has a basic variable for each row, so Problem A satisfies the
Full Rank proviso. If a linear program satisfies the Full Rank proviso, its equa-
tions must be consistent, and no basic tableau has a trite row.

A definition

For linear programs that satisfy the Full Rank proviso, each basis pre-
scribes a set of shadow prices, one per constraint. Their definition is high-
lighted below.

Each basis assigns to each constraint of a linear program a shadow price

whose numerical value equals the change that occurs in the basic solu-
tion’s objective value per unit change in that constraint’s RHS value in
the original linear program.

Evidently, each shadow price is a rate of change of the objective value with
respect to the constraint’s right-hand-side (RHS) value. (In math-speak, each
shadow price is a partial derivative.)

Necessarily, the unit of measure of a constraint’s shadow price equals

the unit of measure of the objective divided by the unit of measure of that
constraint. As an example, suppose that the objective is measured in dollars
per week ($/week) and that a particular constraint’s right-hand side value is
measured in hours per week (hours/week); this constraint’s shadow price is
measured in dollars per hour ($/hour) because

($/week) ÷ (hours/week) = ($/week) × (weeks/hour) = ($/hour).

An illustration of shadow prices

Problem A’ is now used to illustrate shadow prices. It satisfies the Full

Rank proviso because system (1) has one basic variable per equation. When
applied to Problem A’, the simplex method encountered three bases, each of
which has its own set of shadow prices.

For the final basis, whose basic solution is in rows 22-25 of Table 4.2, the
shadow price for the 2nd constraint will now be computed. That constraint’s
138 Linear Programming and Generalizations

RHS value in the original linear program value equals 7. Let us ask ourselves:
What would happen to this basic solution if the RHS value of the 2nd con-
straint were changed from 7 to 7â•›+â•›δ? Table 4.4, below, will help us to answer
this question. Table 4.4 differs from the initial tableau (rows 2-7 of Table 4.1)
in that the dashed line records the locations of the “=” signs and in that the
variable δ appears on the right-hand-side of each equation with a coefficient
of 1 in the 2nd constraint and with coefficients of 0 in the other constraints. Ef-
fectively, the RHS value of the 2nd constraint has been changed from 7 to 7â•›+â•›δ.

Table 4.4.↜渀 Initial tableau for Problem A’ with perturbed RHS.

x y s1 s2 s3 s4 –z

----- -------- ---------

RHS δ
2 3 0 0 0 0 1 0 0
1 0 1 0 0 0 0 6 0
1 1 0 1 0 0 0 7 1
0 2 0 0 1 0 0 9 0
╇ –1 3 0 0 0 1 0 9 0

The variables s2 and δ have identical columns of coefficients in Table 4.4.

Recall from Chapter 3 that identical columns stay identical after any sequence
of Gaussian operations. Thus, performing on Table 4.4 the exact sequence of
Gaussian operations that transformed rows 3-7 of Table 4.1 into rows 21-25
of Table 4.2 produces Table 4.5, in which the column of coefficients for δ du-
plicates that of s2 .

Table 4.5.↜渀 The current tableau after the same two pivots.
----- -------- ---------

x y s1 s2 s3 s4 –z RHS δ
0 0 0 –9/4 0 –1/4 1 –18 –9/4
0 0 1 –3/4 0 1/4 0 3 –3/4
1 0 0 3/4 0 –1/4 0 3 3/4
0 0 0 –1/2 1 –1/2 0 1 –1/2
0 1 0 1/4 0 1/4 0 4 1/4

Casting the basic solution to Table 4.5 in the format of a dictionary pro-
duces system (8), below. Equation (8.0) shows that the rate of change of the
 Chapter 4: Eric V. Denardo 139

objective value with respect to the RHS value of the 2nd constraint equals 9/4.
Thus, the shadow price of the 2nd constraint equals 9/4 or 2.25.

(8.0) z = 18 + (9/4)δ

(8.1) s1 = 3 − (3/4)δ

(8.2) x = 3 + (3/4)δ

(8.3) s3 = 1 − (1/2)δ

(8.4) y = 4 + (1/4)δ

The “range” of a shadow price

System (8) prescribes the values of the basic variables in terms of the
change δ in the right-hand side of the 2nd constraint of Problem A. The range
of a shadow price is the interval in its RHS value for which the basic solution
remains feasible. It’s clear from equations (8.1) through (8.4) that the basic
variables stay nonnegative for the values of δ that satisfy the inequalities

s1 = 3 − (3/4)δ ≥ 0,

x = 3 + (3/4)δ ≥ 0,

s3 = 1 − (1/2)δ ≥ 0,

y = 4 + (1/4)δ ≥ 0.

These inequalities are easily seen to hold for δ in the interval

−4 ≤ δ ≤ 2.

The largest value of δ for which the perturbed basic solution remains
feasible is called the allowable increase. The negative of the smallest value of
δ for which the perturbed basic solution remains feasible is called the allow-
able decrease. In this case, the allowable increase equals 2 and the allowable
decrease equals 4.
140 Linear Programming and Generalizations

A break-even price

Evidently, if the RHS value of the 2nd constraint can be increased at a per-
unit cost p below 2.25 (which equals 9/4), it is profitable to increase it by as
many as 2 units, perhaps more. Similarly, if the RHS value of the 2nd constraint
can be decreased at a per-unit revenue p above 2.25, it is profitable to decrease
it by as many as 4 units, perhaps more.

In this example and in general, each constraint’s shadow price is a break-

even price that applies to increases in a constraint’s RHS value up to the
“allowable increase” and to decreases down to the “allowable decrease.”

Economic insight

It’s often the case that the RHS values of a linear program represent levels
of resources that can be adjusted upward or downward. When this occurs, the
shadow prices give the break-even value of small changes in resource levels –
they suggest where it is profitable to invest, and where it is profitable do divest.

Why the term, shadow price?

The term, shadow price, reflects the fact that these break-even prices are
endogenous (determined within the model), rather than by external market
forces.

Shadow prices for “≤” constraints

In Table 4.5, the reduced cost of the slack variable s2 for the 2nd con-
straint equals –9/4, and the shadow price of the 2nd constraint equals 9/4. This
is not a coincidence. It is a consequence of the fact that identical columns stay
identical. In brief:

In any basic tableau, the shadow price of each “≤” constraint equals (−1)
times the reduced cost of that constraint’s slack variable.

In Table 4.5, for instance, the shadow prices for the four constraints are 0,
9/4, 0 and 1/4, respectively.

Shadow prices for “≥” constraints

Except for a factor of (–1), the same property holds for each “≥” con-
straint.
 Chapter 4: Eric V. Denardo 141

In any basic tableau, the shadow price of each “≥” constraint equals the
reduced cost of that constraint’s surplus variable.

This property also holds because identical columns stay identical.

Shadow prices for nonbinding constraints

An inequality constraint in a linear program is said to be binding when it

holds as an equation and to be nonbinding when it holds as a strict inequal-
ity. Suppose the ith constraint in the original linear program is an inequality,
and suppose that the current basis equates this constraint’s slack or surplus
variable to a positive value. Being positive, this variable is basic, so its reduced
cost (top-row coefficient) equals zero. In brief:

If a basic solution causes an inequality constraint to be nonbinding, that

constraint’s shadow price must equal 0.

This makes perfect economic sense. If a resource is not fully utilized, a

small change in the amount of that resource has 0 as its marginal benefit.

Graphical illustration

For a graphical interpretation of shadow prices and their ranges, we re-

turn to Problem A. Figure 4.4 graphs its feasible region for various values of
the RHS of its 2nd constraint, i.e., with that constraint as xâ•›+â•›y ≤ 7â•›+â•›δ and with
δ between –4 andâ•›+â•›2. Figure 4.4 includes the objective vector, which is (2, 3).
The optimal solution to Problem A is the feasible solution that lies farthest
in the direction of its objective vector. For δ between –4 and +2, this optimal
solution lies at the intersection of the lines

(9) −x + 3y = 9 and x + y = 7 + δ.

Solving these two equations for x and y gives

(10) x = 3 + (3/4)δ and y = 4 + (1/4)δ,

and substituting these values of x and y in the objective function gives

(11) z = 2x + 3y = 16 + (6/4)δ + 12 + (3/4)δ = 18 + (9/4)δ.

This reconfirms that the shadow price of the 2nd constraint equals 9/4.
142 Linear Programming and Generalizations

Note in Figure 4.4 that as δ ranges between –4 andâ•›+â•›2, the optimal solu-

tion shifts along the heavily-outlined interval in Figure 4.4. When δ equals
+2, the constraint 2y ≤ 9 holds as an equation. When δ exceeds +2, the per-
turbed solution violates the constraint 2y ≤ 9. This reconfirms +2 as the al-
lowable increase. A similar argument shows why 4 is the allowable decrease.

Figure 4.4.↜  Perturbing the constraint xâ•›+â•›y ≤ 7.

y
objective vector
9
x+y≤9
8
3
–x + 3y ≤ 9
7
x+y≤7
6 2

5 x+y≤3

4
2y ≤ 9
3

2 feasible region
x≥0 1

0 x
0 1 2 3 4 5 6 7 8 9

Perturbing multiple RHS values

For Problem A, consider the effect of adding δ2 units to the RHS of the
2nd constraint and adding δ4 units to the RHS of the 4th constraint. Let us
ask ourselves: What effect would this have on the basic solution for the basis
in Table 4.5? Inserting δ4 on the RHS of Table 4.4 with a coefficient of +1
in the 4th constraint and repeating the above argument (the variables s4 and
δ4 have identical columns of coefficients) indicates that the basic solution
becomes
 Chapter 4: Eric V. Denardo 143

(12.0) z = 18 + (9/4)δ2 + (1/4)δ4 ,

(12.1) s1 = 3 − (3/4)δ2 + (1/4)δ4 ,

(12.2) x = 3 + (3/4)δ2 − (1/4)δ4 ,

(12.3) s3 = 1 − (1/2)δ2 − (1/2)δ4 ,

(12.4) y = 4 + (1/4)δ2 + (1/4)δ4 .

Evidently, the objective value changes by (9/4)δ2 + (1/4)δ4 . In this case

and in general, the shadow prices apply to simultaneous changes in two or
more RHS values.

These shadow prices continue to be break-even prices as long as the val-

ues of the basic variables s1 , x, s3 and y remain nonnegative. In particular,
the RHS of equation (12.1) is nonnegative for all values of δ2 and δ4 that
satisfy the inequality

3 − (3/4)δ2 + (1/4)δ4 ≥ 0.

In Chapter 3, it was noted that the set of ordered pairs (δ2 , δ4 ) that sat-
isfy a particular linear inequality, such as the above, is a convex set. It was also
observed that the intersection of convex sets is convex. In particular, the set
of pairs (δ2 , δ4 ) for which the basic solution remains feasible (nonnegative)
is convex. In brief:

Perturbed RHS values: Each basis’s shadow prices apply to simultane-

ous changes in two or more RHS values, and the set of RHS values for
which the basis remains feasible is convex.

Note also that perturbing the RHS values of the original tableau af-
fects only the RHS values of the current tableau. It has no effect on the
coefficients of the decision variables in any of the equations. In particular,
these perturbations have no effect on the reduced costs (top-row coef-
ficients). If the reduced costs satisfy the optimality conditions before the
perturbation occurs, they continue to satisfy it after perturbation occurs.
It is emphasized:
144 Linear Programming and Generalizations

Optimal basis: Consider a basis that is optimal. If one or more RHS

value is perturbed, its basic solution changes, but it remains optimal as
long as it remains feasible.

In Chapter 5, we will see that the shadow prices are central to a key idea in
economics, namely, the “opportunity cost” of doing something new. In Chap-
ter 12, the shadow prices will emerge as the decision variables in a “dual”
linear program.

Computer output, multipliers and the proviso

Every computer code that implements the simplex method finds and re-
ports a basic solution that is optimal. Most of these codes also report a shadow
price for each constraint, along with an allowable increase and an allowable
decrease for each RHS value.

If the Full Rank proviso is violated, not all of the constraints can have
shadow prices. These computer codes report them anyhow! What these codes
are actually reporting are values of the basis’s “multipliers” (short for Lagrange
multipliers). In Chapter 11, it will be shown that these “multipliers” coincide
with the shadow prices when they exist and, even if the shadow prices do not
exist, the multipliers account correctly for the marginal benefit of perturbing
the RHS values in any way that keeps the linear program feasible.

8.  Review

Linear programming has its own specialized vocabulary. Learning the

vocabulary eases access to the subject. In this review, the specialized termi-
nology that was introduced in this chapter appears in italics. A crucial idea in
this chapter is the feasible pivot. Before proceeding, make certain that you un-
derstand what feasible pivots are and that you can execute them on a spread-
sheet.

Recap of the simplex method

Listed below are the most important of the properties of the simplex
method.

• The simplex method pivots from one basic feasible tableau to another.
 Chapter 4: Eric V. Denardo 145

• Geometrically, each basic feasible tableau identifies an extreme point of

the feasible region.

• In each basic feasible tableau, the reduced cost of each nonbasic vari-
able equals the amount by which the basic solution’s objective value
changes if that nonbasic variable is set equal to 1 and if the values of
the basic variables are adjusted to preserve a solution to the equation
system.

• The entering variable in a simplex pivot can be any nonbasic variable

whose reduced cost is positive in the case of a maximization problem,
negative in the case of a minimization problem.

• Each simplex pivot occurs on a positive coefficient of the entering vari-

able, and that coefficient has the smallest ratio (of RHS value to coef-
ficient).

• If the RHS value of the pivot row is positive, the pivot is nondegenerate.
Each nondegenerate simplex pivot improves the basic solution’s objec-
tive value.

• If the RHS value of the pivot row is zero, the pivot is degenerate. Each
degenerate pivot changes the basis, but causes no change in the basic
solution or in its objective value.

• The simplex method identifies an optimal solution when it encounters

a basic feasible tableau for which the reduced cost of each nonbasic
variable is nonpositive in a maximization problem, nonnegative in a
minimization problem.

• The simplex method identifies an unbounded linear program if the en-

tering variable for a simplex pivot has nonpositive coefficients in every
row other than the one for which –z is basic.

• A linear program satisfies the Full Rank proviso if any basis has as many
basic variables as there are constraints in the linear program’s Form 1
representation.

• If the Full Rank proviso is satisfied, each basic feasible tableau has these
properties:
146 Linear Programming and Generalizations

–╇The shadow price of each constraint equals the rate of change of the
basic solution’s objective value with respect to the constraint’s RHS
value.

–╇These shadow prices apply to simultaneous changes in multiple RHS

values.

–╇If only a single RHS value is changed, the shadow price applies to
increases as large as the allowable increase and to decreases as large
as the allowable decrease.

What has been omitted?

This chapter is designed to enable you to make intelligent use of comput-

er codes that implement the simplex method. Facets of the simplex method
that are not needed for that purpose have been deferred to later chapters. In
later chapters, we will see that:

• Phase I of the simplex method determines whether or not the linear

program has a feasible solution and, if so, constructs a basic feasible
tableau with which to initiate Phase II.

• Rule #1 can cause the simplex method to cycle, and the ambiguity in
Rule #1 can be resolved in a way that precludes cycling, thereby guar-
anteeing finite termination.

• In some applications, decision variables that are unconstrained in sign

are natural. They can be accommodated directly, without forcing the
linear program into the format of Form 1.

• If the Full Rank proviso is violated, each basis still has “multipliers” that
correctly account for the marginal value of any perturbation of the RHS
values that keeps the linear program feasible.

Not a word has appeared in this chapter about the speed of the simplex
method. For an algorithm to be useful, it must be fast. The simplex method
is blazingly fast on nearly every practical problem. But examples have been
discovered on which it is horrendously slow. Why that is so has remained a
bit of a mystery for over a half century. Chapter 6 touches lightly on the speed
of the simplex method.
 Chapter 4: Eric V. Denardo 147

9.  Homework and Discussion Problems

1. On a spreadsheet, execute the simplex method on Problem A with x as the

entering variable for the first pivot. Use Figure 4.3 to interpret its progress.

2. Rule #1 picks the most positive entering variable for a simplex pivot on a
maximization problem. State a simplex pivot rule that makes the largest
possible improvement in the basic solution’s objective value. Use Problem
A to illustrate this rule.

3. In system (1), execute a pivot on the coefficient of x in equation (1.2).

What goes wrong?

4. (graphical interpretation) Each part of this problem refers to Figure 4.3.

(a) The coefficient of y in equation (1.1) equals zero. How is this fact re-
flected in Figure 4.3? Does a similar interpretation apply to the coef-
ficient of x in equation (1.3)?

(b) With x as the entering variable, no ratio was computed for equa-
tion (1.4). If this ratio had been computed, it would have equaled
9/(–1)â•›=â•›–9. Use Figure 4.3 to interpret this number.

(c) True or false: Problem A’ has a feasible basis whose nonbasic variables
are x and s4 .

5. (graphical interpretation) It is clear from Figure 4.3 that system (1) has 5

bases that include –z and are feasible. Use Figure 4.3 to identify each basis
that includes –z and is not feasible.

6. (graphical interpretation) True or false: for Problem A’, every set that in-
cludes –z and all but two of the variables x, y and s1 through s4 is a basis.

7. Consider this linear program: Maximize {x}, subject to the constraints

– x + y ≤ 1,

x + y ≤ 4,

x – y ≤ 2,
x≥0, y ≥ 0.
(a) Solve this linear program by executing simplex pivots on a spread-
sheet.
148 Linear Programming and Generalizations

(b) Solve this linear program graphically, and use your graph to trace the
progress of the simplex method.

8. (no extreme points?) Consider this linear program: Maximize (A – B) sub-

ject to the constraints

â•› A – B ≤ 1,

– A + B ≤ 1.

(a) Plot this linear program’s feasible region. Does it have any extreme
points?

(b) Does this linear program have an optimal solution? If so, name one.

(c) Apply the simplex method to this linear program. What happens?

9. Consider this linear program: Maximize {xâ•›+â•›1.5y}, subject to the con-

straints

â•› x â•›≤ 4,
â•›– x + y â•›≤ 2,
2x + 3y â•› ≤ 12,
x≥0, y ≥ 0.

(a) Solve this linear program by executing simplex pivots on a spread-

sheet.

(b) Execute a feasible pivot that finds a second optimal solution to this
linear program.

(c) Solve this linear program graphically, and use your graph to trace the
progress of the simplex method.

(d) How many optimal solutions does this linear program have? What are
they?

10. For the linear program that appears below, construct a basic feasible sys-
tem, state its basis, and state its basic solution.
 Chapter 4: Eric V. Denardo 149

Maximize {2y – 3z}, subject to the constraints

â•…â•…â•…â•…â•… x + y – z = 16,
â•›y +â•›z â•›≤ 12,
╅╅╅╅╅╅╅╇╛2y –â•›z ≥ – 10,
x ≥ 0, y ≥ 0, z ≥ 0.

(a) True or false: Problem A’ has a feasible basis whose nonbasic variables
are x and s2 .

(b) True or false: Problem A’ has a feasible basis whose nonbasic variables
are x and s3 .

11. (an unbounded linear program) Draw the feasible region for Problem B’. Ap-
ply the simplex method to Problem B’, selecting y (and not x) as the entering
variable for the first pivot. What happens? Interpret your result graphically.

12. (degeneracy in 2-space) This problem concerns the variant of Problem A in

which the right-hand-side value of equation (1.4) equals 0, rather than 9.

(a) On a spreadsheet, execute the simplex method, with y as the entering

variable for the first pivot.

(b) Draw the analog of Figure 4.3 for this linear program.

(c) List the bases and basic solutions that were encountered. Did a degen-
erate pivot occur?

(d) Does this linear program have a redundant constraint (defined

above)?

13. (degeneracy in 3-space) This problem concerns the linear program: Maxi-
mize {xâ•›+â•›1.5yâ•›+â•›z} subject to the constraints x + y ≤ 1, y + z ≤ 1, x ≥ 0,
y ≥ 0, z ≥ 0.

(a) Use the simplex pivots with Rule #1 to solve this linear program on a
spreadsheet. Did a degenerate pivot occur?

(b) Plot this linear program’s feasible region. Explain why a degenerate
pivot must occur.

(c) True or false: If a degenerate pivot occurs, the linear program must
have a redundant constraint.
150 Linear Programming and Generalizations

14. (degeneracy in 3-space) This problem concerns the linear program:

Maximize {0.1xâ•›+â•›1.5yâ•›+â•›0.1z} subject to the constraints x + y ≤ 1,
y + z ≤ 1, x ≥ 0, y ≥ 0, z ≥ 0.

(a) Use the simplex pivots with Rule #1 to solve this linear program on a
spreadsheet. Did a degenerate pivot occur?

(b) True or false: The simplex method stops when it encounters an opti-
mal solution.

15. True or false:

(a) A nondegenerate pivot can result in a degenerate basic system.

(b) A degenerate pivot can result in a nondegenerate basic system.

16. Consider a basic feasible tableau that is nondegenerate, so that its basic
solution equates all variables to positive values, with the possible excep-
tion of –z. Complete the following sentence and justify it: A feasible pivot
on this tableau will result in a degenerate tableau if and only if a tie occurs
for_______.

17. True or false: For a linear program in Form 1, feasible pivots are the only
pivots that keep the basic solution feasible.

18. (redundant constraints) Suppose that you need to learn whether or not the
ith constraint in a linear program is redundant.

(a) Suppose the ith constraint is a “≤” inequality. How could you find out
whether or not this constraint is redundant? Hint: use a linear pro-
gram.

(b) Suppose the ith constraint is an equation. Hint: use part (a), twice.

19. (maximizing a decision variable) Alter Program 1 so that its objective is to

maximize y, but its constraints are unchanged. Adapt the simplex method
to accomplish this directly, that is, without introducing an equation that
defines z as the objective value. Execute your method on a spreadsheet.

20. (bases and shadow prices) This problem refers to Table 4.1.

(a) For the basis solution in rows 2-7, find the shadow price for each con-
straint.
 Chapter 4: Eric V. Denardo 151

(b) For the basic solution in rows 11-16, find the shadow price for each
constraint.

(c) Explain why some of these shadow prices equal zero.

21. Adapt Table 4.4 and Table 4.5 to compute the shadow price, the allowable
increase, and the allowable decrease for the optimal basis of the RHS val-
ue of the constraint – xâ•›+â•›3y ≤ 9. Which previously nonbinding constraint
becomes binding at the allowable increase? At the allowable decrease?

22. On the plane, plot the set S that consists of all pairs (δ2 , δ4 ) for which
the basic solution to system (12) remains feasible. For each point on the
boundary of the set S that you have plotted, indicate which constraint(s)
become binding.

23. Suppose every RHS value in Problem A is multiplied by the same positive
constant, for instance, by 10.5. What happens to the optimal basis? To the
optimal basic solution? To the optimal value? To the optimal tableau? Why?

24. This concerns the minimization problem whose Form 1 representation is

given in the tableau that follows.

(a) It this a basic tableau? Is its basis feasible?

(b) To make short work of Phase I, pivot on the coefficient of B in equa-

tion (1.2). Then continue Phase II to optimality.

25. True or false: When the simplex method is executed, a variable can:

(a) Leave the basis at a pivot and enter at the next pivot. Hint: If it entered,
to which extreme point would it lead?

(b) Enter at a pivot and leave at the next pivot. Hint: Maximize {2yâ•›+â•›x},
subject to the constraints 3y + x ≤ 3, x ≥ 0, y ≥ 0.

26. The simplex method has been applied to a maximization problem in

Form 1 (so that all variables other than –z are constrained to be nonnega-
tive). At some point in the computation, the tableau that is shown below
has been encountered; in this tableau, u, v, w and x denote numbers.
152 Linear Programming and Generalizations

State the conditions on u, v, w and x such that:

(a) The basic solution to this tableau is the unique optimal solution.

(b) The basic solution to this tableau is optimal, but is not the unique
optimal solution.

(c) The linear program is unbounded.

(d) The linear program has no feasible solution.

27. (nonnegative column) For a maximization problem in Form 1, the follow-

ing tableau has been encountered. In it, * stands for an unspecified data
element. Prove there exist no values of the unspecified data for which it
is optimal to set A > 0. Hint: If a feasible solution exists with A > 0, show
that it is profitable to decrease A to zero and increase the values of B, D
and F in a particular way.

28. (nonpositive row) For a maximization or a minimization problem in Form

1, the following tableau has been encountered. In it, * stands for an un-
specified data element.

(a) Prove that B is basic in every basic feasible tableau.

(b) Prove that deleting B and the equation for which it is basic can have
no effect either on the feasibility of this linear program or on its opti-
mal value.
 Chapter 5: Analyzing Linear Programs

1.╅ Preview����������������������������������尓������������������������������������尓���������������������� 153

2.╅ All Terrain Vehicles����������������������������������尓������������������������������������尓���� 154
3.╅ Using Solver ����������������������������������尓������������������������������������尓�������������� 158
4.╅ Using the Premium Solver for Education����������������������������������尓���� 162
5.╅ Differing Sign Conventions!����������������������������������尓�������������������������� 163
6.╅ A Linear Program as a Model����������������������������������尓���������������������� 165
7.╅ Relative Opportunity Cost����������������������������������尓���������������������������� 168
8.╅ Opportunity Cost����������������������������������尓������������������������������������尓������ 175
9.╅ A Glimpse of Duality* ����������������������������������尓���������������������������������� 179
10.╇ Large Changes and Shadow Prices* ����������������������������������尓������������ 183
11.╇ Linear Programs and Solid Geometry*����������������������������������尓�������� 184
12.╇ Review����������������������������������尓������������������������������������尓������������������������ 186
13╇ Homework and Discussion Problems����������������������������������尓������������ 187

1.  Preview

Dozens of different computer packages can be used to compute optimal

solutions to linear programs. From this chapter, you will learn how to make
effective use of these packages.

This chapter also addresses the fact that a linear program – like any math-
ematical model – is but an approximation to the situation that is under study.
The information that accompanies the optimal solution to a linear program
can help you to determine whether or not the approximation is a reasonable
one.

E. V. Denardo, Linear Programming and Generalizations, International Series 153

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_5, © Springer Science+Business Media, LLC 2011
154 Linear Programming and Generalizations

Also established in this chapter is a close relationship between three eco-

nomic concepts – the break–even (or shadow) price on each resource, the
relative opportunity cost of engaging in each activity, and the marginal ben-
efit of so doing. It will be seen that “relative opportunity cost” carries a some-
what different meaning than “opportunity cost,” as that term is used in the
economics literature.

Three sections of this chapter are starred because they can be read inde-
pendently of each other. One of these starred sections provides a glimpse of
duality.

2.  All Terrain Vehicles

Much of the material in this chapter will be illustrated in the context of

the optimization problem that appears below as

Problem A (All Terrain Vehicles)1.╇ Three models of All Terrain Vehicle

(ATV) are manufactured in a facility that consists of five shops. Table  5.1
names the vehicles and the shops. It specifies the capacity of each shop and
the manufacturing time of each vehicle in each shop. It also specifies the con-

Table 5.1.↜渀 The ATV Manufacturing Facility.

â•…â•…â•…   Manufacturing times

Shop Capacity Standard Fancy Luxury
Engine 120 3 2 1
Body 80 1 2 3
Standard Finishing 96 2
Fancy Finishing 102 3
Luxury Finishing 40 2
Contribution 840 1120 1200
Note on units of measure: In Table 5.1, capacity is measured in hours per
week, manufacturing time in hours per vehicle, and contribution in dollars
per vehicle.

This example has a long history. An early precursor appears in the article by Robert
1╇

Dorfman, “Mathematical or ‘linear’ programming: A nonmathematical exposition,”

The American Economic Review, V. 13, pp. 797-825, 1953.
 Chapter 5: Eric V. Denardo 155

tribution (profit) earned by manufacturing each vehicle. The plant manager

wishes to learn the production rates (numbers of each vehicle to produce per
week) that maximize the profit that can be earned by this facility.

Contribution

“Contribution,” as used in this book, takes its meaning from accounting.

When one contemplates taking an action, a variable cost is an expense that
is incurred if the action is taken and only if the action is taken. When one
contemplates an action, a fixed cost is a cost that has occurred or will occur
whether or not the action is taken. Decisions should not be influenced by
fixed costs.

When one is allocating this week’s production capacity, the variable cost
normally includes the material and energy that will be consumed during pro-
duction, and the fixed cost includes depreciation of existing structures, prop-
erty taxes, and other expenses that are unaffected by decisions about what to
produce this week.

The contribution of an action equals the revenue that it creates less its
variable cost. This usage abbreviates the accounting phrase, “contribution to-
ward the recovery of fixed costs.” Table 5.1 reports $840 as the contribution
of each Standard model vehicle. This means that $840 equals the sales price
of a Standard model vehicle less the variable cost of manufacturing it. When
profit is used in this book, what is meant is contribution.

Maximizing contribution

The manager of the ATV plant seeks the mix of activities that maximizes
the rate at which contribution is earned, measured in dollars per week. At a
first glance, the Luxury model vehicle seems to be the most profitable. It has
the largest contribution. Each type of vehicle consumes a total of 4 hours of
capacity in the Engine and Body shops, where congestion is likely to occur.
But we will see that no Luxury model vehicles should be manufactured, and
we will come to understand why that is so.

The decision variables

Let us formulate the ATV problem for solution via linear programming.
Its decision variables are the rates at which to produce the three types of ve-
hicles, which are given the names:
156 Linear Programming and Generalizations

S = the rate of production of Standard model vehicles (number per week),

F = the rate of production of Fancy model vehicles (number per week),

L = the rate of production of Luxury model vehicles (number per week).

Evidently, mnemonics (memory aids) are being used; the labels S, F and
L abbreviate the production rates for Standard, Fancy and Luxury model ve-
hicles.

Inequality constraints

The ATV problem places eight constraints on the values taken by the
decision variables. Three of these constraints reflect the fact that the produc-
tion quantities cannot be negative. These three are S ≥ 0, F ≥ 0, and L ≥ 0.
The remaining five constraints keep the capacity of each shop from being
over-utilized. The top line of Table 5.1 shows that producing at rates S, F, and
L vehicles per week consumes the capacity of the Engine shop at the rate of
3Sâ•›+â•›2Fâ•›+â•›1L hours per week, so the constraint

3S + 2F + 1L ≤ 120

keeps the number of hours consumed in the Engine shop from exceeding its
weekly capacity. The expression, {840Sâ•›+â•›1120Fâ•›+â•›1200L}, measures the rate at
which profit is earned. The complete linear program is:

Program 1:╇ Maximize {840S╛+╛1120F╛+╛1200L}, subject to the constraints

Engine: 3S + 2F + 1L ≤ 120,
Body: 1S + 2F + 3L ≤ 80,
Standard Finishing: 2S ≤ 96,
Fancy Finishing: ╅╇╛3F ≤ 102,
Luxury Finishing: ╅╅╅╅╅╛╛╛╛╛2L ≤ 40,

S ≥ 0, F ≥ 0, L ≥ 0.
 Chapter 5: Eric V. Denardo 157

Integer-valued variables?

As written, Program 1 allows the decision variables to take fractional val-

ues. This makes sense. The manager wishes to determine the profit-maximiz-
ing rate of production of each vehicle. For instance, setting Sâ•›=â•›4.25 amounts
to producing Standard model vehicles at the rate of 4.25 per week.

If the production quantities had been required to be integer-valued, Pro-

gram 1 would be an “integer program,” rather than a linear program, and
could be a great deal more difficult to solve. Integer programming is dis-
cussed in later chapters.

A spreadsheet

Table 5.2 prepares the ATV problem for solution on a spreadsheet. Note

that:

• Information about Standard, Fancy and Luxury model vehicles appears

in columns B, C and D, respectively. In particular:

–╇Cells B2, C2 and D2 contain the labels of these decision variables.

–╇Cells B9, C9 and D9 are reserved for the values of these decision
variables, each of which has been set equal to 1, temporarily.

–╇Cells B3, C3 and D3 contain their contributions.

–╇Cells B4:D4 contain the manufacturing times of each vehicle in the

Engine shop.

–╇Rows 5 through 8 contain comparable data for the other four shops.

• Column E contains (familiar) sumproduct functions, and cells E4

through E8 record their values when each decision variable is set equal
to 1.

• Column F contains “<=” signs. These are memory aids; they remind us
that the quantities to their left must not exceed the RHS values to their
right.

• Column G contains the capacities of the five shops.

158 Linear Programming and Generalizations

It remains for Solver to select values in cells B9, C9 and D9 that maximize
the value in cell E3 while enforcing the constraints of Program 1.

Table 5.2.↜  A Spreadsheet for the ATV problem.

A standard format

Table 5.2 presents the data for the ATV problem in a standard format,
which consists of:

• One row for the labels of the decision variables (row 2 in this case).

• One row for the values of the decision variables (row 9 in this case).

• One row for the contribution of each decision variable (row 3).

• One row for each constraint (rows 4 to 8).

• One column for the coefficients of each decision variable, one column
for the sumproduct functions that measure the consumption of each
resource, one column for the RHS values, and one (optional) column
that records the sense of each constraint.

This standard format is a handy way in which to prepare a linear program

for solution on a spreadsheet. It will be used repeatedly in this book.

3.  Using Solver

This is the first of two sections that describe slightly different ways to
compute the optimal solution to the ATV problem. This section is focused
 Chapter 5: Eric V. Denardo 159

on Solver, which comes with Excel. The next section is focused on Premium
Solver for Education, which is on the disc that accompanies this book.

The Solver Parameters dialog box

Figure 5.1 displays a Solver dialog box, which has been filled out. This
dialog box identifies E3 as the cell whose value we wish to maximize, it speci-
fies cells B9:D9 as the changing cells, and it imposes constraints that keep
the quantities in cells B9:D9 nonnegative and keep the quantities in cells E4
through E8 from exceeding the quantities in cells G4 through G8, respec-
tively.

Figure 5.1.↜  A Solver Dialog box for the ATV problem.

Chapter 2 tells how to fill out this dialog box. As was indicated in Chapter
2, the Solver dialog box for Excel 2010 differs slightly from the above, but is
filled out in a similar way.

Getting an optimal solution

After you have reproduced Figure 5.1, remember to click on the Options

button and, on the menu that appears, click on Assume Linear Model, then
on OK. Then click on the Solve button. In a flash, Solver will report that it has
found an optimal solution, which is Sâ•›=â•›20, Fâ•›=â•›30 and Lâ•›=â•›0. Solver will also
report an optimal value of 50,400. Your spreadsheet will resemble Table 5.3,
but no cells will (yet) be shaded.
160 Linear Programming and Generalizations

Table 5.3.↜  An optimal solution to the ATV problem.

Binding constraints

Let us recall from Chapter 4 that an inequality constraint in a linear pro-

gram is said to be binding when it holds as an equation and to be nonbind-
ing when it holds as a strict inequality. Evidently, the optimal solution in
Table 5.3 fully utilizes the capacities of the Engine and Body shops, but not
of the finishing shops. This optimal solution has three binding constraints,
which are:

• the nonnegativity constraint on L,

• the constraint on Engine shop’s capacity,

• the constraint on Body shop’s capacity.

In Table  5.3, the shaded cells identify these binding constraints. It will
soon be argued that it is optimal to keep these constraints binding even if the
model’s data are somewhat inaccurate.
 Chapter 5: Eric V. Denardo 161

A sensitivity report

The Solver Results dialog box (see Table 5.3) has a window containing
the word “sensitivity.” Clicking on it creates a sheet containing a Sensitivity
Report that is reproduced as Table 5.4.

Table 5.4.↜  A Sensitivity Report for the ATV example.

Each constraint in the ATV problem is an inequality, so the slack variables

form a basis for its Form 1 representation. This guarantees that the Full Rank
proviso is satisfied, hence that the shadow prices exist. Recall from Chapter 4
that the information in the Sensitivity Report has this interpretation:

• The shadow price for a constraint equals the rate of change of the basic
solution’s objective value with respect to the RHS value of that con-
straint. The basic solution remains feasible (and hence optimal) for in-
creases in its RHS value up to the Allowable Increase and for decreases
up to the Allowable Decrease.

• The shadow prices apply to simultaneous changes, albeit with smaller

ranges.

• The optimal solution is unchanged if a single contribution is increased

by an amount up to its Allowable Increase or is decreased by an amount
up to its Allowable Decrease.
162 Linear Programming and Generalizations

• The reduced cost of each basic variable equals zero, and the reduced
cost of each nonbasic variable equals the amount by which the optimal
value changes if that variable is set equal to 1 and the values of the basic
variables are adjusted accordingly.

In particular:

• The capacity of the Engine shop has a break-even value of 140 $/hour,
and this value applies to increases of up to 56 hs/wk and to decreases up
to 16 hr/wk. Hence, an increase of up to 56 hours per week of Engine
shop capacity is profitable if it can be obtained at a price below 140 dol-
lars per hour. And a decrease in Engine shop capacity of up to 16 hours
per week is profitable if it be put to an alternative use that is worth more
than 140 dollars per hour.

• The optimal solution to the ATV problem is unchanged if the contribu-

tion of the Standard model vehicle is between $560 (because 560â•›=â•›840
– 280) and $1,040 (because 1040â•›=â•›840â•›+â•›200).

• Making one Luxury model vehicle reduces the rate of profit by

200 $/wk.

You forgot?

Sooner or later, nearly everyone who uses Solver will forget to check off
Assume Linear Model before solving a linear program. If you forget, the “en-
gine” that solves your linear program will not be the simplex method, but a
more general algorithm. It computes the correct shadow prices but calls them
Lagrange multipliers, it computes the correct reduced costs, but calls them
reduced gradients, and it omits the Allowable Increases and Allowable De-
creases because it presumes the problem is nonlinear.

4.  Using the Premium Solver for Education

Premium Solver for Education has added features and fewer bugs than do
the earlier versions of Solver. If you have a choice, use the Premium version.
Chapter 2 tells how to install and activate it. After it is activated, Premium
Solver will appear on the Add-Ins tab of the Excel? File menu.
 Chapter 5: Eric V. Denardo 163

To illustrate the use of the Premium Solver dialog box, we arrange for
it to solve the ATV problem. The first step is to replicate the spreadsheet in
Table 5.3. Next, click on the Add-Ins tab on the File menu. An icon labeled
Premium Solver will appear at the left, just below the File tab. Click on it. A
Solver Parameters dialog box will appear. To make it look like that in Fig-
ure 5.2, follow the procedure that is described in Chapter 2. After you suc-
ceed, click on Solve. In a flash, a solution will appear, along with the usual box
that accords you the opportunity to obtain a sensitivity report.

Figure 5.2.↜  A Premium Solver dialog box for the ATV problem.

5.  Differing Sign Conventions!

Solver and Premium Solver report reduced costs and shadow prices as
they are defined in Chapter 4. Some computer packages use different conven-
tions as to the signs (but not the magnitudes) of the reduced costs and the
shadow prices. If you are using a different software package to solve linear
programs, you will need to figure out what sign conventions it employs. An
easy way to do that is described below.

A maximization problem

To see what sign conventions a particular computer package uses for max-
imization problems, you can ask it to solve a simple linear program, such as
164 Linear Programming and Generalizations

Example 1.╇ Maximize {2x╛ + ╛4y}, subject to

3x + 3y ≤ 6,
x ≥ 0 ,â•… y ≥ 0.

Clearly, the optimal solution to Example 1 is xâ•›=â•›0 and yâ•›=â•›2, and its opti-
mal value equals  8. It is equally clear that:

• Increasing the RHS value of its constraint from 6 to 7 increases y by 1/3,

so the shadow price of the constraint equals 4/3.

• Perturbing the optimal solution by setting xâ•›=â•›1 reduces y by 1 and

changes the optimal value by –2, so the reduced cost of y equals –2.

Whatever sign conventions your computer program reports for Example

1 will hold for all maximization problems. If it reverses the sign of the reduced
cost in Example 1, it will do so for all maximization problems.

A simple minimization problem

Similarly, to find out what sign conventions a computer package employs

when it is solving minimization problems, you can ask it to solve Example 2,
below, and ask for a sensitivity report.

Example 2.╇ Minimize {2x +  4y}, subject to

â•›3x + 3y ≥ 6,
â•› x ≥ 0,â•… y ≥ 0.

The optimal solution to Example 2 is xâ•›=â•›2 and yâ•›=â•›0, and its optimal value
equals  4. Evidently:

• Increasing the RHS value of its constraint from 6 to 7 increases x by 1/3,

so the shadow price of the 1st constraint equals 2/3.

• Perturbing the optimal solution by setting yâ•›=â•›1 reduces x by 1 and

changes the optimal value by 2, so the reduced cost of y equals 2.

Whatever sign conventions a particular computer package reports for Ex-

ample 2 will apply to all minimization problems.
 Chapter 5: Eric V. Denardo 165

6.  A Linear Program as a Model

Like any mathematical model, a linear program is an inexact representa-

tion of reality. Linear programs approximate reality in three ways – by elimi-
nating uncertainty, by aggregating activities, and by suppressing nonlineari-
ties. The ATV problem is now used to illustrate these approximations.

Uncertain data

This model’s data are uncertain because they cannot be measured pre-
cisely and because they can fluctuate in unpredictable ways. In the ATV
model, it is presumed that, in an “representative” week, 120 machine hours
are available in the Engine shop; 120 equals the number of machine hours
during which the Engine shop is open for business less allowances for routine
maintenance, machine breakdowns, shortages of vital parts, absences of key
workers, power failures, and other unforeseen events. The actual number of
machine hours available in a particular week could be larger or smaller than
120, depending on how things turned out that week.

Similarly, the contribution of $840 per Standard model approximates an

uncertain quantity. The actual contribution could be larger or smaller than
this figure, depending on the current prices of raw materials, defects that re-
quire abnormal amounts of rework, market conditions that affect the sales
revenues, and changes in inventory carrying costs. Uncertainty in the data is
one reason why models are approximate.

Aggregation

A second reason why models are approximate is aggregation, which re-

fers to the lumping together of several activities in a single entity. The assem-
bly times in the ATV model reflect aggregation. The Engine shop is modeled
as a single entity, but it is actually a system consisting of people, tools and
machines that can produce the engines and drive trains for the three vehicles
at different rates. In our simplified view, it takes 3 hours of Engine shop time
to make each Standard model vehicle. Aggregation is useful when it avoids
detail that is unimportant.

Linearization

Linearization is the third way in which the ATV model is approximate.

The capacity constraint for the Engine shop is 3Sâ•›+â•›2Fâ•›+â•›1L ≤ 120. This con-
166 Linear Programming and Generalizations

straint presumes linear interactions among the three types of vehicles that are
produced there. The actual interactions are more complicated and are some-
what nonlinear. For example, this constraint accounts crudely for the set-up
times that are needed to change from the production of one model to another.

The value of an approximation

The ATV example is aggregated and simplified. It has to be. Imagine how
intractable. this model would become if it incorporated all of the complexi-
ties and details just mentioned. Yet, there is merit in starting with a simple
and aggregated model. It will be relatively easy to build and debug. And if it
is artfully built, its simplicity will cause the main insights to stand out starkly.

Robustness

It would be foolish to believe the results of a simplified model without

first considering how its simplifications influenced its optimal solution. If the
insights obtained from the model hold up in the real world, the model is said
to be robust. To ascertain whether a model is robust, we can execute a sensi-
tivity analysis – change those data that are suspect, rerun the model, and see
whether the insights obtained from it are preserved.

In the case of a linear program, the Sensitivity Report provided by the

simplex method helps us with a sensitivity analysis. The ATV example illus-
trates this point. Each Allowable Increase and Allowable Decrease in Table 5.4
is fairly large, which suggests that the optimal basis remains unchanged over
fairly broad ranges of objective coefficients and RHS values.

The Perturbation Theorem

The Sensitivity Report describes how a change in a single datum affects

the optimal solution to a linear program. What happens if we change several
elements of data? A partial answer to this question lies in:

Proposition 5.1. (the Perturbation Theorem).╇ Suppose that Premium

Solver is used to find an optimal solution to a linear program via the simplex
method and that its sensitivity report states that each Allowable Increase and
each Allowable Decrease is positive. If the data of this linear program are per-
turbed by small amounts, then:
 Chapter 5: Eric V. Denardo 167

• No change occurs in the optimal basis.

• No change occurs in the set of binding constraints.

• The values taken by the basic variables may change.

Illustration

The ATV problem is used to illustrate the Perturbation Theorem.

Table 5.4 shows that each Allowable Increase and Decrease is positive. Thus,
the Perturbation Theorem shows that the binding constraints (identified by
the shaded cells in Table 5.3) stay binding if any or all of the data in Table 5.1
are perturbed. An optimal solution to the ATV problem can be described in
either of these ways:

• Make Standard model vehicles at the rate of 20 per week, make Fancy
model vehicles at the rate of 30 per week and make no Luxury model
vehicles.

• Keep the Engine and Body shops busy making Standard and Fancy
model vehicles, and make no Luxury model vehicles.

If the model’s data were exact (and that never occurs), both descriptions
of the optimal solution would be correct. If the model’s data are close, the lat-
ter is correct because it keeps the binding constraints binding.

Sketch of a proof

An air-tight proof of the Perturbation Theorem would entail an interpre-

tation of the shadow prices that does not appear until Chapter 11, but a sketch
of a proof can be provided here. The hypothesis of this theorem guarantees
that:

• Each basis for the LP’s equation system contains one variable per equa-
tion. (In other words, the Full Rank proviso is satisfied.)

• The linear program has only one optimal solution, and this optimal so-
lution is nondegenerate. (It sets each basic variable that is constrained
to be nonnegative to a positive value.)

Coupling these observations with the fact that the inverse of a matrix is a
continuous function of its data would prove the theorem.
168 Linear Programming and Generalizations

7.  Relative Opportunity Cost

Linear programming is a natural environment to describe and illustrate

three economic concepts, which are:

• The break-even price (or shadow price) of a resource.

• The relative opportunity cost of engaging in an activity.

• The marginal benefit of so doing.

It will soon be seen that three concepts are very closely linked.

Context

These three concepts are described in the context of a linear program that
has been cast in Form 1 – with equality constraints and nonnegative decision
variables. In such a linear program, each decision variable is now said to rep-
resent an activity. Each constraint (other than the nonnegativity constraints
on the variables) measures the consumption of a resource and requires its
consumption to equal its availability (RHS value). Each basis is now said to
engage in those activities (decision variables) that the basis includes. The val-
ue assigned to each decision variable in a basic solution is now said to be the
level of the activity that it represents.

Relative opportunity cost

Shadow prices and marginal benefit are familiar from Chapter 4, but rela-
tive opportunity cost was not discussed there. Like the other two terms, relative
opportunity cost is defined in the context of a particular basis. The relative op-
portunity cost of each activity equals the reduction in contribution that occurs
the levels of the activities in which the basis is engaged are altered so as to free up
(make available) the resources needed to set the level of that activity equal to 1.

The ATV example

Shadow prices, relative opportunity costs and marginal benefit are de-
fined for every basis, not merely for the optimal basis. To illustrate these con-
cepts – and the relationship between them – we focus on the basis (and basic
solution) that is optimal for the ATV problem. Table 5.4 reports its shadow
prices. For convenient reference, these shadow prices are recorded in Ta-
ble 5.5, with a label and unit of measure of each.
 Chapter 5: Eric V. Denardo 169

Table 5.5.↜渀 Label and value of each constraint’s shadow prices for

the optimal solution to the ATV problem.

Constraint label Value

Engine shop capacity E 140 $/hr
Body shop capacity B 420 $/hr
Standard Finishing shop capacity SF ╇╇╇╇╛0 $/hr
Fancy Finishing shop capacity FF ╇╇╇╇╛0 $/hr
Luxury Finishing shop capacity LF ╇╇╇╇╛0 $/hr

Let us recall that these shadow prices are break-even prices. For instance,
a change of δ in the RHS of the Engine shop capacity constraint causes a
change of 140 δ in the basic solution’s objective value. These break-even pric-
es apply to simultaneous changes in several RHS values.

The Luxury model vehicle

Table  5.5 presents the shadow prices for the optimal basis. With refer-
ence to that basis, the relative opportunity cost of making one Luxury model
vehicle is now computed. Making one Luxury model vehicle requires 1 hour
in the Engine shop, 3 hours in the Body shop, and 2 hours in the Luxury Fin-
ishing shop. The shadow prices apply to simultaneous changes in several RHS
values, and the prices in Table 5.5 show that:

relative opportunity cost of

 
(1) = (1) × (140) + (3) × (420) + (2) × (0),
one Luxury model vehicle
 = $ 140 + $ 1,260 + $ 0 = $ 1,400.

Evidently, contribution is reduced by $1,400 when the levels of the activi-

ties in which the basis is engaged are adjusted so as to free up the recourses
needed to make one Luxury model vehicle.

The marginal profit of any activity equals its contribution less the relative
opportunity cost of freeing up the resources needed to accomplish that activ-
ity. In particular,

marginal profit of one its its relative

     
(2) = − ,
Luxury model vehicle contribution opportunity cost
 = $ 1,200 − $ 1,400 = −$ 200.
170 Linear Programming and Generalizations

Equation (2) tells us nothing new because Table 5.4 reports –200 as the
reduced cost of L, namely, as the change in profit if the basic solution is per-
turbed by setting Lâ•›=â•›1 and adjusting the values of the basic variables accord-
ingly.

Why the optimal solution is what it is

Equation (2) tells us nothing new, but equation (1) does. It indicates why
the Luxury model vehicle is unprofitable. Making one Luxury model vehicle
requires 3 hours in the Body shop, which has a break-even price of $420/hour,
and (3) × (420) = $1, 260.
260 This (alone) exceeds the contribution of the Luxu-
ry model vehicle. In this example and in general:

To learn why the optimal solution to a linear program is what it is, use
the shadow prices to parse the relative opportunity cost of each activity
in which it does not engage.

The Luxury model vehicle would become profitable if its relative oppor-
tunity cost could be reduced below $1,200, and equation (1) shows that this
would occur if the time it required in the Body shop could be reduced below
2 11/21 hours.

The Nifty model vehicle

To further illustrate the uses of relative opportunity cost, suppose that

the manager of the ATV plant has been asked to manufacture a new mod-
el, the Nifty. She wonders whether it will turn a profit. Discussions with the
engineering department lead her to conclude that making each Nifty model
would require about 2.5 hours in the Engine shop, 1.5 hours in the Body shop,
and 3 hours in the Fancy Finishing shop. From information provided by the
marketing and operations departments, she estimates that making each Nifty
would contribute approximately $900. To determine whether or not Nifties
are profitable, she calculates their relative opportunity cost and marginal
profit:

relative opportunity cost of

 
= (2.5) × (140) + (1.5) × (420) + (3) × (0)
one Nifty model vehicle
= 350 + 630 = $ 980,
marginal profit of
 
= $ 900 − $ 980 = −$ 80.
one Nifty model vehicle
 Chapter 5: Eric V. Denardo 171

Making one Nifty reduces profit by approximately $80. Nifties are not
profitable. Their relative opportunity cost shows that they would become
slightly profitable. if their manufacturing time in the Body shop could be re-
duced by 0.2 hours.

The Standard model vehicle

Still in the context of the optimal plan for the ATV facility, let’s compute
the relative opportunity cost of making one Standard model vehicle. To do
so, we must free up the resources that it requires, which are 3 hours in the
Engine shop (at a cost of $140 per hour), 1 hour in the Body shop (at a cost
of $420 per hour) and 2 hours in the Standard finishing shop (at a cost of $0
per hour), so that:

relative opportunity cost of

 
(3) = (3) × (140) + (1) × (420) + (2) × (0),
one Standard model vehicle
= $420 + $420 + $0 = $ 840.

Note that $840 is the contribution of one Standard model vehicle. Is it
an accident that the relative opportunity cost of one Standard model vehicle
equals its contribution? No. It makes perfect economic sense because:

• Contribution is maximized by producing at the three vehicles at the

rates Sâ•›=â•›20, Fâ•›=â•›30 and Lâ•›=â•›0.

• If we remove the resources needed to make one Standard model ve-

hicle, it must be optimal to produce at the rates Sâ•›=â•›19, Fâ•›=â•›30 and Lâ•›=â•›0.

• Doing that reduces contribution by exactly $840.

Please pause to verify that the relative opportunity cost of one Fancy
model vehicle equals $1,120. This illustrates a point that holds in general and
is highlighted below:

Consider any basis for a linear program. The contribution of each basic
variable equals its relative opportunity cost.

The above has been justified on economic grounds. It also follows from
the fact that the reduced cost of each basic variable equals zero.
172 Linear Programming and Generalizations

Computing the shadow prices

The observation that is highlighted above shows how to compute the

shadow prices for any basis. To illustrate, consider the basis for the ATV
problem that includes the decision variables S, L, s3 , s4 and s5 . The relative
opportunity cost of each basic variable equals its contribution, so the shadow
prices for this basis must satisfy:

S is basic â•… ⇒  3E + 1B + 2SF = 840

L is basic╅╛╛⇒  1E + 3B + 2LF = â•›1200
s3 is basicâ•…â•… ⇒        ╇ 1SF = 0
s4 is basicâ•…â•… ⇒              ╇ 1FF = â•› 0
s5 is basicâ•…â•… ⇒                      1LF = â•› 0

That’s five linear equations in five unknowns. The lower three equations
set SFâ•›=â•›FFâ•›=â•›LFâ•›=â•›0. This reduces the upper two equations to 3Eâ•›+â•›1Bâ•›=â•›840 and
1Eâ•›+â•›3Bâ•›=â•›1200; their solution is Eâ•›=â•›165 and Bâ•›=â•›345.

By the way, the shadow price that this basis assigns to a particular con-
straint could have been computed by adding δ to its RHS value and seeing
what happens to the basic solution and finding the change in its objective
value. For the ATV example, this would have required solution of six lin-
ear equations (not 5). And it would have given us only one of the shadow
prices.

No shadow price?

The ATV problem satisfies the Full Rank proviso because each constraint
in its Form-1 representation has a slack variable. This guarantees that the
equation system has a solution and that it continues to have a solution if a
RHS value is perturbed, hence that each basis assigns a shadow price to each
constraint.

What about a linear program whose Form-1 representation violates the

Full Rank proviso? Every basic feasible solution to this linear program has at
least one trite row. The linear program becomes infeasible if at least one RHS
value is perturbed. Not every row can have a shadow price. What then?
 Chapter 5: Eric V. Denardo 173

Multipliers

Solver is actually reporting values of the multipliers. If a constraint has a

shadow price, its multiplier is that shadow price. Constraints that lack shadow
prices have infinitely many multipliers, but every set of multipliers can be
shown to have the property that is highlighted below:

The relative opportunity cost of each decision variable is determined by

using the multpliers as though they were shadow prices.

Demonstration that this is so is deferred to Chapter 11. An interpretation

is provided here.

An illustration

To illustrate the role of multipliers in a linear program that lacks shadow

prices, we turn our attention to

Program 2.╇ Maximize {4x1 + 2x2 + 5x3 }, subject to

y1 : 1x1 + 2x2 + 3x
1x3 1=+6,2x2 + 3x3 = 6,
y2 : 2x1 + 4x2 + 6x3 = 12,
2x + 4x + 6x3 = 12,
y3 : 3x1 + 2x2 + 1x3 1= 6, 2
x1 ≥ 0, x2 ≥ 0, 3xx13 +≥2x0. 2 + 1x3 = 6,
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0.

“Multipliers” y1 , y2 and y3 have been assigned to the constraints of

Program 2.

The 2nd constraint in Program 2 is twice the 1st constraint. If the RHS
value of either of these constraints was perturbed, the linear program would
become infeasible. Neither constraint can have a shadow price. Solver and
Premium Solver report “shadow prices” anyhow.

Multipliers, by hand

What these software packages are actually reporting are values of the “multi-
pliers.” As noted above, the values of the multipliers y1 , y2 and y3 are such that
the relative opportunity cost of each basic variable equals its contribution. For
the basis that includes x1 and x3 and excludes x2 , the multipliers must satisfy:
x1 is basic ⇒ 1y1 + 2y2 + 3y3 = 4,
x3 is basic ⇒ 3y1 + 6y2 + 1y3 = 5.
174 Linear Programming and Generalizations

That’s 2 equations in 3 unknowns. It cannot have a unique solution. One

of its solutions is y1 = 11/8, y2 = 0, and y3 = 7/8. It can be shown (and
is shown in Chapter 11) that that this solution and all others correctly account
reduced cost of the nonbasic variable x2 :

reduced cost of x2 = 2 − [2y1 + 4y2 + 2y3 ] = 2 − 36/8 = −2.5.

The reduced cost of x2 is negative, so an optimal solution to Program 2

is at hand.

Multipliers, with Premium Solver

Premium Solver has been used to solve Program 2. The sensitivity report
in Table 5.6 records its optimal solution and multipliers, which are

x1 = 3/2, x2 = 0, x3 = 3/2, y1 = 11/8, y2 = 0, y3 = 7/8.

This is the same basis and the same set of multipliers that are computed
above.

Table 5.6.↜  A sensitivity report for Program 2.

Premium Solver reports 0 as the Allowable Increase and Decrease of the

RHS values of the 1st and 2nd constraints. This is correct. These constraints are
linearly dependent, and the LP becomes infeasible if the RHS value of either
is perturbed.
 Chapter 5: Eric V. Denardo 175

Multipliers, with Solver

When Solver is applied to Program 2, it reports the same optimal solution

as in Table 5.6. The version of Solver with which Excel 2010 is equipped fails
to report correct values of the Allowable Increase and Allowable Decrease of
the RHS values of linearly dependent constraints.

Sneak preview

As mentioned earlier, the current discussion of multipliers anticipates

material in Chapter 11. That chapter includes a version of the simplex meth-
od that uses multipliers to determine which columns might profitably enter
the basis. That version is known by two names, which are the “revised simplex
method” and the “simplex method with multipliers.”

8.  Opportunity Cost

This section is focused on opportunity cost, as that term is used in eco-

nomics. Its definition appears below:

The opportunity cost of doing something is the benefit one can obtain
from the best alternative use of the resources needed to do that thing.

Friedrich von Wieser

The Austrian economist, Friedrich von Wieser (1851-1926), is credited

with coining the term, “opportunity cost.” To illustrate his use of it, consider a
businessperson who has purchased a quantity of iron and who contemplates
using it for a particular purpose. That person should be concerned with the
direct profit (contribution) obtained from that particular use less its opportu-
nity cost, the latter being the profit from the best alternative use of the same
quantity of iron. Barring ties, only one use of this quantity of iron will have a
direct profit that exceeds its opportunity cost, and that use will be best.

Paul Samuelson

In Paul Samuelson’s classic text on microeconomics, the concept of op-

portunity cost is illustrated using Robinson Crusoe. He is thinking of devot-
176 Linear Programming and Generalizations

ing the afternoon to picking raspberries. His best alternative use of the time
and effort he would spend picking raspberries is in picking strawberries. Cru-
soe anticipates that spending the afternoon picking strawberries or raspber-
ries will be equally pleasurable. The opportunity cost of picking raspberries
is the value that Crusoe places on the strawberries he might have picked that
afternoon.

In both illustrations, the motivating idea is to ascertain the marginal ben-

efit of a decision – this being the direct benefit (contribution) less its opportu-
nity cost. The definition works well in settings for which the “best alternative
use” exists and is easily identified.

A difficulty

Consider a slightly more complicated situation, namely, the ATV prob-

lem. What is the opportunity cost of the resources needed to make one Luxu-
ry model vehicle? There is no alternative use of this bundle of resources. They
cannot have a best alternative use. Their opportunity cost is not defined.

A puzzle

At the 2005 meeting of the American Economics Association, Paul J. Fer-

raro and Laura J. Taylor2 posed the question that is paraphrased below:
“You have been given a ticket to see an Eric Clapton concert tonight. This
ticket has no resale value. Bob Dylan is performing tonight, and is your
next-best alternative activity. Tickets to see Dylan cost $40. If the concerts
were on separate evenings, you would be willing to pay up to $50 to see
Dylan whether or not you see Clapton tonight. There are no other costs to
seeing either performer. The opportunity cost of seeing Clapton is one of
the following:

a) $0, b) $10, c) $40, d) $50.

Which is it?”

Fewer than 22% of the roughly 200 professional economists who respond-
ed to this question got the correct answer. That is pretty dismal performance.

Paul J. Ferraro and Laura O. Taylor, “Do Economists Recognize an Opportunity

2╇

Cost When They See One? A Dismal Performance from the Dismal Science,” Contri-
butions to Economic Analysis & Policy: Vol. 4, Issue 1, Article 7, 2005.
 Chapter 5: Eric V. Denardo 177

If every one of them had chosen amongst the four choices at random (a pure
guess), a statistic this low would occur with a probability that is below 0.2.

Extra work

To place opportunity cost and relative opportunity cost on a “level play-

ing field,” consider an example in which the bundles of resources needed to
engage in each activity has at least one alternative use. We wish to determine
the program (set of activities) that is most profitable (optimal). How can we
check whether or not a particular program is optimal? This question is an-
swered twice.

• With relative opportunity costs:

–╇Compute the shadow prices for this program of activities.

–╇Then, for each activity that is excluded from the program, use these
shadow prices to compute the relative opportunity cost of the activity.

–╇The current program is optimal if no activity’s contribution exceeds

its relative opportunity cost.

• With opportunity costs:

–╇For each activity, find the best alternative use of the bundle of re-
sources it requires. (This requires solution of one optimization prob-
lem per activity.)

–╇The current program is optimal if the contribution of each activity

in which it is engaged is at least as large as that activity’s opportunity
cost and if the contribution of each activity in which it is not engaged
is no greater than that activity’s opportunity cost.

Using opportunity costs to determine whether or not a particular pro-

gram is optimal can require the solution of one optimization problem per
activity.

Recap

The relative opportunity cost focuses the decision maker on the marginal
benefit of an action – this being the benefit (contribution) obtained from the
action less the cost of freeing up the resources needed to make it possible. The
178 Linear Programming and Generalizations

classic definition of opportunity cost is also motivated by marginal analysis,

but it is not always defined, it can be difficult to grasp, and it can require ex-
tensive side-computation.

The connection between shadow prices, relative opportunity costs and

marginal benefit has been presented in the context of a resource allocation
problem, but it holds in general. It is central to constrained optimization and
to competitive equilibria. It has not yet been fully assimilated into mainstream
economics, however.

The economic insight of George B. Dantzig

Virtually all of the ideas that have appeared so far in this book are due
to George B. Dantzig. Not all of the terminology is due to him. In his classic
text3, Dantzig used the term relative cost instead of “reduced cost.” Relative
cost reflects the fact that it is relative to the current basis. In place of “shadow
price,” he used the terms price and multiplier, the latter as an abbreviation
of Lagrange multiplier. Dantzig understood that the shadow prices exist if
and only if the Full Rank proviso is satisfied. He fully understood the role of
multipliers in marginal analysis and in the “revised” simplex method. Con-
sider this:

• Prior to Dantzig’s simplex method, no systematic method existed to

determine an optimal allocation of resources.

• Dantzig’s simplex method has remained the principal tool for finding
an optimal allocation of resources ever since he devised it.

• Dantzig fully understood the relation of the simplex method to eco-

nomic reasoning, including the fact that each basis has multipliers
(prices) that, even when non-unique, correctly account for the mar-
ginal benefit of every feasible perturbation of the its basic solution.

A perplexing decision

In 1975, the Nobel Prize in Economics was awarded to Leonid V. Kan-

torovich and Tjalling C. Koopmans for their contributions to the “optimal
allocation of resources.” That is perplexing. George B. Dantzig had done the
above work well before 1975. At the time the prize was awarded, he was with-

George B. Dantzig, Linear programming and extensions, R-366-PR, The RAND Cor-
3╇

poration, August, 1963 and Princeton University Press, Princeton, NJ, 1963.
 Chapter 5: Eric V. Denardo 179

out peer as concerns the optimal allocation of resources. Since that time, no
one has approached his stature in this field.

9.  A Glimpse of Duality*

This is the first of three starred sections that are starred because they
are independent of each other. Here, the ATV problem is used to glimpse a
topic that is known as duality. Let’s begin by recalling the labels of the shadow
prices – E for Engine shop, B for Body shop, SF for Standard Finishing shop,
and so forth. Table 5.5 reports the values that the optimal solution assigns to
the shadow prices, namely,

E = 140, B = 420, SF = 0, FF = 0, LF = 0.

These shadow prices are break-even prices, and their unit of measure is
$/hour.

Bidding for the resources

The labels E, B, SF, FF and LF will be used to describe the decision vari-
ables in a second linear program. Think of yourself as an outsider who wishes
to rent the ATV facility for one week. Imagine that you face the situation in:

Problem B (renting the ATV facility).╇ The ATV company has agreed to rent
its ATV facility to you for one week according to the following terms:

• You must offer them a price for each unit of capacity of each shop.

• You must set each price high enough that they have no economic mo-
tive to withhold any capacity of any shop from you.

What prices should you set, and how much must you spend to rent the
facility for one week?

The ATV company can earn $50,400 by operating this facility for one
week. You must set your prices high enough that they have no motive to with-
hold any capacity from you. Intuitively, it seems clear that you will need to pay
at least $50,400 to rent their entire capacity. But must you spend more than
$50,400? And what prices should you offer? To answer these questions, we
will build a linear program.
180 Linear Programming and Generalizations

Decision variables

The decision variables in this linear program are the prices that you will
offer. By agreement, you must offer five prices, one per shop. These prices are
labeled:

E╛ ╇ = the price ($/hour) you offer for each unit of Engine shop capacity.

B╛╇ = the price ($/hour) you offer for each unit of Body shop capacity.

SF = the price ($/hour) you offer for each unit of Standard Finishing shop
capacity.

FF â•›= the price ($/hour) you offer for each unit of Fancy Finishing shop
capacity.

LF â•›= the price ($/hour) you offer for each unit of Luxury Finishing shop
capacity.

Renting the capacity

Let us compute the cost you will incur for renting the entire capacity of
the ATV facility. The Engine shop has a capacity of 120 hours, and you must
pay E dollars for each hour you rent. The cost to you of renting the entire
capacity of the Engine shop is 120E. The cost of renting the entire capacity of
the Body shop capacity is 80B. And so forth. The total cost that you will pay
to rent every unit of every shop’s capacity is given by

{120E + 80B + 96SF + 102FF + 40LF}.

You wish to minimize this expression, which is your rental bill, subject to
constraints that keep the ATV company from withholding any capacity from
you.

Leaving resources idle

The ATV company need not make full use of the capacity of any of its
shops. That fact constrains the prices that you can offer. For instance, the
capacity constraint on the Engine shop is the inequality,

3S + 2F + 1L ≤ 120.
 Chapter 5: Eric V. Denardo 181

Can you offer a price E that is negative? No. If you did, the ATV company
would not rent you any of the capacity of its Engine shop. Instead, it would
leave those resources idle. You must offer a price E that is nonnegative. The
decision variable E must satisfy the constraint E  ≥  0.

Each shop’s capacity constraint is a “≤” inequality. For this reason, each
of the prices that you offer must be nonnegative. In other words, the decision
variables must satisfy the constraints,

E ≥ 0, B ≥ 0, SF ≥ 0, FF ≥ 0, LF ≥ 0.

Producing vehicles

The ATV facility can be used to manufacture vehicles. Your prices must
be high enough that manufacturing each type of vehicle becomes unprofit-
able. The price of the bundle of resources needed to produce each vehicle
must be at least as large as its contribution.

Let us begin with the Standard model vehicle. Column B of Table  5.2
shows that the company would earn $840 for each Standard model vehicle
that it made, that making this vehicle would require 3 hours in the Engine
shop, 1 hour in the Body shop, and 2 hours in the Standard Finishing shop.
Thus, it becomes unprofitable to make any Standard model vehicles if the
prices you offer satisfy

S: S: 3E + 1B + 2SF ≥ 840.

Similarly, the data in column C of Table 5.2 shows that it becomes unprof-

itable to make any Fancy model vehicles if you offer prices that satisfy

F: F: 2E + 2B + 3FF ≥ 1120.

In the same way, the data in column D of Table  5.2 shows the Luxury
model vehicle becomes unprofitable if the prices you offer satisfy

L: L: 1E + 3B + 2LF ≥ 1200.

A price-setting linear program

The constraints and objective of a linear program that rents the ATV
manufacturing facility have now been presented. Assembling them produces
182 Linear Programming and Generalizations

Program 3.╇ Minimize {120E╛+╛80B╛+╛96SF╛+╛102FF╛+╛40LF}, subject to

S: 3E + â•›1B + â•›2SF â•›≥ 840,

F: 2E +â•› 2B + 3FF â•› ≥ 1120,
L: 1E +â•› 3B + 2LF ≥ 1200,
â•… E ≥ 0, B ≥ 0, SF ≥ 0, FF ≥ 0, LF ≥ 0.

Program 3 calculates the prices that minimize the cost of renting the fa-
cility for one week, subject to constraints that make it unprofitable for the
ATV company to withhold any capacity from you, the renter. From Solver, we
could learn that the optimal solution to this linear program is

E = 140, B = 420, SF = 0, FF = 0, LF = 0,

that its optimal value is 50,400 $/wk, and the shadow prices of its three con-
straints are

S = 20, F = 30, L = 0.

Program 1 and Program 3 have the same optimal value, and the shadow
prices of each form an optimal solution to the other!

Duality

The properties exhibited by Programs 1 and 3 are no coincidence. They

illustrate a general “duality” principle that is highlighted below:

Duality: Each linear program is paired with another. If either linear

program in a pair is feasible and bounded, then:

•â•‡The other linear program is feasible and bounded, and both linear
programs have the same optimal value.

•â•‡The shadow prices (multipliers) for either linear program form an

optimal solution to the other.

Is duality a curiosity? No! From a practical viewpoint, a number of com-

petitive situations can be formulated not as a single linear program, but as
a linear program and its dual. Also, in economic reasoning, there is often
 Chapter 5: Eric V. Denardo 183

a duality (pairing) of production quantities and break-even prices. Finally,

from a theoretical viewpoint, duality is a widely-used tool in the analysis of
optimization problems.

A surprise?

Did duality surprise you? If so, you are in very good company. It sur-
prised George B. Dantzig too. In retrospect, it’s eminently reasonable. To see
why, we note that the optimal solution to Program 1 and its shadow prices
have these properties:

• Each constraint in Program 1 is a “≤” inequality and increasing its RHS

value can only cause the optimal value to improve, so each shadow
price is nonnegative.

• These shadow prices are such that no vehicle’s contribution exceeds its
relative opportunity cost, so they satisfy the “≥” constraints of Program 3.

Thus, these shadow prices form a feasible solution to Program 3. It re-

mains to argue that multiplying each constraint’s shadow price by its RHS
value and summing them up totals 50,400. That’s not difficult; see Problem 7.

10.  Large Changes and Shadow Prices*

Each shadow price is accompanied by an “allowable increase” and an

“allowable decrease” that determines the range of RHS values over which it
applies. This section probes the question: What happens if a RHS value is
perturbed by an amount that falls outside the range for which the shadow
price applies?

From Table 5.4, we see that the shadow price on Engine shop capacity is
140 $/hr, with an Allowable Increase of 56 and an Allowable Decrease of 16.
This price applies for capacity levels in the range between 104 hours (because
104â•›=â•›120 – 16) and 176 hours (because 176â•›=â•›120â•›+â•›56).

We can use Solver to find optimal solutions to Program 1 for Engine shop
capacities that are just below 104 and just above 176, find the range and shad-
ow price in each case, and repeat. Figure 5.3 plots the result. This figure shows
that the slope decreases with quantity. This is no accident. A linear program
184 Linear Programming and Generalizations

exhibits decreasing marginal return on each capacity (right-hand side value).

On reflection, the reason is clear. When there is only a tiny amount of a par-
ticular capacity, that capacity is as profitable as it can be. As this capacity in-
creases, other resources become more fully utilized, slack constraints become
tight, and one can make less and less profitable use of the added capacity.

Figure 5.3.↜  Contribution versus Engine shop capacity.

contribution
$ 60,000

$ 50,000
slope = 0
$ 40,000
slope = 140
$ 30,000
slope = 165
slope = 240
$ 20,000
slope = 560
$ 10,000
slope = 1200 Engine
$0 shop
0 20 40 76 104 176 capacity

Figure  5.3 exhibits decreasing marginal return because the marginal

benefit (slope) can only decrease as the quantity increases. This means that, in
an important sense, the current shadow prices are the most favorable. Start-
ing with a capacity of 120, small decreases in capacity cost $140 per unit.
Larger decreases cost $165 per unit. Still larger increases cost even more. And
so forth. Similarly, starting with a capacity of 120, small increases earn $140
per unit. Larger increases earn less. It is emphasized:

The current shadow prices are the most favorable; larger increases will
be less profitable, and larger decreases will be more costly.

11.  Linear Programs and Solid Geometry*

The example in Chapter 4 had only two decision variables, so its geom-
etry could be visualized on the plane. The ATV problem has three decision
variables, which are S, F and L, so a geometric view of it requires three-di-
 Chapter 5: Eric V. Denardo 185

mensional (or solid) geometry. Solid geometry has been familiar since birth,
even if it is omitted from typical high-school geometry courses.

Cartesian coordinates

In Chapter 4, Cartesian coordinates were used to identify each ordered

pair (x, y) of real numbers with a point in the plane. The point (0, 0) on the
page was called the origin, and the point (1, 2) was located 1 unit to the right
of the origin and 2 units toward the top of the page.

In a similar way, Cartesian coordinates identify each ordered triplet

(A, B, C) of real numbers with a point in three-dimensional space. The point
(0, 0, 0) on the page is called the origin, and the point (1, 2, 3) is located 1 unit
toward the right of the origin, 2 units toward the top of the page, and 3 units
above the page, for instance.

In this way, Cartesian coordinates identify every feasible solution (S, F, L)

to Program 1 with a point in three-dimensional space. The feasible region (set
of feasible solutions) becomes the polyhedron in Figure 5.4. For instance, the
triplet (40, 0, 0) lies 40 units to the right of the origin, and the triplet (0, 34, 0)
lies 34 units toward the top of the page from the origin. And the triplet (0, 0, 20)
lies 20 units in front of the origin.

Figure 5.4.↜  Feasible region for Program 1, and its extreme points.

F
(0, 34, 0) (12, 34, 0)
(0, 34, 4) (20, 30, 0)

(0, 0, 0) S

(0, 10, 20) (40, 0, 0)

(0, 0, 20) (20, 0, 20) (35, 0, 15)

L
186 Linear Programming and Generalizations

Vertices, edges and faces

The feasible region for the ATV problem is polyhedron that has 10 verti-
ces (extreme points), 15 edges, and 7 faces. One constraint is binding on each
face, two on each edge, and three at each vertex. Two vertices are adjacent if
an edge connects them.

When the simplex method is applied to the ATV example, each pivot
shifts it from a vertex to an adjacent vertex whose objective value is improved.
It stops pivoting after reaching vertex (20, 30, 0).

Watching Solver pivot

It is possible to watch Solver pivot. To do so, open the Excel spreadsheet

for Chapter 5, and click on the sheet entitled Table 5.3. Then click on Solver
or on Premium Solver. Either way, click on its Options box, and then click on
the Show Iteration Results window. When you run Solver or Premium Solver,
you will see that the first pivot occurs to the triplet (S, F, L) given by (0, 0, 20),
the second to (0, 10, 20), the third to (20, 0, 20), the fourth to (34, 0, 15) and
the fifth to (20, 30, 0). Each of these triplets corresponds to an extreme pivot
of the feasible region in Figure 5.4, each pivot occurs to an adjacent extreme
point, and each pivot improves the basic solution’s objective value.

Higher dimensions

Plane geometry deals with only two variables, solid geometry with three.
Linear programs can easily have dozens of decision variables, or hundreds.
Luckily, results that hold for plane geometry and for solid geometry tend to
remain valid when there are many variables. That is why geometry is relevant
to linear programs.

12.  Review

This chapter has shown how to formulate a linear program for solution
by Solver and by Premium Solver for Education. A focal point of this chapter
has been the interpretation of the information accompanies the optimal solu-
tion. We have seen:

• How the shadow prices determine the relative opportunity costs of the
basic and nonbasic variables.
 Chapter 5: Eric V. Denardo 187

• How the relative opportunity costs help us to understand why the opti-
mal solution is what it is.

• How the Allowable Increases and Allowable Decreases help determine

whether the optimal solution is robust.

• How the optimal solution responds to small changes in the model’s

data; the values of the basic variables may change, but the binding con-
straints stay binding.

Relative opportunity cost has been used to determine whether or not a

plan of action can be improved. It has been argued that it is more difficult –
and that it may be impossible – to use the classic definition of opportunity
cost to determine whether or not a plan of action can be improved.

Material in later chapters has been glimpsed. In Chapter 11, shadow pric-
es, relative opportunity cost, and marginal profit will be used in to guide the
“revised” simplex method as it pivots. Chapter 12 is focused on duality and
its uses. In Chapter 14, duality will be used to construct a simple (stylized)
model production and consumption in an economy in general equilibrium.
Nonlinear programs will be studied in Chapter 20, where it is seen that the
natural generalization of decreasing marginal return produces nonlinear pro-
grams that are relatively easy to solve.

13.  Homework and Discussion Problems

1. For the ATV problem, Table 5.4 (on page 161) reports an Allowable De-
crease on the objective coefficient of L of ∞. That is no accident. Why?

2. For the ATV problem, Table 5.4 (on page 161) reports a shadow price of
140 for the Engine shop constraint and an Allowable Decrease of 16 in
this constraint’s right-hand-side value. Thus, renting 4 hours of Engine
shop for one week decreases contribution by $560 because 560â•›=â•›4 × 140.
Without re-running the linear program, show how the optimal solution
changes when the Engine shop capacity is decreased by 4. Hint: The Per-
turbation Theorem reduces this problem to solving two equations in two
unknowns.
188 Linear Programming and Generalizations

3. Eliminate from the ATV problem the Luxury model vehicles. The linear
program that results has only two decision variables, so its feasible region
is a portion of the plane.

(a) Write down this linear program.

(b) Display its feasible region graphically.

(c) Display its iso-profit lines and objective vector graphically.

(d) Show, graphically, that its optimal solution sets Sâ•›=â•›20 and Fâ•›=â•›30, so
that its optimal value equals 50,400.

(e) Set aside the capacity needed to make one Luxury model vehicle. Re-
solve the linear program graphically. Show that making one Luxury
model vehicle decreases contribution by 200.

4. Suppose, in the ATV problem, that the contribution of the Standard mod-
el vehicle is $900 apiece for the first 12 made per week and $640 apiece for
production above that level.

(a) Revise Program 1 to account for this diseconomy of scale, and solve it.

(b) You can figure out what the optimal solution would be without doing
Part (a). How? Support your answer.

5. Consider a company that can use overtime labor at an hourly wage rate
that is 50% in excess of regular time labor cost. Does this represent an
economy of scale? A diseconomy of scale? Will a profit-maximizing linear
program have an unintended option? If so, what will it be? Will this unin-
tended option be selected by optimization, or will it be ruled out?

6. Consider a linear program that is feasible and bounded. Let us imagine

that each right-hand-side value in this linear program was multiplied by
0.75. Complete the following sentences, and justify them. (↜Hint: To edu-
cate your guess, re-solve Program 1 with each right-hand-side value mul-
tiplied by 0.75.)

(a) The optimal solution would be multiplied by _______, and the opti-
mal value would be multiplied by _______.
 Chapter 5: Eric V. Denardo 189

(b) On the other hand, the shadow prices would be _______.

(c) There was nothing special about the factor 0.75 because ________.

7. The shadow prices are supposed to apply to small changes in right-hand

side values. Compute the amount that the manager of the ATV shop could
earn by renting the entire capacity of her shops at the shadow prices. Is
this amount familiar? If so, why? Hint: It might help to review the preced-
ing problem.

8. The sensitivity report seems to omit the shadow prices of the nonnegativ-
ity constraints. True or false:

(a) In a maximization problem, the reduced cost of each nonnegative

variable x equals the shadow price of the constraint x ≥ 0.

(b) In a minimization problem, the reduced cost of each nonnegative

variable x equals the shadow price of the constraint x ≥ 0.

9. In a linear program, a decision variable x is said to be free if neither the

constraint x  ≥  0 nor the constraint x ≤ 0 is present in the linear program.
A free variable is allowed to take values that are positive, negative or zero.
In the optimal solution to a maximization problem, what can you say
about the reduced cost of each free variable? Why?

10. This problem refers to the survey by Paul J. Ferraro and Laura J. Taylor
that is cited in Section 8 of this chapter.

(a) With the economists’ definition of opportunity cost, which of the four
answers is correct?

(b) Suppose that exactly 200 professional economists answered their

question and that 44 of them got the correct answer (that’s 22%).
Of the 44 answered correctly, 10 taught micro and knew it. The rest
guessed. What is the probability of as few as 34 correct answers from
the remaining 190 economists if each of them answered at random?

(c) Suppose you had planned to attend the Dylan concert when someone
offers you a free ticket to the Clapton concert. What can you say of
the relative opportunity cost of seeing the Clapton concert?
190 Linear Programming and Generalizations

11. This problem refers to the Robinson Crusoe example that is discussed in
Section 8. Suppose Crusoe has a third alternative. In addition to spending
the afternoon picking strawberries or strawberries, he could spend it loll-
ing on the beach.

(a) Suppose he would rather loll on the beach than pick strawberries.
Carefully describe:

(i) The opportunity cost of picking raspberries.

(ii) The opportunity cost of lolling on the beach.

(b) Suppose he planned to pick raspberries when the sun came out, at
which time it occurred to him that he might enjoy an afternoon on
the beach. Describe the relative opportunity cost of an afternoon on
the beach.

12. Write down linear programs that have each of these properties:

(a) It has no feasible solution.

(b) It is feasible and unbounded.

(c) Its feasible region is bounded, and it has multiple optima.

(d) It is bounded, it has an unbounded feasible region, it has multiple

optimal solutions, but only one of them occurs at an extreme point.

(e) Its feasible region is unbounded, and it has multiple optima, none of
which occur at an extreme point.

13. Perturbing the optimal solution to Program 1 by making one Luxury

model vehicle decreases profit by $200. Complete the following sen-
tence and justify it: Perturbing this optimal solution by making 10
Luxury model vehicles decreases profit by at least _______ because
________________________.

14. In Table 5.6, the Allowable Increase and Allowable Decrease for rows 5
and 6 are zero because Program 2 becomes infeasible if the RHS value of
either of the first two constraints is perturbed. These constraints do not
have shadow prices. Table 5.6 reports that row 7 does have a shadow price,
and it reports an Allowable Increase and Allowable Decrease for its RHS
 Chapter 5: Eric V. Denardo 191

value. Why does this row have a shadow price? What accounts for the
range on this shadow price?

15. Consider a constraint in a linear program that has no shadow price. Does
this constraint have a multiplier? What can be said about the Allowable
Increase and the Allowable Decrease of that constraint’s RHS value?

16. With the Engine shop capacity fixed at 120 hours per week, use Solver to
compute the optimal value of Program 1 for all values of the Body shop
capacity. Plot the analog of Figure 5.3. Do you observe decreasing mar-
ginal return?

17. With Figure 5.4 in view, have Solver or Premium Solver use the simplex
method to solve Program 1, but use the Options tab to have it show the
results of each iteration. Record the sequence of basic solutions that it fol-
lowed. Did it pivot from extreme point to extreme point? Did each pivot
occur along an edge?

18. (A farmer) A 1,200 acre farm includes a well that has a capacity of 2,000
acre-feet of water per year. (One acre-foot is one acre covered to a depth
of one foot.) This farm can be used to raise wheat, alfalfa, and beef. Wheat
can be sold at $550 a ton and beef at $1,300 a ton. Alfalfa can be bought or
sold at the market price of $220 per ton. Each ton of wheat that the farmer
produces requires one acre of land, $50 of labor, and 1.5 acre-feet of water.
Each ton of alfalfa that she produces requires 1/3 acre of land, $40 of labor
and 0.6 acre-feet of water. Each ton of beef she produces requires 0.8 acres
of land, $50 of labor, 2 acre-feet of water, and 2.5 tons of alfalfa. She can
neither buy nor sell water. She wishes to operate her farm in a way that
maximizes its annual profit. Below are the data in a spreadsheet formula-
tion, the solution that Solver has found, and a Sensitivity Report.
192 Linear Programming and Generalizations

(a) Write down the linear program. Define each variable. Give each vari-
able’s unit of measure. Explain the objective function and each con-
straint. Explain why the constraint AS ≥ 0 is absent. What is the unit
of measure of the objective? What is the unit of measure of each con-
straint?

(b) State the optimal solution in a way that can executed when the data
are inexact.

(c) As an objective function coefficient or right-hand side value varies

within its allowable range, how does she manage the farm? That is, in
which activities does she engage, and which resources does she use to
capacity?

(d) What would have to happen to the price of wheat in order for her to
change her production mix?

(e) What would have to happen to the price of alfalfa for her to change
her production mix?

Note: Parts (f) through (i) refer to the original problem and are inde-
pendent of each other.

(f) The government has offered to let her deposit some acreage in the
“land bank.” She would be paid to produce nothing on those acres. Is
she interested? Why?

(g) The farmer is considering soybeans as a new crop. The market price
for soybeans is $800 per ton. Each ton of soybeans requires 2 acres of
land, 1.8 acre feet of water and $60 of labor. Without re-running the
 Chapter 5: Eric V. Denardo 193

linear program, determine whether or not soybeans are a profitable

crop.

(h) A neighbor has a 400 acre farm with a well whose capacity is 500
acre-feet per year. The neighbor wants to retire to the city and to rent
his entire farm for $120,000 per year. Should she rent it? If so, what
should she do with it?

(i) The variable AS is unconstrained in sign. Rewrite the linear program

with AS replaced by (ASOLD – ABOUGHT), where ASOLD and
ABOUGHT are nonnegative decision variables. Solve the revised
linear program. Did any changes occur? Does one formulation give
more accurate results than the other? If so, how and why?

19. (pollution control) A company makes two products in a single plant. It

runs this plant for 100 hours each week. Each unit of product A that that
the company produces consumes 2 hours of plant capacity, earns the
company a contribution of $1,000 and causes, as an undesirable side ef-
fect, the emission of 4 ounces of particulates. Each unit of product B that
the company produces consumes 1 hour of capacity, earns the company
a contribution of $2,000 and causes, as undesirable side effects, the emis-
sion of 3 ounces of particulates and of 1 ounce of chemicals. The EPA
(Environmental Protection Agency) requires the company to limit par-
ticulate emission to at most 240 ounces per week and chemical emission
to at most 60 ounces per week.

(a) Formulate this problem for solution by linear programming. In this

linear program, what is the unit of measure of each decision variable?
Of the objective function? Of each shadow price?

(b) Solve this linear program on a spreadsheet. Describe its optimal solu-
tion in a way that can be implemented when its data are inexact.

(c) What is the value to the company of the EPA’s relaxing the constraint
on particulate emission by one ounce per week? What is the value to
the company of the EPA’s relaxing the constraint on Chemical emis-
sions by one ounce per week?

(d) (an emissions trade-off) By how much should the company be willing
to reduce its weekly emission of chemicals if the EPA would allow it
to emit one additional ounce of particulates each week?
194 Linear Programming and Generalizations

(e) (an emissions tax) The EPA is considering the control of emissions
through taxation. Suppose that the government imposes weekly tax
rates of P dollars per ounce of particulate emissions and C dollars per
ounce of chemical emission. Find tax rates, P and C, that keep the
company’s pollutants at or below the current levels and minimize the
company’s tax bill. Hint: With the constraints on emissions deleted,
the feasible region becomes a triangle, so the tax rates must be suffi-
ciently large to make the extreme point(s) that cause excess pollution
to become undesirable.

(f) By how much does the taxation scheme in part (e) reduce profit?
 Chapter 6: The Simplex Method, Part 2

1.╅ Preview����������������������������������尓������������������������������������尓���������������������� 195

2.╅ Phase I����������������������������������尓������������������������������������尓������������������������ 196
3.╅ Cycling����������������������������������尓������������������������������������尓������������������������ 203
4.╅ Free Variables����������������������������������尓������������������������������������尓������������ 207
5.╅ Speed����������������������������������尓������������������������������������尓�������������������������� 210
6.╅ Review����������������������������������尓������������������������������������尓������������������������ 215
7.╅ Homework and Discussion Problems����������������������������������尓���������� 215

1.  Preview

This chapter completes this book’s introductory account of the simplex

method. In this chapter, you will see:

• How Phase I of the simplex method determines whether a linear pro-

gram has a feasible solution and, if so, how it constructs a basic feasible
tableau with which to initiate Phase II.

• That the simplex method can “cycle” (fail to terminate finitely) and that
it can be kept from doing so.

• That the simplex method readily accommodates variables that are not
required to be nonnegative.

Also discussed here is the speed of the simplex method. Decades after
its discovery and in spite of the best efforts of scores of brilliant researchers,
the simplex method remains the method of choice for solving large linear
programs.

E. V. Denardo, Linear Programming and Generalizations, International Series 195

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_6, © Springer Science+Business Media, LLC 2011
196 Linear Programming and Generalizations

2.  Phase I

Phase II of the simplex method is initialized with a linear program for

which a basic feasible tableau has been found. Thus, two tasks remain for
Phase I:

• Determine whether or not a linear program has a feasible solution.

• If it has a feasible solution, find a basic feasible tableau with which to

initiate Phase II.

These tasks can be accomplished in several different ways. And several

different versions of Phase I appear in the literature. No matter how it is or-
ganized, Phase I is a bit complicated. The version of Phase I that is presented
here appends one artificial variable α to the original linear program. The coef-
ficients of α are selected so that a single pivot creates a basic feasible tableau,
except that the basis will include α and the basic solution will equate α to a
positive value. The simplex method will then be used to drive α toward 0. If α
can be reduced to 0, it is removed from the basis, and a basic feasible tableau
for the original linear program results. If α cannot be reduced to zero, the
linear program has no feasible solution.

This method is described below as a six-step procedure. Each step is il-

lustrated in the context of

Problem A.╇ Maximize {4p╛+╛ 1q╛+╛ 2r}, subject to the constraints

(1.1) â•›– 1p + â•›1q + 2r ≥ 6,

(1.2) ╛╇╛╛1p – 3.5q – 3r = –10,

(1.3) â•›– 2p – 3q ≤ 0,

(1.4) ╛╛ p ≥ 0, q ≥ 0, r ≥ 0.

Step 1 of Phase I

The 1st step of Phase I is to cast the linear program in Form 1, preserving
its sense of optimization. Executing this step on Problem A rewrites it as
 Chapter 6: Eric V. Denardo 197

Program 1.╇ Maximize {z}, subject to the constraints

(2.0) â•›4p + â•›1q + 2r – z = â•› 0,

(2.1) – 1p + â•›1q + 2r – s1 = â•›6,

(2.2) â•›1p – 3.5q – 3r = –10,

(2.3) – 2p – 3q + s3 â•›= â•›0,

(2.4) p ≥ 0, q ≥ 0, r ≥ 0, s1 ≥ 0, s3 ≥ 0.

In Program 1, equation (2.0) defines z as the value of the objective. The

surplus variable s1 converts inequality (1.1) to an equation, and the slack vari-
able s3 converts inequality (1.3) to an equation.

Step 2

The 2nd step of Phase I is to ignore the nonnegativity constraints on the

decision variables and apply Gauss-Jordan elimination to the equations that
remain, keeping –z basic for the top-most equation. This step either con-
structs a basic system or an inconsistent equation. If it finds an inconsistent
equation, no solution exists to the equation system, so no feasible solution
can exist to the linear program, which has additional constraints. Presented
in Table 6.1 is the result of executing Step 2 on Program 1.

Rows 2-6 of Table 6.1 mirror system (2). Rows 4 and 5 lack basic vari-
ables. A choice exists as to the elements on which to pivot. Table 6.1 exhibits
the result of pivoting on the coefficient of s1 in row 4 and then on the coef-
ficient of p in row 11. These pivots produce the basic tableau in rows 14-18.
This tableau’s basic solution sets pâ•›=â•›–10 and s3â•›=â•›–20. If this solution were
feasible, Phase I would be complete. It is not feasible, so Phase I continues.

Step 3

The 3rd step is to insert on the left-hand side of the equation system an
artificial variable α with a coefficient of –1 in each equation whose RHS
value has the wrong sign and with a coefficient of 0 in each of the remaining
equations. Displayed in rows 20-24 of Table 6.2 is the result of executing Step
3 on Program 1.
198 Linear Programming and Generalizations

Table 6.1.↜  Pivoting to create a basic system for Program 1.

Table 6.2.↜  Creating a basic feasible tableau, except that α > 0.

 Chapter 6: Eric V. Denardo 199

To see what Step 3 accomplishes, we write the equations represented by

rows 20-24 of Table 6.2 in dictionary format, with the nonbasic variables on
the right.

(3.0) z â•›= –40 + 15q + 14r,

(3.1) s1 = 4 – â•›2.5q – 1r,

(3.2) p â•›= –10 + 3.5q + 3r + 1α,

(3.3) s3 = –20 + 11q + 6r + 1α.

In system (3), setting qâ•›=â•›0, râ•›=â•›0 and αâ•›≥â•›20 equates the variables s1, p and
s3 to nonnegative values. Moreover, a pivot on the coefficient of α in the equa-
tion for which s3 is basic will remove s3 from the basis and will produce a basic
solution in which αâ•›=â•›20. This motivates the next step.

Step 4

Step 4 is to select equation whose RHS value is most negative and pivot
upon the coefficient of α in that equation. When applied to the tableau in
rows 20-24 of Table 6.2, this pivot occurs on the coefficient of α in row 24.
This pivot produces the basic tableau in rows 26-30. That tableau’s basic solu-
tion sets s1â•›=â•›4, pâ•›=â•›10 and αâ•›=â•›20, exactly as predicted from system (3)

Step 4 has produced a Phase I simplex tableau, namely, a basic tableau

in which the artificial variable α is basic and whose basic solution equates all
basic variables (with the possible exception of –z) to nonnegative values.

Step 5

What remains is to drive α down toward zero, while keeping the basic
variables (other than –z) nonnegative. This will be accomplished by a slight
adaptation of the simplex method. To see how to pivot, we write the equations
represented by rows 26-30 in dictionary format, as:

(4.0) z = – 40 + 15q + 14r,

(4.1) s1 = 4 – 2.5q – 1r,

(4.2) pâ•›= 10 – 7.5q – 3r + 1s3,

(4.3) α â•›= 20 – 10q – 6r + 1s3.
200 Linear Programming and Generalizations

The goal is to decrease the value of α, which is the basic variable for equa-
tion (4.3). The nonbasic variables q and r have negative coefficients in equa-
tion (4.3), so setting either of them positive decreases α.

In a Phase I simplex tableau, the entering variable can be any nonba-

sic variable that has a positive coefficient in the equation for which α is ba-
sic. (The positive coefficients became negative when they were switched to
the right-hand side.) The usual ratios keep the basic solution feasible. As in
Phase II, no ratio is computed for the row for which –z is basic, and no ra-
tio is computed for any row whose coefficient of the entering variable is not
positive. The pivot occurs on a row whose ratio is smallest. But if the row for
which α is basic has the smallest ratio, pivot on that row because it removes α
from the basis. In brief:

In a Phase I simplex pivot for Form 1, the entering variable and pivot
element are found as follows:

•â•‡The entering variable can be any nonbasic variable that has a posi-
tive coefficient in the row for which α is basic.

•â•‡The pivot row is selected by the usual ratios, which keep the basic
solution feasible and keep –z basic.

•â•‡But if the row for which α is basic ties for the smallest ratio, pivot
on that row.

To reduce the ambiguity in this pivot rule, let’s select as the entering vari-
able a nonbasic variable that has the most positive coefficient in the row for
which α is basic. Table 6.3 displays the Phase I simplex pivots that result.

Rows 26-30 of Table 6.3 indicate that for the first of these pivots, q is the
entering variable (its coefficient in row 30 is most positive), and row 29 has
the smallest ratio, so q enters the basis and p departs. Rows 35-38 result from
that pivot.

Rows 34-38 of Table 6.3 indicate that for the second pivot, r is the enter-
ing variable (its coefficient in row 38 is most positive), and α is the departing
variable because the row for which α is basic ties for the smallest ratio.

Rows 40-44 display the basic tableau that results from that pivot. The
variable α has become nonbasic. The numbers in cells H42-H44 are non-
 Chapter 6: Eric V. Denardo 201

Table 6.3.↜  Illustration of Step 5.

negative, so this tableau’s basic solution equates the basic variables q, r and
s1 to nonnegative values. Deleting α and its column of coefficients deleted
produces a basic feasible tableau for Program 1.

Step 6 of Phase I

The 6th and final step is to delete α and its column of coefficients. This
step produces a basic feasible tableau with which to begin Phase II. Applying
this step to rows 40-44 of Table 6.3 casts Program 1 as the linear program:

Program 1.╇ Maximize {z}, subject to the constraints

(5.0) 5.556p – 0.444s3 – z = – 6.667,

(5.1) – 0.556p + 1s1 + 0.444s3 = 0.667,

(5.2) 0.667p + 1q – 0.333s3 â•›= 0.000,

(5.3) – 1.111p + 1r + 0.389s3 = 3.333,

(5.4) p ≥ 0, q ≥ 0, r ≥ 0, s1 ≥ 0, s3 ≥ 0.
202 Linear Programming and Generalizations

Phase II of the simplex method commences by selecting p as the entering

variable and executing a (degenerate) pivot on the coefficient of p in equation
(5.2).

No entering variable?

One possibility has not yet been accounted for. Phase I pivots can result
in a basic tableau whose basic solution sets αâ•›>â•›0 but in which no nonbasic
variable has a positive coefficient in the row for which α is basic. If this oc-
curs, no entering variable for a Phase I simplex pivot can be selected. What
then?

To illustrate this situation, imagine that we encounter rows 34-38 of

Table  6.3, except that the coefficients of r and s3 in row 38 are –1.616 and
–0.333. Row 38 now represents the equation

(6) α = 4.615 + 1.538p + 1.616r + 0.333s3.

The basic solution to this equation system has αâ•›=â•›4.615, and the variables
p, r and s3 are constrained to be nonnegative, so equation (6) demonstrates
that no feasible solution can have α < 4.615. The artificial variable α cannot be
reduced below 4.615, so the linear program is infeasible.

Recap – infeasible LP

To recap Phase I, we first consider the case of a linear program that is

infeasible. When an infeasible linear program is placed in Form 1 and Phase I
is executed, one of these two things must occur:

• Gauss-Jordan elimination produces an inconsistent equation.

• Gauss-Jordan elimination produces a basis whose basic solution is in-

feasible. An artificial variable α is inserted, and the simplex method
determines that the value of α cannot be reduced to zero.

Recap – feasible LP

Now consider a linear program that is feasible. Neither of the above con-
ditions can occur. If Gauss-Jordan elimination (Step  2) produces a feasible
basis, Phase II is initiated immediately. If not, an artificial variable α is in-
serted, and a pivot produces a basic solution that is feasible, except that it
 Chapter 6: Eric V. Denardo 203

creates α to a positive value. The simplex method reduces the value of α to 0

and eliminates α from the basis, thereby exhibiting a feasible basis with which
to initiate Phase II.

Commentary

Let’s suppose that certain variables are likely to be part of an optimal ba-
sis. Phase I can be organized for a fast start by pivoting into the initial basis
as many as possible of these variables.

A disconcerting feature of Phase I is that the objective value z is ignored

while feasibility is sought. Included in Chapter 13 is a one-phase scheme (that
is known by the awkward name “the parametric self-dual method”) that uses
one artificial variable, α, and seeks feasibility and optimality simultaneously.

3.  Cycling

Does the simplex method terminate after finitely many pivots? The an-
swer is a qualified “yes.” If no care is taken in the choice of the entering vari-
able and the pivot row, the simplex method can keep on pivoting forever.
If care is taken, the simplex method is guaranteed to be finite. This section
describes the difficulty that can arise and shows how to avoid it.

The difficulty

This difficulty is identified in this subsection. A linear program has fi-

nitely many decision variables. It can only have finitely many bases because
each basis is a subset of its decision variables, and there are only finitely many
such subsets. Let us recall from Chapter 4 that:

• Each nondegenerate simplex pivot changes the basis, changes the basic
solution, and improves its objective value.

• Each degenerate pivot changes the basis, but does not change any RHS
values, hence causes no change in the basic solution or in the basic so-
lution’s objective value.

As a consequence, each nodegenerate simplex pivot results in a basis

whose basic solution improves on all basic solutions seen previously. That
is good. No nondegenerate pivot can result in a basis that had been visited
204 Linear Programming and Generalizations

previously. Also, since there are finitely many bases, so only finitely many
nondegenerate simplex pivots can occur prior to termination.

On the other hand, a sequence of degenerate pivots (none of which

changes the basic solution) can cycle by leading to a basis that had been vis-
ited previously. That is not good! If the simplex method cycles once and if it
employs a consistent rule for selecting the entering variable and the pivot row,
it will cycle again and again.

Ambiguity in the pivot element

Whether or not the simplex method cycles depends on how the ambigui-
ty in its pivot rule is resolved. The entering variable can be any variable whose
reduced cost is positive in a maximization problem, negative in a minimiza-
tion problem. The pivot row can be any row whose ratio is smallest.

Rule A

To specify a particular pivot rule, we must resolve these ambiguities.

For a linear program that is written in Form 1, each decision variable is
assigned a column of coefficients, and these columns are listed from left to
right. Let us dub as Rule A the version of the simplex method that chooses
as follows:

• In a maximization (minimization) problem, the entering variable is a

nonbasic variable whose reduced cost is most positive (negative). Ties,
if any, are broken by picking the variable that is listed farthest to the left.

• The pivot row has the lowest ratio. Ties, if any, are broken by picking
the row whose basic variable is listed farthest to the left.

The tableau in Table 6.4 will be used to illustrate Rule A. In that tableau,

lower-numbered decision variables are listed to the left. For the first pivot,
x1 is the entering variable because it has the most positive reduced cost, and
rows 4 and 5 tie for the smallest ratio. The basic variable for row 4 is x5, and
the basic variable for row 5 is x6. Row 4 is the pivot row because its basic vari-
able x5 is listed to the left of x6. No ties occur for the second pivot. A tie does
occur for the third pivot, which occurs on row 16 because x1 is listed to the
left of x2. Evidently, the first three pivots are degenerate. They change the basis
but do not change the basic solution.
 Chapter 6: Eric V. Denardo 205

Table 6.4.↜  Illustration of Rule A.

A cycle

Rule A can cycle. In fact, when Rule A is applied to the linear program
in Table 6.4, it does cycle. After six degenerate pivots, the tableau in rows 3-6
reappears.

An anti-cycling rule

Abraham Charnes was the first to publish a rule that precludes cycling.
The key to his paper, published in 19521, was to pivot as though the RHS
values were perturbed in a way that breaks ties. Starting with a basic feasible
tableau (either in Phase I or in Phase II), imagine that the RHS value of the
1st non-trite constraint is increased by a very small positive number ε, that
the RHS value of the 2nd non-trite constraint is increased by ε 2 , and so forth.
Standard results in linear algebra make it possible to demonstrate that, for
all sufficiently small positive values of ε, there can be no tie for the smallest
ratio. Consequently, each basic feasible solution to the perturbed problem
equates each basic variable (with the possible exception of –z) to a positive
value. In the perturbed problem, each simplex pivot is nondegenerate. This
guarantees that the simplex method cannot cycle. Termination must occur
after finitely many pivots.

Charnes, A [1952]., “Optimality and Degeneracy in Linear Programming,” Econo-

1╇

metrica, V. 20, No. 2, pp 160-170.

206 Linear Programming and Generalizations

The perturbation argument that Charnes pioneered has had a great many
uses in optimization theory. From a computational viewpoint, perturbation is
unwieldy, however. Integrating it into a well-designed computer code for the
simplex method requires extra computation that slows down the algorithm.

A simple cycle-breaker

In 1977, Robert Bland published a simple and efficient anti-cycling rule2.

Let’s call it Rule B; it resolves the ambiguity in the simplex pivot in this way:

• The entering variable is a nonbasic variable whose reduced cost is posi-

tive in a maximization problem (negative in a minimization problem).
Ties are broken by choosing the variable that is listed farthest to the left.

• The pivot row has the smallest ratio. Ties are broken by picking the row
whose basic variable is listed farthest to the left.

When Rule B is applied to a maximization problem, the entering vari-

able has a positive reduced cost, but it needn’t have the largest reduced cost.
Among the variables whose reduced costs are positive, the entering variable
is listed farthest to the left.

Rule B is often called Bland’s rule, in his honor. Proving that Bland’s rule
precludes cycles is a bit involved. By contrast, incorporating it in an efficient
computer code is easy, and it adds only slightly to the computational burden.
Bland’s rule can be invoked after encountering a large number of consecutive
degenerate pivots.

The early days

Initially, it was not clear whether the simplex method could cycle if no
special care was taken to break ties for the entering variable and the pivot row.
George Dantzig asked Alan Hoffman to figure this out. In 1951, Hoffman
found an example in which Rule A cycles. The data in Hoffman’s example
entail the elementary trigonometric functions (sin ϕ, cos2 ϕφ and so forth). In
Hoffman’s memoirs3, he reports:

2╇
Robert G. Bland, “New finite pivot rules for the simplex method, Mathematics of
Operations Research, V. 2, pp. 103-107, 1977.
3╇
Page 171 of Selected Papers of Alan Hoffman with Commentary, edited by Charles
Micchelli, World Scientific, Rver Edge, NJ.
 Chapter 6: Eric V. Denardo 207

“On Mondays, Wednesdays and Fridays I thought it could (cycle). On Tues-

days, Thursdays and Saturdays I thought it couldn’t. Finally, I found an ex-
ample which showed it could … I was never able to … explain what was in
my mind when I conceived the example.”

The example in Table 6.4 is simpler than Hoffman’s; it was published by

E. M. L. Beale in 1955.

Charnes was the first to publish an anti-cycling rule, but he may not have
been the first to devise one. In his 1963 text, George Dantzig4 wrote,
“Long before Hoffman discovered his example, simple devices were pro-
posed to avoid degeneracy. The main problem was to devise a way to avoid
degeneracy with as little extra work as possible. The first proposal along
these lines was presented by the author in the fall of 1950 …. Later, A. Or-
den, P. Wolfe and the author published (in 1954) a proof of this method
based on the concept of lexicographic ordering ….”

Perturbation and lexicographic ordering are two sides of the same coin;
they lead to the same computational procedure, and it is a bit unwieldy.

Following Charnes’s publication in 1952 of his perturbation method, a

heated controversy developed as to whether he or Dantzig was primarily re-
sponsible for the development of solution methods for linear programs. Re-
searchers found themselves drawn to one side or the other of that question. A
quarter century elapsed before Robert Bland published his anti-cycling rule.
It is Bland’s rule that achieves the goal articulated by Dantzig – avoid cycles
with little extra work.

4.  Free Variables

In a linear program, a decision variable is said to be free if it is not con-

strained in sign. A free variable can take any value – positive, negative or zero.
Free variables do occur in applications. To place a linear program that has one
or more free variables in Form 1, we must replace each free variable by the
difference of two nonnegative variables. That is no longer necessary. Modern

Page 231 of Linear Programming and Extensions by George B. Dantzig, published by

4╇

Princeton University Press, 1963.

208 Linear Programming and Generalizations

codes of the simplex method accommodate free variables. How they do so is

the subject of this section.

Form 2

Form 2 generalizes Form 1 by allowing any subset of the decision vari-

ables to be free, that is, unconstrained in sign. In the presence of free variables,
the simplex method must pivot a bit differently. To see how, we consider:

Problem B.╇ Max (–0.5a – 1.25b – 5.00c + 3d + 10e + 25f), subject to

╅╅╅╇ 0.8a – 1.30b â•›– 1d = 12.0,

╅╇ 1b – 1c â•›– 1e = 0.6,
╅╇ 1c – 1f = 1.2,
╅╇╇╛1b ≤ 2.5,
â•…â•… 1c ≤ 9.6,
╅╇ 0.5a + 0.8b + â•›4c ≤ 45,
╅╇ 0.9a + 1.5b ≤ 27,
a ≥ 0, b ≥ 0, c ≥ 0.

In Problem B, the decision variables d, e and f are free; they can take any
values.

Free variables do arise in applications. One of their uses is to model the

quantity of a commodity that can be bought or sold at the same (market)
price. In Problem B, the decision variables d, e and f can represent the net
sales quantities of commodities whose market prices are $3, $10 and $25 per
unit, respectively.

Getting started

Problem B is placed in Form 2 by inserting slack variables in the bottom

four constraints and introducing an equation that defines z as the objective
value. The tableau in rows 2-10 of Table 6.5 results from this step. In this tab-
leau, rows 4, 5 and 6 lack basic variables.

The tableau in rows 12-20 of Table 6.5 results from pivoting on the coef-
ficients of d, e and f in rows 4, 5 and 6, respectively. This tableau is basic. Its
basic solution is feasible because d, e and f are allowed to take any values.
 Chapter 6: Eric V. Denardo 209

Table 6.5.↜  Phase I for Problem B.

Keeping free variables basic

Once a free variable becomes basic, the RHS value of the equation for
which it is basic can have any value, positive, negative or zero. To keep a
free variable basic, compute no ratio for the row for which it is basic. To
keep d, e and f basic for the equations represented by rows 14, 15 and 16 of
Table 6.5, we’ve placed “none” in cells N14, N15 and N16. In this example
and in general:

After a free variable becomes basic, compute no ratio for the equation
for which it is basic. This keeps the free variable basic, allowing it to
have any sign in the basic solution that results from each simplex pivot.

Rows 13-20 of Table 6.5 are a basic feasible tableau with which to initiate
Phase II, and its first pivot occurs on the coefficient of c in row 18. Pivoting con-
tinues until the optimality condition or the unboundedness condition occurs.

Nonbasic free variables

Problem B fails to illustrate one situation that can arise: In a basic feasible
tableau for a linear program that has been written in Form 2, a free vari-
210 Linear Programming and Generalizations

able can be nonbasic. Let us suppose we encounter a basic feasible tableau in

which the free variable xj is not basic and in which the reduced cost of xj is not
zero. What then?

• If the reduced cost of xj is positive, select xj as the entering variable, and

pivot as before.

• If the reduced cost of xj is negative, aim to bring xj into the basis at a

negative level by computing ratios for rows whose coefficients of xj are
negative and selecting the row whose ratio is closest to zero (least nega-
tive).

• In either case, compute no ratio for any row whose basic variable is free.

• If no row has a ratio, the linear program is unbounded.

Needless work

Accommodating free variables is easy. To conclude this section, let’s see

why it is a good idea to do so. To cast Problem B in Form 1, we would need
to introduce one new column per free variable. The coefficients in these col-
umns would be opposite to the coefficients of d, e and f. Columns that start
opposite stay opposite. Even so, updating opposite columns requires extra
work per pivot. Furthermore, if a pivot reduces a previously-free variable to
zero, the next pivot is quite likely to introduce its opposite column. That’s an
extra pivot. Finally, forcing a linear program into Form 1 can cause the ranges
of the shadow prices to become artificially low, which makes the optimal ba-
sis seem less robust than it is.

5.  Speed

In a Form 1 representation of a linear program, let m denote the number

of equations (other than the one defining z as the objective value), and let n
denote the number of decision variables (other than z).

Typical behavior

The best codes of the simplex method quickly solve practical linear pro-
grams having m and n in the thousands or tens of thousands. No one re-
ally understands why the simplex method is as fast as it is. On carefully-con-
 Chapter 6: Eric V. Denardo 211

structed examples (one of which appears as Problem 5), the simplex method
is exceedingly slow. Any attempt to argue that the simplex method is fast “on
average” must randomize in a way that bad examples occur with miniscule
probability.

In Chapter 12 of his text, Robert J. Vanderbei5 provided a heuristic ratio-

nale as to why the parametric self-dual method (that is described in Chap-
ter 13) should require approximately (mâ•›+â•›n)/2 pivots, and he reported the
number of pivots required to solve each member of a standard family of test
problems that is known as the NETLIB suite6. He made a least-squares fit of
the number of pivots to the function α(m + n)β , and he found that the best
fit is to the function

(7) 0.488(m + n)1.0515 ,

moreover, that the quality of the fit is quite good. Expression (7) is strikingly
close to (mâ•›+â•›n)/2.

Atypical behavior

The simplex method does not solve all problems quickly. In their 1972 pa-
per, Klee and Minty7 showed how to construct examples having m equations
and 2 m decision variables for which Rule A requires 2mâ•›–â•›1 pivots. (Problem
5 presents their example for the case mâ•›=â•›3.) Even at the (blazing) speed of one
million pivots per second, it would take roughly as long as the universe has
existed for Rule A to solve a Klee-Minty example with mâ•›=â•›100.

A conundrum

The gap between typical performance of roughly (mâ•›+â•›n)/2 pivots and

atypical performance of 2mâ•›−â•›1 pivots has been a thorn in the side of every per-
son who wishes to measure the efficiency of a computational procedure by its
worst-case performance. Over the decades, several brilliant works have been
written on this issue. The interested reader is referred to a paper by Daniel

5╇
Vanderbei, Robert J., Linear Programming: Foundations and Extensions, Kluwer
Academic Publishers, Boston, Mass., 1997.
6╇
Gay, D. “Electronic mail distribution of linear programming test problems,” Math-
ematical Programming Society COAL Newsletter, V. 13, pp 10-12, 1985.
7╇
V. Klee and G. J. Minty, “How good is the simplex algorithm?” In O. Shisha, editor,
Inequalities III, pp. 159-175, Academic Press, New York, NY, 1972.
212 Linear Programming and Generalizations

Spielman and Shang-Huia Tang that has won both the Gödel and the Fulker-
son prize8.

The ellipsoid method

In 1979, Leonid G. Kachian9 created a sensation with the publication of his

paper on the ellipsoid method. It is a divide-and-conquer scheme for finding
the solutions to the inequalities that characterize optimal solutions to a linear
program and its dual. An upper bound on the number of computer opera-
tions required by the ellipsoid method (this counts the square root as a single
operation) is a fixed constant times n4 L, where L is the number of bits needed
to record all of the nonzero numbers in A, b and c, along with their locations.

From a theoretical viewpoint, Kachian’s work was a revelation. It showed

that linear programs can be solved with a method whose worst-case work
bound is a polynomial in the size of the problem. From a computational
viewpoint, the ellipsoid method was disappointing, however. It is not used
because it solves practical linear programs far more slowly than does the sim-
plex method.

Interior-point methods

In 1984, Narendra Karmarkar created an even greater sensation with the

publication of his paper on interior-point methods.10 These methods move
through the interior of the feasible region, avoiding the extreme points entirely.
One of the methods in his paper has the same worst-case work bound as the
ellipsoid method, and Karmarkar claimed running times that were many
times faster than the simplex method on representative linear programs.

A controversy erupted. Karmarkar’s running times proved to be difficult

to duplicate, and they seemed to be for an “affine scaling” method that was
not polynomial.

8╇
Spielman, D and S.-H. Teng, “Smoothed analysis of algorithms: Why the simplex
method usually takes polynomial time,” Journal of the ACM, V. 51, pp. 385-463
(2004).
9╇
L. G. Kachian, “A polynomial algorithm in linear programming,” Soviet Mathemat-
ics Doklady, V. 20, pp 191-194 (1979).
10╇
N. Karmarkar, “A new polynomial-time algorithm for linear programming,” Pro-
ceedings of the 16th annual symposium on Theory of Computing, ACM New York,
pp 302-311 (1979).
 Chapter 6: Eric V. Denardo 213

AT&T weighs in

In an earlier era, when AT&T had been a highly-regulated monopoly, it

had licensed its patents free of charge. By 1984, when Karmarkar published
his work, this had changed. AT&T had become eager to earn royalties from
its patents. AT&T sought and obtained several United States patents that were
based on Karmarkar’s work. This was surprising because:

• Patents are routinely awarded for processes, rarely for algorithms.

• Interior-point methods were hardly novel; beautiful work on these

methods had been by done in the 1960’s by Fiacco and McCormick11,
for instance.

• The “affine scaling” method in Karmarkar’s paper had been published

in 1967 by Dikin12.

• Karmarkar’s fastest running times seemed to have been for Dikin’s

method.

• Karmakar’s claims of faster running times than the simplex method

could not be substantiated, and AT&T would not release the test prob-
lems on which these claims were based!

The AT&T patents on Karmarkar’s method have not been challenged

in a United States court, however. The validity of these patents might now
be moot, as the interior-point methods that Karmarkar proposed have since
been eclipsed by other approaches.

A business unit

Aiming to capitalize on its patents for interior-point methods, AT&T

formed a business unit named Advanced Decision Support Systems. The sole
function of this business unit was to produce and sell a product named KO-
RBX (short for nothing) that consisted of a code that implemented interior

11╇
Fiacco, A. V. and G. McCormick, Nonlinear programming: sequential unconstrained
minimization techniques,” John Wiley & Sons, New York, 1968, reprinted as Classics
in applied mathematics volume 4, SIAM, Philadelphia, Pa., 1990.
12╇
Dikin, I. I.. “Iterative solution of problems of linear and quadratic programming,”
Soviet Math. Doklady, V. 8, pp. 674-675, 1967.
214 Linear Programming and Generalizations

point methods on a parallel computer made by Alliant Corporation of Acton,

Massachusetts.

This implementation made it difficult (if not impossible) to ascertain

whether KORBX ran faster than the simplex method. This implementation
also made it difficult for AT&T to keep pace with the rapid improvement in
computer speed.

As a business unit, Advanced Decision Support Systems existed for about

seven years. It was on a par, organizationally, with AT&T’s manufacturing
arm, which had been Western Electric and which would be spun off as Lucent
Technologies. At its peak, in 1990, Advanced Decision Support Systems had
roughly 200 full-time employees. It sold precisely two KORBX systems, one
to the United States Military Airlift Command, the other to Delta Airlines.
As a business venture, Advanced Decision Support Systems was unprofitable
and, in the eyes of many observers, predictably so.

Seminal work

Karmarkar’s 1984 paper sparked an enormous literature, however. Hun-

dreds of brilliant papers were written by scores of talented researchers. Any
attempt to cite a few of these papers overlooks the contributions of others as
well as the many ways in which researchers interacted. That said, the candi-
dates for the fastest interior-point methods may to be the “path-following”
algorithm introduced by J. Renegar13 and the self-dual homogeneous method
of Y. Ye, M. Todd and S. Muzino14. While this research was underway, the
simplex method was vastly improved by incorporation of modern sparse-
matrix techniques.

What’s best?

For extremely large linear programs, the best of the interior-point method
might run a bit faster than the simplex method. The simplex method enjoys
an important advantage, nonetheless. In Chapter 13, we will see how to solve
an integer program by solving a sequence of linear programs. The simplex

13╇
Renegar, J., “A polynomial-time algorithm, based on Newton’s method for linear
programming,” Mathematical Programming, V. 40, pp 59-93, 1988
√
14╇
Ye, Yinyu, Michael J. Todd and Shinji Mizuno, “On O( n L) iteration homogeneous
and self-dual linear programming algorithm,” Mathematics of Operations Research,
V. 19, pp. 53-67, 1994.
 Chapter 6: Eric V. Denardo 215

method is far better suited to this purpose because it finds an optimal solu-
tion that is an extreme point; interior-point methods find an extreme point
only if the optimum solution is unique.

Currently, the main use of interior-point methods is to solve classes of

nonlinear programs for which the simplex method is ill-suited. For comput-
ing optimal solutions to linear programs, large or small, Dantzig’s simplex
method remains the method of choice.

6.  Review

The key to the version of Phase I that is presented here is to introduce a

single artificial variable and then attempt to pivot it out of the basis. The same
device will be used in Chapter 15 to compute solutions to the “bi-matrix game.”

The simplex method can cycle, and cycles can be avoided. Bland’s method
for avoiding cycles is especially easy to implement. Even so, the perturbation
method of Charnes (equivalently, the lexicographic method of Dantzig) has
proved to be a useful analytic tool in a number of settings.

Decision variables that are not constrained in sign are easy to accommo-
date within the simplex method. Once a free variable is made basic, it is kept
basic by computing no ratio for the equation for which it is basic.

No one fully understands why the simplex method is as fast as it is on

practical problems. Any effort to prove that the simplex method is fast on av-
erage (in expectation) must assign miniscule probabilities to “bad examples.”

Modern interior-point methods may run a bit faster than the simplex
method on enormous problems, but the simplex method remains the method
of choice, especially when integer-valued solutions are sought.

7.  Homework and Discussion Problems

1. (Phase I) In Step 2 of Phase I, would any harm be done by giving the arti-
ficial variable α a coefficient of –1 in every equation other than the one for
which –z is basic?
216 Linear Programming and Generalizations

2. (Phase I) For the tableau in rows 35-39 of Table 6.3, rows 37 and 38 tie for
the smallest ratio. Execute a pivot on the coefficient of r in row 37. Does
this result in a basis that includes α and whose basic solution sets αâ•›=â•›0? If
so, indicate how to remove α from the basis and construct a basic feasible
tableau with which to initiate Phase II.

3. (Phase I) In Phase 2, an entering variable can fail to have a pivot row, in

which case the linear program is unbounded. This cannot occur in Phase I.
Why?

4. (Phases I and II) Consider this linear program: Maximize {2xâ•›+â•›6y}, subject
to the constraints

╇╛╛╛2x – 5y ≤ –3,

╇╛╛4x – 2y + 2z ≤ –2,

╅╇ 1x + 2y â•›≤ 4,
x ≥ 0, y ≥ 0, z ≥ 0.

(a) On a spreadsheet, execute Phase I of the simplex method.

(b) If Phase I constructs a feasible solution to the linear program, execute

Phase II on the same spreadsheet.

5. The spreadsheet that appears below is a Klee-Minty example in which the

number m of constraints equals 3 and the number n of decision variables
(other than –z) equals 2 m. The goal is maximize z.

(a) For this example, execute the simplex method with Rule A. (You will
need seven pivots.)

(b) For each extreme point encountered in part (a), record the triplet (x1,
x2, x3,).
 Chapter 6: Eric V. Denardo 217

(c) Plot the triplets you recorded in part (b). Identify the region of which
they are the extreme points. Does it resemble a deformation of the unit
cube? Could you have gotten from the initial extreme point to the final
extreme point with a one simplex pivot?

(d) What do you suppose the comparable example is for the case mâ•›=â•›2?
Have you solved it?

(e) Write down but do not solve the comparable example for mâ•›=â•›4.

6. Apply the simplex method with Rule A to the maximization problem in

Table 6.4, but stop when a cycle occurs.

7. Apply the simplex method with Rule B to the maximization problem in

Table 6.4. Did it cycle? Identify the first pivot at which Rule B selects a dif-
ferent pivot element than does Rule A.

8. In Rule B, ties are broken by picking the variable that is farthest to the left.
Would it work equally well to pick the variable that is farthest to the right?

9. The idea that motivates Charnes’s perturbation scheme is to resolve the am-
biguity in the variable that will leave the basis by perturbing the RHS values
by miniscule amounts, but in a nonlinear way. The tableau that appears be-
low reproduces rows 2-6 of Table 6.4, with the dashed line representing the
“=” signs and with the quantity ε j added to the jth constraint, for jâ•›=â•›1, 2, 3.

(a) Execute Charnes’s pivot rule (for maximization) on this tableau, se-
lecting the nonbasic variable whose reduced cost is most positive as
the entering variable.

(b) Identify the first pivot at which Charnes’s rule selects a different pivot
element than does Rule A.

(c) Complete and justify the sentence: If a tie were to occur for the smallest
ratio when Charnes’s pivot rule is used, two rows would need to have
coefficients of ε 1 , ε 2 , and ε 3 that are _________, and that cannot oc-
cur because elementary row operations keep ______ rows ______.
218 Linear Programming and Generalizations

(d) There is a sense in which Charnes’s rule is lexicographic. Can you

spot it? If so, what is it?

10. Cycling can occur in Phase I. Cycling in Phase I can be precluded by Rule
B or by Charnes’s perturbation scheme. At what of the six steps of Phase
I would Charnes perturb the RHS values? Which RHS values would he
perturb?

11. Consider a linear program that is written in Form 1 and is feasible and
bounded. By citing (but not re-proving) results in this chapter, demon-
strate that this linear program has a basic feasible solution that is optimal.

12. (free variables) This problem concerns the maximization problem that is
described by rows 12-20 of Table 6.5, in which d, e and f are free.

(a) On a spreadsheet, execute the simplex method with Rule A, but com-
puting no ratios for the rows whose basic variables are free.

(b) Did any of the free variables switch sign? If so, what would have oc-
curred if this problem had been forced into Form 1 prior to using
Rule A? Remark: Part (b) requires no computation.

13. (free variables) The tactic by which free variables are handled in Section 4
of this chapter is to make them basic and keep them basic. Here’s an alter-
native:

(i) After making a free variable basic, set aside this variable, and set aside
the equation for which it just became basic. (This reduces by one the
number of rows and the number of columns.)

(ii) At the end, determine the values taken by the free variables from the
values found for the other variables.

Does this work? If it does work, why does it work? And how would you
determine the “values taken by the free variables.”

14. (extreme points and free variables) A feasible solution to a linear program
is an extreme point of the feasible region if that feasible solution is not
a convex combination of two other feasible solutions. Consider a linear
program that is written in Form 2. Suppose this linear program is feasible
and bounded. Is it possible that no extreme point is an optimal solution?
Hint: can a feasible region have no extreme points?
 Part III–Selected Applications

Part III surveys optimization problems that involve one decision-maker.

Chapter 7. A Survey of Optimization Problems

This chapter is built upon 10 examples. When taken together, these ex-
amples suggest the range of uses of linear programs and their generalizations.
These examples include linear programs, integer programs, and nonlinear
programs. They illustrate the role of optimization in operations management
and in economic analysis. Uncertainty plays a key role in several of them.
Also discussed in this chapter are the ways in which Solver and Premium
Solver can be used to solve problems that are not linear.

Chapter 8. Path-Length Problems and Dynamic Programming

This chapter is focused on the problem of finding the shortest or longest

path from one node to another in a directed network. Several methods for
doing so are presented. Linear programming is one of these methods. Path-
length problems are the ideal setting in which to introduce “dynamic pro-
gramming,” which is a collection of ideas that facilitate the analysis of deci-
sion problems that unfold over time.

Chapter 9. Flows in Networks

Described in this chapter are “network flow” models and the uses to
which they can be put. If the “fixed” flows such a model are integer-valued,
the simplex method is shown to find an integer-valued optimal solution.
Chapter 7: A Survey of Optimization Problems

1.╅ Preview����������������������������������尓������������������������������������尓���������������������� 221

2.╅ Production and Distribution����������������������������������尓������������������������ 222
3.╅ A Glimpse of Network Flow����������������������������������尓�������������������������� 224
4.╅ An Activity Analysis����������������������������������尓������������������������������������尓�� 226
5.╅ Efficient Portfolios ����������������������������������尓������������������������������������尓���� 229
6.╅ Modeling Decreasing Marginal Cost����������������������������������尓������������ 235
7.╅ The Traveling Salesperson����������������������������������尓���������������������������� 240
8.╅ College Admissions*����������������������������������尓������������������������������������尓�� 244
9.╅ Design of an Electric Plant*����������������������������������尓�������������������������� 248
10.╇ A Base Stock Model ����������������������������������尓������������������������������������尓�� 251
11.╇ Economic Order Quantity����������������������������������尓���������������������������� 253
12.╇ EOQ with Uncertain Demand*����������������������������������尓�������������������� 256
13.╇ Review����������������������������������尓������������������������������������尓������������������������ 261
14.╇ Homework and Discussion Problems����������������������������������尓���������� 261

1.  Preview

The variety of optimization problems that can be formulated for solution

by linear programming and its generalizations is staggering. The “survey” in
this chapter is selective. It must be. Each problem that appears here illustrates
one or more of these themes:

• Exhibit the capabilities of the Premium Solver software package.

• Relate optimization to economic reasoning.

E. V. Denardo, Linear Programming and Generalizations, International Series 221

in Operations Research & Management Science 149,
DOI 10.1007/978-1-4419-6491-5_7, © Springer Science+Business Media, LLC 2011
222 Linear Programming and Generalizations

• Relate optimization to operations management.

• Relate optimization to situations in which uncertainty plays a central

role.

Only a few of the optimization problems in this chapter are linear

programs. That’s because of the need to make room for optimization
problems that include integer-valued variables and nonlinearities. Linear
programs are strongly represented in three other chapters – in Chapter 8
(dynamic programming), in Chapter 9 (network flow) and in Chapter 14
(game theory).

Three sections of this chapter are starred. The starred sections delve into
probabilistic modeling. These starred sections present all of the “elementary”
probability that they employ, but readers who are new to that subject may find
those sections to be challenging.

To a considerable extent, each section is independent of the others. They

can be read selectively. An exception occurs in the starred sections. The “nor-
mal loss function” is introduced in the first starred section, and it is used in
all three. Another exception consists of Sections 10-12. They form a coherent
account of basic ideas in operations management and might best be read as
a unit.

2.  Production and Distribution

The initial example is a rudimentary version of a problem that is faced in

the petroleum industry.

Problem 7.A╇ A vertically-integrated petroleum products company produces

crude oil in three major fields, which are labeled U, V and W, and ships it to
four refineries, which are labeled 1 through 4. The top nine rows of Table 7.1
contain the relevant data. Cells H5, H6 and H7 of this table specify the pro-
duction capacities of fields U, V and W, respectively. Cells I5, I6 and I7 con-
tain the production costs for these fields. Cells D9 through G9 contain the
demand for crude oil one week hence at the refineries 1 through 4. These
demands must be met by production during the current week. Each entry in
the array D5:G7 is the cost of shipping from the field in its row to the refinery
 Chapter 7: Eric V. Denardo 223

in its column. Capacities and demands are measured in thousands of barrels

per week. Production and shipping costs are measured in dollars per barrel.
The company wants to minimize the cost of satisfying these demands. How
shall it do this?

Table 7.1.↜  Spreadsheet formulation of Problem 7.A.

A tailored spreadsheet

In earlier chapters, a “standardized” spreadsheet was used to build a lin-

ear program. Each decision variable was represented by a column, and each
constraint was depicted as a row. For Problem 7.A, the decision variables are
the shipping quantities, and it is natural to organize them in the same pattern
as the shipping costs.

The “tailored” spreadsheet in Table 7.1 presents the shipping quantities

in the array D12:G14. The sum across a row of this array is the quantity pro-
duced in the corresponding field, and the sum down a column of this array is
the quantity shipped to the corresponding refinery.
224 Linear Programming and Generalizations

A linear program

The functions in cells E18 and F18 compute the shipping and pro-
duction costs. Solver has been asked to minimize the quantity in cell G18,
which is the total cost. Its changing cells are the shipping quantities in cells
D12:G14. Its constraints are H12:H14 ≤ H5:H7 (production quantities cannot
exceed production capacities), D15:H15 = D9:H9 (demands must be met)
and D12:G14 ≥ 0 (shipping quantities must be nonnegative). Table 7.1 reports
the optimal solution to this linear program.

The petroleum industry

In Chapter 1, it had been observed that a paper on the use of a linear

program to find a blend of aviation fuels had excited great interest in the pe-
troleum industry. Problem 7.A suggests why. Linear and nonlinear programs
offered the promise of integrating the production, refining, distribution, and
marketing of petroleum products in ways that maximize after-tax profit.

A coincidence?

Table 7.1 reports an optimal solution that is integer-valued. This is not

an accident. Problem 7.A happens to be a type of “network flow” problem for
which every basic solution is integer-valued.

3.  A Glimpse of Network Flow

Figure 7.1 depicts the constraints of Problem 7.A as a network flow mod-

el. Each “flow” occurs on a “directed arc” (line segment with an arrow). The
amount flowing into each node (circle) must equal the amount flowing out
of that node. All “flows” are nonnegative. Some flows are into a node, some
flows are out of a node, and some flows are from one node to another. The
flows can have bounds, and they can be fixed.

Figure 7.1 has 7 nodes, one for each field and one for each refinery. The
node for field U accounts for the production in that field and for its shipment
to the four refineries. The node for refinery 1 accounts for the demand at its
refinery and the ways in which this demand can be satisfied. The flow into
node U cannot exceed 250, which is the capacity of field 1, and the flow out of
node 1 must equal 200, which is the demand at refinery 1.
 Chapter 7: Eric V. Denardo 225

Figure 7.1.  A network flow interpretation of Problem 7.A.

= 200
1
≤ 250
U
= 300
2
≤ 400
V
= 250
3
≤ 350
W
= 150
4

The Integrality Theorem

A network flow model is said to have integer-valued data if each of its

bounds and each of its fixed flows is integer-valued. In Figure 7.1, each fixed
flow and each bound is integer-valued. This network flow model does have
integer-valued data. This model’s costs are not integer-valued, but that does
not matter. An important property of network flow models is highlighted
below.

The Integrality Theorem: Consider a network flow model that has

integer-valued data. Each of its basic solutions is integer-valued.

The Integrality Theorem is proved in Chapter 9.

The simplex method for network flow

Let us consider what happens when the simplex method is applied to a

network flow model that has integer-valued data. The simplex method pivots
from one basic solution to another. Each basic solution that it encounters is
integer-valued. The simplex method stops with a basic solution that is opti-
mal, and it too is integer-valued. For this class of optimization problems, the
simplex method is guaranteed to produce an optimal solution that is integer-
valued.

The Integrality Theorem is of little consequence in Problem 7.A. Petro-

leum is no longer shipped in barrels. Even if it were, little harm would be done
by rounding off any fractions to the nearest integer.
226 Linear Programming and Generalizations

In other contexts, such as airline scheduling, it is vital that the decision

variables be integer-valued. If the network flow model of an airline sched-
uling problem has iteger-valued data, the simplex method produces a basic
solution that is optimal and is integer-valued.

4.  An Activity Analysis

An activity analysis is described in terms of goods and technologies. Each

technology transforms one bundle of goods into another. The inputs to a
technology are the goods it consumes, and the outputs of a technology are the
goods it produces. Each technology can be operated at a range of nonnegative
levels. The decision variables in an activity analysis include the level at which
to operate each technology. If a model of an activity analysis has constant
returns to scale, it leads directly to a linear program. To illustrate this type of
model, consider

Problem 7.B. (Olde England).╇ In an early era, developing nations shifted their
economies from agriculture toward manufacturing. Old England had three
principal technologies, which were the production of food, yarn and clothes.
It traded the inputs and outputs of these technologies with other countries.
In particular, it exported the excess (if any) of yarn production over internal
demand.

The Premier asked you to determine the production mix that would
maximize the net value of exports for the coming year. Your first step was
to accumulate the “net output” data that appear in cells B4:D10 of Table 7.2.
Columns B records the net output for food production; evidently, producing
each unit of food requires that Olde England import £0.50 worth of goods
(e.g., fertilizer), consume 0.2 units of food (e.g., fodder to feed to animals),
consume 0.5 units of labor, and use 0.9 units of land. Column C records the
net outputs for yarn production; producing each unit of yarn requires that
Olde England import £1.25 worth of goods, consume 1 unit of labor, and use
1.5 units of land. Column D records the net outputs for clothes production;
producing each unit of clothes requires the nation to import £5.00 worth of
goods, consume 1 unit of yarn, and consume 4 units of labor.

Cells J5:J7 record the levels of internal consumption of food, yarn and
clothes, respectively; in the coming year, Olde England will consume 11.5
 Chapter 7: Eric V. Denardo 227

million units of food, 0.6 million units of yarn and 1.2 million units of clothes.
Cells J9:J12 record the nation’s capacities, which are 65 million units of labor
and 27 million units of land, as well as the capability to produce yarn at the
rate of 10.2 million units per year and clothes at the rate of 11 million units
per year.

Row 4 records the world market prices of £3 per unit for food, £10 per
unit for yarn and £16 per unit for clothes. The amounts that Olde England
imports or exports will have negligible effect on these prices.

Table 7.2.↜  An activity analysis for Olde England.

Decision variables

This activity analysis has two types of decision variables. The symbols FP,
YP and CP stand for the quantity of food, yarn and clothes to produce in the
coming year. The symbols FE, YE and CE stand for the net exports of food,
yarn and clothes during the coming year. The unit of measure of each of these
quantities is millions of units per year. The production quantities FP, YP and
CP must be nonnegative, of course. The net export quantities FE, YE and CE
can have any sign; setting FEâ•›=â•›−1.5 accounts for importing 1.5 million units
of food next year, for instance.

A linear program

The linear program whose data appear in Table  7.2 maximizes the net
value of exports. Column H contains the usual sumproduct functions. Cell
H4 measures the contribution (value of net exports). Rows 5-7 account for
the uses of food, yarn and clothes. Rows 9-10 account for the uses of land and
228 Linear Programming and Generalizations

labor. Rows 11 accounts for the loom capacity, and row 12 accounts for the
clothes-making capacity.

The decision variables in cells B3:D3 are required to be nonnegative, but

the decision variables in cells E3:G3 are not. Solver has been asked to maxi-
mize the value of net exports (the number in cell H4) subject to constraints
H5:H7 = J5:J7 and H9:H12 ≤ J9:J12. Row 3 of Table 7.2 reports the optimal
values of its decision variables.

Evidently, the net trade balance is maximized by making full use of the
land, full use of the capacity to weave yarn, and full use of the capacity to pro-
duce clothes. Clothes are exported. The nation produces most, but not all, of
the food and yarn it requires.

Some “what if ” questions

Activity analyses like this one make it easy to respond to a variety of “what
if ” questions. Here are a few: What would occur if Olde England decided that
it ought to be self-sufficient as concerns food? Would it pay to increase the
capacity to produce yarn? What would occur if the market price of clothes
decreased by 20%?

A bit of the history

The phrase “activity analysis” was first used by Tjalling Koopmans; its ini-
tial appearance is in the title1 of the proceedings of a famous conference that
he organized shortly after George Dantzig developed the simplex method.
Well before that time (indeed, well before any digital computers existed) Was-
sily Leontief (1905-1999) built large input-output models of the American
economy and used them to answer “what if ” questions. Leontief received the
Nobel Prize in 1973 “for the development of the input-output method and for
its application to important economic problems.” As Leontief had observed,
an activity analysis is the natural way in which to describe the production
side of a model of an economy that is in general equilibrium. One such model
appears in Chapter 14.

Activity analysis of production and allocation: Proceedings of a conference, Tjalling C.

1╇

Koopmans, ed., John Wiley & Sons, 1951.

 Chapter 7: Eric V. Denardo 229

5.  Efficient Portfolios

The net return (profit) that will be earned on an investment is uncertain.

Table 7.3 specifies the net return that will be earned by the end of a six-month
period per unit invested in each of three assets. These returns depend on the
state that will occur at that time. Cells C4 through C6 specify the probability
distribution over these states. Cells D4 through D6 specify the net return R1
per unit invested in asset 1 if each state occurs. Cells E4 through E6 and F4
through F6 specify similar information about assets 2 and 3. Evidently, if state
a occurs, these assets have return rates of −20%, 40% and −30%, respectively.
The returns on these assets are dependent; if you know the value taken by the
return on one of the assets, you know the state and, consequently, the returns
on the other assets.

Table 7.3.↜  Rates of return on three assets.

The functions in row 8 of this table compute the mean (expected) rate of
return on each asset; these are 5%, 3%, and 4%, respectively.

A portfolio

A portfolio is a set of levels of investment, each in a particular asset. The

net return (profit) R on a portfolio is uncertain; it depends on the state that
will occur. The portfolio that invests the fractions 0.6, 0.3 and 0.1 in assets
1, 2, and 3, respectively, is evaluated in Table 7.4. The functions in cells G4
230 Linear Programming and Generalizations

through G6 specify the value taken by R under outcomes a through c. Cell G4

reports that, if outcome a occurs, the rate of return on this portfolio will be
−0.03 = (0.6)*(−0.2)â•›+â•›(0.3)*(0.4)â•›+â•›(0.1)*(−0.3), for example.

The function in cell G11 computes the expectation E(R) of the return on
this portfolio. The functions in cells H4 through H6 compute the difference
Râ•›−â•›E(R) between the return R and its expectation if states a, b and c occur.
The function in cell H11 computes Var(R) because the variance equals the
expectation of the squared difference between the outcome and the mean.

Table 7.4↜.  The return on a particular portfolio.

Efficiency

Individuals and companies often take E(R) as a measure of desirability

(higher expectation being preferred) and Var(R) as a measure of risk (lower
variance being preferred). With these preferences, a portfolio is said to be
efficient if it achieves the smallest variance in profit over all portfolios whose
expected profit is at least as large as its expected profit. If a portfolio is not ef-
ficient, some other portfolio has less risk and has mean return that is at least
as large. To illustrate the construction of an efficient portfolio, consider

Problem 7.C, part (a). ╇ For the data in Table 7.3, find the minimum-variance
portfolio whose expected rate return rate is at least 3%.

It is not difficult to show (we omit this) that Var(R) is a convex quadratic
function of the fractions invested in the various assets. For that reason, mini-
 Chapter 7: Eric V. Denardo 231

mizing Var(R) subject to a constraint that keeps E(R) from falling below a
prescribed bound is a garden-variety (easily solved) nonlinear program.

The spreadsheet in Table 7.5 exhibits the portfolio that minimizes Var(R)

subject to E(R)â•›≥â•›0.03. The data and functions in Table 7.4 are reproduced in
Table 7.5. In addition, cell C9 contains the lower bound on the return rate,
which equals 0.03, and cell B9 contains a function that computes the sum
(f1â•›+â•›f2â•›+â•›f3) of the fractions invested in the three assets.

The GRG nonlinear code has been used to minimize the variance in the
return (cell H9) with the fractions invested in the three assets (cells D9:F9) as
the changing cells, subject to constraints that keep the fractions nonnegative,
keep their total equal to 1, and keep the mean return (cell G9) at least as large
as the number in cell C9. This portfolio invests roughly 47% in asset 2 and
roughly 53% in asset 3. It achieves a mean return rate of 3.53%. The standard
deviation in its rate of return is roughly 0.005. Evidently, if an investor seeks
a higher mean rate of return than 3.53%, she or he must accept more risk (a
higher variance, equivalently, a higher standard deviation).

Table 7.5↜  An efficient portfolio

The efficient frontier

The set of all pairs [E(R), Var(R)] for efficient portfolios is called the
efficient frontier. If a rational decision maker accepts E(R) as the measure of
desirability and Var(R) as the measure of risk, that person chooses a portfolio
on the efficient frontier. If a portfolio is not on the efficient frontier, some
other portfolio is preferable.

Problem 7.C, part (b). ╇ For the data in Table 7.3, find the portfolios that are
on the efficient frontier.
232 Linear Programming and Generalizations

No asset returns more than 5%, so placing a value greater than 0.05 in
cell C9 guarantees infeasibility. To find a family of portfolios that are on the
efficient frontier, one can repeat the calculation whose result is exhibited in
Table 7.5 with the number in cell C9 equal to a variety of values between 0.03
and 0.05. There is a technical difficulty, however.

Using Solver repeatedly

Suppose we solve the NLP with 0.034 in cell C9, then change that num-
ber to 0.038, and then solve again. The new solution replaces the only one.
This difficulty has been anticipated. Row 9 contains all of the information we
might want to keep from a particular run. Before making the 2nd run, “Copy”
row 9 onto the Clipboard and then use the Paste Special command to put only
its “Values” in row 14. After changing the entry in cell C9 and re-optimizing,
use the Paste Special command to put the new “Values” in row 15. And so
forth. Reported in Table 7.6 is the result of a calculation done with values of
C9 between 0.03 and 0.05 in increments of 0.004.

Table 7.6.↜  Portfolios on the efficient frontier.

Piecewise linearity

These portfolios exhibit piecewise linearity. As the mean rate of return in-
creases from 3.53% to 4.34%, the portfolio varies linearly. When the mean rate
of return reaches 4.34%, the fraction invested in asset 3 decreases to 0. As the
rate of return increases from 4.34% to 5%, the portfolio again varies linearly,
with f3 = 0 in this interval. Evidently, as the mean return rate increases, the
optimal portfolio “pivots” from one extreme point to another. This is the sort
of behavior that one expects in the optimal solution to a linear program. One is
led to wonder whether this nonlinear program is mimicking a linear program.
 Chapter 7: Eric V. Denardo 233

Using Premium Solver repeatedly

The calculation whose results are reported in Table 7.6 is a bit unwieldy.

To change the value in cell C9, one needs to close Solver, insert the new
number, and then reopen Solver. That’s because Solver is “modal.” When
Premium Solver is run off the Tools menu, it too is modal, and it is equally
unwieldy.

When Premium Solver is operated off the ribbon, it is “modeless,” and it

can easily be used to solve an optimization problem repeatedly with a variety
of values of a parameter. How to accomplish this is described with reference
to Figure 7.2. The left-hand side of this figure displays the pull-down menu
that appears when you click on Premium Solver on the ribbon. If you then
click on the drop-down entry entitled Model, the dialog box to the right of
Figure 7.2. appears.

Figure 7.2↜.  Premium Solver on the ribbon.

Suppose you wish to solve the portfolio optimization with cell C9 (the
lower bound on the mean return) equal to the 11 equally-spaced values 0.03,
0.032, 0.034, …, 0.05. To do so, follow this protocol:
234 Linear Programming and Generalizations

• Select cell C9 of the spreadsheet exhibited in Table 7.5.

• Click on the Premium Solver on the ribbon. The drop-down menu to

the left of Figure 7.2. will appear. On it, click on Parameters and then
click on Optimization. In the dialog box that appears, enter 0.03 as the
Lower value and 0.05 as the Upper value. This causes the function =
PsiOptParam(0.03,0.05) to appear in cell C9.

• Next, click again on Premium Solver on the ribbon. On the drop-down

menu that appears, click on Model. The dialog box to the right of Fig-
ure 7.2 will appear. In the menu at its top, click on “Plat…” A dialog box
will appear. In the “Optimizations to Run” window, enter 11

• Next, return to the Model tab on the dialog box to the right of Fig-
ure 7.2 Then click on the row containing the variables, cells D9:F9 in
this case. Make sure that the Monitor Value of these cells is set equal to
True. (If it is set equal to False, switch it.)

• Finally, either click on the green triangle to the right of the dialog
box that is displayed to the right of Figure 7.2 or click on Optimize
in the drop-down menu to the left of Figure 7.2 Either action causes
Premium Solver to solve the 11 optimization problems that you have
specified.

You can then scroll through the solutions to these optimization problems
by clicking on the window in a ribbon that currently reads “Opt. 11.” You can
also create a chart by clicking Charts on the drop-down menu.

The ribbon

The ribbon can also be used to specify an optimization problem. The

drop-down menu at the left of Figure 7.2 lets you specify the model’s decision
cells, constraints and objective. Using the ribbon can be easier because it al-
lows you to alter your spreadsheet without closing Premium Solver.

Measures of risk

By longstanding tradition, variance is used as the measure of risk. As

noted in Chapter 1, the variance puts heavy weight on observations that
are far from the mean. With μâ•›=â•›E(R), it might make better sense to accept
MAD(R) = E|R − µ| as the measure of risk.
 Chapter 7: Eric V. Denardo 235

These two measures of risk share a defect; Var(R) and MAD(R) place
large penalties on outcomes that are far better than the mean. It might make
still better sense to minimize the expectation of the amount by which the
mean exceeds the outcome, i.e., to accept E[(µ − R)+ ] as the measure of risk.

With either E|R − µ| or E[(µ − R)+ ] as the measure of risk, an efficient

portfolio can be found by solving a linear program, rather than a nonlinear
program. The optimal portfolio will continue to be a piecewise linear func-
tion of E(R), but the Allowable Increase and Allowable Decrease will deter-
mine the points at which the basis changes.

Getting the data

If the assets in a portfolio are common stocks of widely-traded compa-

nies, data from which to build a model like that in Table 7.3 can be obtained
from the historical record. For each of, say, twenty six-month periods, record
the “real” rate of return on each stock, this being the excess of its return over
the “risk-free” return, i.e., that of a six-month treasury bill for the same pe-
riod. Place the real rates of return for each period in a row. Assume that each
row represents a state that occurs with probability 1/20.

This approach relies on the “efficient markets hypothesis,” which states

that all of the publicly-available information about the future of a company
is contained in the current price of its stock. This hypothesis discounts the
possibility of “bubbles.” It does not predict the violent swings in market prices
that occur from time to time. It is widely used, nonetheless.

A bit of the history

The ideas and results in this section were developed by Harry Markow-
itz while he was a Ph. D. student at the University of Chicago. He published a
landmark paper in 1952, and he shared in the 1990 Nobel Prize in Economics,
which was awarded for “pioneering work in the theory of financial economics.”

6.  Modeling Decreasing Marginal Cost

As was noted in Chapter 1, when a linear program is used to model

increasing marginal cost, unintended options are introduced, but they are
ruled out by optimization. The opposite occurs when one attempts to use a
236 Linear Programming and Generalizations

linear program to model decreasing marginal cost; unintended options are

introduced, and they are selected by optimization. Decreasing marginal cost
– equivalently, increasing marginal profit – cannot be handled by linear pro-
grams. A method that does handle these situations will be developed in the
context of

Problem 7.D.╇ This problem appends to the ATV problem in Chapter 5 the
possibility of leasing tools that improve efficiency and thereby lower manu-
facturing costs. Tools α and β facilitate more efficient manufacture of Fancy
and Luxury model vehicles, respectively. Leasing tool α costs $1,800 per week,
and this tool reduces the cost of manufacturing each Fancy model vehicle by
$120. Similarly, leasing tool β costs $3,000 per week, and that tool reduces the
cost of producing each Luxury model vehicle by $300. The goal remains un-
changed; it is to operate the ATV plant in a way that maximizes contribution.
What production rates accomplish this?

Binary variables

Problem 7.D can be formulated as an optimization problem that differs

from a linear program in that two of its decision variables are required to take
the value 0 or 1. A decision variable whose values are restricted to 0 and 1 is
said to be binary.

An integer program

Throughout this text – and throughout much of the literature – the term
integer program is used to describe an optimization problem that would be a
linear program if requirement that some or all of its decision variables be in-
teger-valued were deleted. An integer program can have no quadratic terms,
for instance. It might be more precise to describe this type of optimization
problem as an “integer linear program,” but that usage never took root. Two
different methods for solving integer programs are discussed in Chapter 14.
Both of these methods solve a sequence – often surprisingly short – of linear
programs.

Break-even values

Our goal is to formulate Problem 7.D for solution as an integer program,

rather than as a more complicated object. Let us begin by computing a break-
even value for each tool. The equation $120 Fâ•›=â•›$1,800 gives the value of F at
which we are indifferent between leasing tool α and not leasing it. Evidently,
 Chapter 7: Eric V. Denardo 237

leasing this tool is worthwhile when Fâ•›>â•›15â•›=â•›$1,800/$120. Similarly, the

break-even equation $300 Lâ•›=â•›$3,000 indicates that leasing tool β is worth-
while if Lâ•›>â•›10.

Binary variables will be used to model the leasing of these tools. Equating
the binary variable a to 1 corresponds to leasing tool α. Equating the binary
variable b to 1 corresponds to leasing tool β. Our goal is to formulate Problem
7.D as an optimization problem that differs from a linear program only in that
the variables a and b are binary.

Accounting for the contribution

Leasing tool α increases the contribution of each Fancy model vehicle

by $140, from $1,120 to $1,340, but it incurs a fixed cost of $1,800. The
contribution of the Fancy model vehicle can be accounted for by using
the binary variable a in the linear expression and constraints that appear
below:

1120 F1 + 1240 F2 − 1800a,

a ∈ {0,1}, F2 ≤ 40a,
F1 ≥ 0, F2 ≥ 0, F = F1 + F2.

The linear expression measures the contribution earned from Fancy

model vehicles. If a = 0, the constraint F2 ≤ 40a keeps F2 = 0, so F = F1 and the
linear expression reduces to 1120 F, which is the contribution earned without
the tool. If a = 1, the linear expression is maximized by setting F1 = 0 and F2
= F, which reduces it to 1240 F − 1800. As noted above, this is preferable to
1120 F if F exceeds 15.

The binary variable b accounts in a similar way for leasing the tool that
reduces the cost of producing Luxury model vehicles.

A spreadsheet

The spreadsheet in Table 7.7 prepared this optimization problem for so-

lution by Solver. Rows 5-9 account for the capacities of the five shops. Rows
10 and 11 model the constraints F2 ≤ 40a and L2 ≤ 40b. Rows 12 and 13
model the constraints Fâ•›=â•›F1â•›+â•›F2 and Lâ•›=â•›L1â•›+â•›L2.
238 Linear Programming and Generalizations

Table 7.7↜.  A spreadsheet for Problem 7.D.

Reported in Table 7.7 is the optimal solution to Problem 7.D. This optimal

solution has been found by maximizing the value in cell K4 with B3:J3 as chang-
ing cells, subject to constraints B3:H3â•›≥â•›0, I3:J3 binary, K5:K11â•›≤â•›M5:M11,
and K12:K13â•›=â•›M12:M13. Evidently, it is profitable to lease tool α but not tool
β. And it remains optimal to produce no Luxury model vehicles.

Constraining variables to be binary

To solve Problem 7.D with Solver or with Premium Solver, we need to

require that the decision variables in cells I3 and J3 be binary. An easy way to
do that is to call upon the “Add Constraints” dialog box in Figure 7.3. In the
left-hand window of Figure 7.3, enter I3:J3. Then click on the center window
and scroll down to “bin” and then release. After you do so, “bin” will appear in
the center window and the word “binary” will appear in the right window. It
will not work to select “=” in the center window and enter the word “binary”
in the right-hand window, incidentally.

Figure 7.3.↜  Specifying binary variables.

 Chapter 7: Eric V. Denardo 239

Solving integer programs

After you formulate your integer program, but before you click on the
Solver button:

• with Solver in Excel 2003, click on “Assume Linear Model;”

• with Solver in Excel 2010, select “Simplex LP;”

• with Premium Solver, select “Standard LP/Quadratic.”

If you follow these rules, a method akin to those in Chapter 14 will be

used, with good results. If you do not follow these rules, a more sophisticated
method will be used. That method seeks a “local optimum,” which may not
be a global optimum.

No shadow prices?

If you present Solver or Premium Solver with an optimization problem

that includes any integer-valued variables, it does not report shadow prices.
Let us see why that is so.

First, consider the case in which all of the decision variables must be
integer-valued. In this case, shadow prices cannot exist because perturbing
a RHS value by a small amount causes the optimization problem to become
infeasible.

Next, consider the case in which only some of the decision variables must
be integer-valued. In this case, perturbing a RHS value may preserve feasibil-
ity, but it may cause an abrupt change in the objective value. When that oc-
curs, the shadow price cannot exist.

Finally, suppose a constraint did have a shadow price. It applies to a

small change in a RHS value, but it gives no information about the effect
of larger changes. If a constraint’s shadow price equals 2, for instance, in-
creasing that constraints RHS value by δ increases the objective by 2δ if δ
is close enough to 0. But the objective could increase by more than 2δ if δ
were larger.
240 Linear Programming and Generalizations

A nonlinear integer program

The term nonlinear integer program is used to describe an optimization

problem that would be a nonlinear program if we omitted the requirement
that some or all of its decision variables be integer-valued. The GRG code
tackles such problems, but it seeks a local optimum, which may or may not be
a global optimum.

Problem 7.D illustrates this phenomenon. It is not hard to show that the
feasible solution Sâ•›=â•›35, Fâ•›=â•›0 and Lâ•›=â•›15 is a local maximum. Perturbing this
solution by setting Lâ•›=â•›1 decreases the objective value by $50, for instance. If
the GRG code encounters this feasible solution, it will stop; it has found a lo-
cal maximum that is not a global maximum.

7.  The Traveling Salesperson

The data in the “traveling salesperson problem” are the number of cities
that the salesperson is to visit and the travel times from city to city. A tour oc-
curs if the salesperson starts at one of these cities and visits each of the other
cities exactly once prior to returning to the city at which he or she began. The
length of the tour is the sum of the times it takes to travel from each city to
the next. The traveling salesperson problem is that of finding a tour whose
length is smallest. The traveling salesperson problem may sound a bit con-
trived, but it arises in a variety of contexts, including

Problem 7.E (scheduling jobs).╇ Five different jobs must be done on a sin-
gle machine. The needed to perform each job is independent of the job that
preceded it, but the time needed to reset the machine to perform each job
does vary with the job that preceded it. Rows 3 to 9 of Table 7.8 specifies the
times needed to reset the machine to accomplish each of the five jobs. “Job 0”
marks the start, and “job 6” marks the finish. Each reset time is given in min-
utes. This table shows, for instance, that doing job 1 first entails a 3-minute
setup and that doing job 4 immediately after job 1 entails a 17-minute reset
time. Reset times of 100 minutes represent job sequences that are not allowed.
The goal is to perform all five jobs in the shortest possible time, equivalently,
to minimize the sum of the times needed to set up the machine to perform
the five jobs.
 Chapter 7: Eric V. Denardo 241

Table 7.8.↜  Data and solution of Problem 7.E.

The offset function

The reset times in Table 7.8 form a two-dimensional array. Excel’s “offset”

function identifies a particular element in such an array. If the Excel function
=OFFSET(X, Y, Z) is entered in a cell, that cell records the number in the cell
that is Y rows below and Z rows to the right of cell X. For instance, entering
the function =OFFSET(C4, 1, 3) in cell K2 would cause the number 21 to ap-
pear in cell K2; this occurs because 21 is the number that’s 1 row below and 3
columns to the right of cell C4.

A job sequence and its reset times

Row 11 of Table 7.8 records a particular sequence in which the jobs are

performed, namely, job 2, then job 1, then job 4, and so forth. The “offset”
functions in row 12 record the times needed to prepare the machine to perform
each of these jobs. Note that the offset function in Cell D12 gives the setup time
needed to do job 2 first. Also, the offset function in cell E12 records the reset
time needed to do job 1 second given that job 2 is done first. And so forth.

The Evolutionary Solver*

This subsection describes a solution method that uses the Standard Evo-
lutionary Solver, which exists only in Premium Solver. If you do not have ac-
cess to Premium Solver, please skip to the next subsection.
242 Linear Programming and Generalizations

Table 7.8 records result of applying the Standard Evolutionary Solver to

Problem 7.e. The quantity in cell I15 was minimized with D11:H11 as chang-
ing cells, subject to constraints that the numbers in cells D11:H11 be integers
between 1 and 5 and that these integers be different from each other. The
requirement that these integers be different from each other was imposed by
selecting “dif ” in the middle window of the Add Constraints dialog box. The
Evolutionary Solver found the solution in Table 7.8. It did not find it quickly,
and that’s for the case of 5 jobs.

The assignment problem

The traveling salesperson problem has been widely studied, and sever-
al different methods of solution have been found to work well even when
the number n of cities is fairly large. One of these methods is based on the
“assignment problem.” A network flow model is called an assignment prob-
lem if it has 2 m nodes and m2 directed arcs with these properties:

• The network has m “supply” nodes, with a fixed flow of 1 into each sup-
ply node.

• The network has m “demand” nodes, with a fixed flow of 1 out of each
demand node.

• It has a directed arc pointing from each supply node to each demand
node. The flows on these arcs are nonnegative.

Each fixed flow equals 1, so the assignment problem has integer-valued

data. The Integrality Theorem guarantees that that each basic solution to the
assignment problem is integer-valued.

An assignment problem with side constraints

In Table 7.9, Problem 7.E is viewed as an assignment problem with “side

constraints.” Rows 2-10 of this spreadsheet are identical to rows 2-10 of Ta-
ble 7.8. These rows have been hidden to save space. The rows that are dis-
played in Table 7.9 have these properties:

• Each cell in the array D12:I17 contains the shipping quantity from the
“supply node” in its row to the “demand node” in its column.

• The SUMPRODUCT function in cell B20 computes the cost of the

shipment.
 Chapter 7: Eric V. Denardo 243

Solver had been asked to find the least-cost assignment. This assignment
ships one unit out of each supply node and one unit into each demand node.
The solution to this assignment problem is not reported in Table 7.9. With x(i, j)
as the flow from source node i to demand node j, the least-cost assignment
sets

1 = x(0, 2) = x(2, 1) = x(1, 4) = x(4, 6) ,

1 = x(3, 5) = x(5, 3) ,

and has 51 minutes as its objective value.

Table 7.9↜.  Viewing Problem 7.E as an assignment problem with side constraints.

Subtours

This optimal solution identifies the job sequences 0-2-1-4-6 and 3-5-3.
Neither of these is a tour. These job sequences correspond to subtours because
neither of them includes all of the jobs (cities in case of a traveling salesperson
problem).

A subtour elimination constraint

To eliminate the subtour 3-5-3, it suffices to append to the assignment

problem the constraint x(3, 5)â•›+â•›x(5, 3)â•›≤â•›1. The function in cell L20 and the
constraint L20â•›≤â•›1 enforce this constraint. There is no guarantee that the re-
sulting linear program will have an integer-valued optimal solution, and there
is no guarantee that it will not have some other subtour.
244 Linear Programming and Generalizations

An optimal solution

Table 7.9 reports the optimal solution to the assignment problem supple-

mented by this constraint. This optimal solution is integer-valued, and it cor-
responds to the tour (job sequence) 0-2-1-4-5-3-6. This job sequence requires
55 minutes of reset time, and no job sequence requires less.

We could have imposed the constraint that eliminates the other subtour.
That constraint is x(0, 2)â•›+â•›x(2, 1)â•›+â•›x(1, 4)â•›+â•›x(4, 6)â•›≤â•›3.

The general situation

In larger problems, it can be necessary to solve the constrained assignment

problem repeatedly, each time with more subtour elimination constraints. It
can be necessary to require particular decision variables to be binary. There is
no guarantee that this approach converges quickly to an optimal solution to
the traveling salesperson problem, but it often does.

8.  College Admissions*

This section discusses a subject with which every college student is famil-
iar. This section is starred because readers who have not studied elementary
probability may find it to be challenging.

Problem 7.F. ╇ You are the Dean of Admissions at a liberal arts college that
has a strong academic tradition and has several vibrant sports programs. You
seek a freshman class of 510 persons. An agreement has been reached with the
head coach of each of several sports. These agreements allow each coach to
admit a limited number of academically-qualified applicants who that coach
seeks to recruit for his or her team. The coaches have selected a total of 280
such persons. From past data, you estimate that each of these 280 people will
join the entering class with probability of 0.75, independent of the others.
Your college has no dearth of qualified applicants. From past experience, you
estimate that each qualified person you accept who has not been selected (and
courted) by a coach will join the entering class with probability of 0.6. Your
provost is willing to risk one chance in 20 of having an entering class that
is larger than the target of 510. How many offers should you make to non-
athletes? What is the expectation of the number of students who will join the
freshman class?
 Chapter 7: Eric V. Denardo 245

The binomial distribution

The “binomial” distribution is the natural model for situations of this

type. If n students are offered admission and if each of them joins the class
with probability p, independently of the others, the number N who join the
class has the binomial distribution with parameters n and p. The mean and
variance of this binomial distribution are easily seen to be E(N)â•›=â•›n p and
Var(N)â•›=â•›n p (1â•›−â•›p).

In particular, the number A of athletes who will join the class has the bi-
nomial distribution with parameters nâ•›=â•›280 and pâ•›=â•›0.75. Thus, the mean and
variance of A are given by

E(A) = 280 × 0.75 = 210 , Var(A) = 280 × 0.75 × 0.25 = 52.5 .

The decision you face as Dean of Admissions is to determine the number

n of offers of admission to make to applicants who are not being recruited for
athletic teams. If you offer admission to n such people, the number N of them
who will join the freshman class also has the binomial distribution, with

E(N) = n × 0.6 , Var(N) = n × 0.6 × 0.4 .

The random variables A and N are mutually independent because stu-

dents decide to come to your college independently of each other. The total
number, Aâ•›+â•›N, of students in the entering class would be binomial if each
person who is admitted joins with the same probability. That is not the case,
however. The total number, Aâ•›+â•›N, of persons in the entering class does not
have the binomial distribution.

A normal approximation

If a binomial distribution with parameters n and p has an expected num-

ber n p of “successes” and an expected number n(1â•›−â•›p) of “failures” that are
equal to 7 or more, it is well-approximated by a random variable that has
the normal distribution with the same mean and variance. The quality of
the approximation improves as the numbers n p and n(1â•›−â•›p) grow larger.
The binomially-distributed random variable A and N have values of n p and
n(1â•›−â•›p) that are far larger than 7, for which reason A and N are very well
approximated by random variables whose distributions are normal.
246 Linear Programming and Generalizations

Adding normal random variables

The sum N1â•›+â•›N2 of independent normal random variables N1 and N2

is a random variable whose distribution is normal. Thus, the number of
people who will join the freshman class is very well approximated by a
random variable C whose distribution is normal with mean and variance
given by

E(C) = n × 0.6 + 280 × 0.75 ,

Var(C) = n × 0.6 × 0.4. + 280 × 0.75 × 0.25 .

A spreadsheet

The spreadsheet in Table 7.10 evaluates the yield from the pool of athletes
and non-athletes. Cell C4 contains the number of offers to make to non-ath-
letes. This number could have been required to be integer-valued, but doing
so would make little difference. The functions in cells F3 and G3 compute
the mean and variance of the yield from the athletes. The functions in cells
F4 and G4 compute the mean and variance of the yield from the others. The
functions in cells C8, C9 and C10 compute the mean, variance and standard
deviation of the class size C. The function in cell C12 computes the probabil-
ity that C does not exceed the target of 510.

Table 7.10.  The yield from admissions.

 Chapter 7: Eric V. Denardo 247

Solver has been asked to find the number in cell C4 such that C12â•›=â•›C13.
Evidently, you should offer admission to approximately 465 non-athletes.

How’s that?

A binomially-distributed random variable N assigns values only to inte-

gers. A normally-distributed random variable X assigns probabilities to inter-
vals; the probability that X takes any particular value equals 0. How can an
integer-valued random variable N be approximated by a normally distributed
random variable X that has the same mean and variance? The approximation
occurs when X is rounded off to the nearest integer. For a given integer t, the
probability that Nâ•›=â•›t is approximated by the probability that X falls in the
interval between tâ•›−â•›0.5 and tâ•›+â•›0.5.

Fine tuning

For example, the probability that the class size does not exceed 510 is well
approximated by the probability that the normally distributed random vari-
able C does not exceed 510.5. A slightly more precise answer to the problem
you face as the Director of Admissions can be found by making these changes
to the spreadsheet in Table 7.10:

• Require that the decision variable in cell C4 be integer-valued.

• Arrange for Solver or Premium Solver to place the largest number in

cell C4 for which P(Câ•›≥â•›510.5) does not exceed 0.05.

If you make these changes, you will find that they result in a near-imper-
ceptible change in the number of non-athletes to whom admission is to be
offered.

What’s in cell C14?

The function in cell C14 requires explanation. The positive part (x)+ of the
number x is defined by (x)+â•›=â•›max{0, x}. Interpret (x)+ as the larger of x and 0.
When D denotes a random variable and q is a number (Dâ•›−â•›q)+ is the random
variable whose value equals the amount, if any, by which D exceeds q.

For a random variable D whose distribution is normal, the quantity

E[(Dâ•›−â•›q)+] is known as the normal loss function and is rather easy to com-
pute. Calculus buffs are welcome to work out the formula, but that is not
248 Linear Programming and Generalizations

necessary. One of the functions in OP_Tools is =NL(q, μ, σ) and this function

returns the value of E[(Dâ•›−â•›q)+] where D is a normally distributed random
variable whose mean and standard deviation equal μ and σ, respectively.

In the College Admissions problem, the random variable C denotes the

class size, and (Câ•›−â•›510)+ equals the amount, if any, by which the class size ex-
ceeds the target of 510 students. This random variable does have the normal
distribution. The function in cell C14 of Table 7.10 computes the expectation
of the excess, if any, of C over 510. This number equals 0.268. Thus, in the
event that C does exceed 510, the expectation of the amount by which C ex-
ceeds 510 equals 0.268/(0.05)â•›=â•›5.36.

9.  Design of an Electric Plant*

This section is starred because readers who have not had a course in
“elementary” probability may find it to be challenging. In many of the United
States, electric utilities are allowed to produce the power required by their
customers, and they are allowed to purchase power from other utilities. Prob-
lem 7.G, below, concerns a utility that is in such a state.1

Problem 7.G (a power plant)╇ You are the chief engineer for a utility company.
Your utility must satisfy the entire demand for electricity in the district it serves.
The rate D at which electricity is demanded by customers in your district is un-
certain (random), and it varies with the time of day and with the season. It is
convenient to measure this demand rate, D, in units of electricity per year, rather
than units per second or per hour. The load curve specifies for each value of t
the fraction F(t) of the year during which D does not exceed t. This load curve
is known. The distribution of D is approximately normal with a mean of 1250
thousand units per year and a standard deviation of 200 thousand units per year.
Your utility has no way to store electricity. It can produce electricity efficiently
with “base load” plant or less efficiently with “peak load” plant. It can also pur-
chase electricity from neighboring utilities that have spare capacity. The “trans-
fer” price at which this occurs has been set – tentatively – at 6.20 dollars per unit
of electricity. Of this transfer price, only the fuel cost is paid to the utility provid-
ing the power; the rest accrues to the state. The transfer price is intended to be

Connecticut is not such a state, and its utility rates in 2009 are exceeded only by
1╇

Hawaii’s.
 Chapter 7: Eric V. Denardo 249

high enough to motivate each utility to satisfy at least 98% of its annual power
requirement from its own production. The relevant costs are recorded in Ta-
ble 7.11. Annualized capital costs are incurred whether or not the plant is being
used to generate electricity. Fuel costs are incurred only for fuel that is consumed.

Table 7.11.↜渀 Capital and fuel costs per unit of electricity.

source of power base load peak load Transfer

plant plant
annualized capital cost ($/yr) 2.00 1.30 0.00
fuel cost ($/unit) 1.10 2.10 6.20

Your goal is to design the plant that minimizes the expected annualized
cost of supplying power to your customers. What is that cost? How much of
each type of plant should your utility possess? Will your utility produce at
least 98% of the power that its customers consume?

The plant

Base load plant is cheaper to operate (see Table 7.11), so you will not use
any peak-load plant unless your base-load plant is operating at capacity. For
the same reason, you will not purchase any electricity from other utilities un-
less your base-load and peak-load capacities are fully utilized. This leads to
the introduction of two decision variables:

q1 =â•›the capacity of the base-load plant.

q2 =â•›the total capacity of the base-load and peak-load plant.

The variables q1 and q2 are measured in units of electricity per year. From
Table 7.11, we see that base-load and peak-load plant have annualized capital
costs of 2.00 dollars per unit of capacity and 1.30 dollars per unit of capacity,
respectively. The annualized cost C of the plant is given by
C = 2.00 q1 + 1.30 (q2 − q1 ) ,

and the unit of measure of C is $/year.

The electricity

To C must be added the expected cost G of the generating or purchasing

the electricity that your utility’s customers consume over the course of the year.
250 Linear Programming and Generalizations

The random variable (Dâ•›−â•›q2)+ equals the annualized rate at which elec-
tricity is purchased from other utilities, this being the excess of D over the to-
tal capacity of the base-load and peak-load plant. This electricity costs $6.20
per unit, so its expected annual cost equals

(1) 6.20 E[(D − q2 )+ ] .

Similarly, the random variable (D − q1 )+ − (D − q2 )+ equals the annu-

alized rate at which electricity is satisfied by peak-load plant, this being the
excess of D over the capacity q1 of the base-load plant less the rate of purchase
from other utilities. Peak-load electricity costs $2.10 per unit. The expecta-
tion of the difference of two random variables equals the difference of their
expectations, even when they are dependent. For these reasons, the expected
annual cost of the fuel burned in peak-load plant equals

(2) 2.10 E[(D − q1 )+ ] − 2.10 E[(D − q2 )+ ].

Finally, D − (D − q1 )+ equals the annualized rate at which electric-

ity is satisfied by base-load plant, this being D less the excess, if any, of D
over the capacity of the base-load plant. This electricity costs $1.10 per
unit. Again, the expectation of the difference equals the difference of the
expectations. The expected annualized cost of the fuel burned in base-load
plant equals

(3) 1.10 E[D] − 1.10 E[(D − q1 )+ ]