10.

34: Numerical Methods

Applied to
Chemical Engineering
Lecture 3:
Existence and uniqueness of solutions
Four fundamental subspaces

1
 Recap

• Scalars, vectors, and matrices

• Transformations/maps

• Determinant

• Induced norms

• Condition number

2
 Recap
• Matrices:

• Matrices are maps between vector spaces!

y = Ax

0 1
-2 1 0
A = @ 1 -2 1 A
0 1 -2
3
 Recap
• Matrices:

• Matrices are maps between vector spaces!

y = Ax

0 1
-2 1 0 1
A = @ 1 -2 1 1 A
0 1 -2 1
4
 Recap
• Matrices:

• Matrices are maps between vector spaces!

y = Ax

0 1
-2 1 0
B 1 -2 1 C
A=B
@ 0
C
1 -2 A
1 1 1
5
 Recap
• Matrices:

• Matrices are maps between vector spaces!

y = Ax

ssT
A=I-
ksk22

6
 Existence and Uniqueness
• Example:
phosphate (P)
5 dirt (D)
phosphate (P) water (W)
dirt (D)
water (W) 6
dryer
1 3 phosphate (P)
washer dirt (D)
phosphate (P) water (W)
phosphate (P)
dirt (D)
dirt (D)
water (W) 4 water (W)
phosphate (P)
dirt (D) 9
water (W)

2 8
mixer decanter
phosphate (P) phosphate (P)
dirt (D) dirt (D)
water (W) water (W) phosphate (P)
7
dirt (D)
water (W)

Stream 1 carries 1800 kg/hr P, 1200 kg/hr D and 0 kg/hr W

Stream 2 carries 0 kg/hr P, 0 kg/hr D and 10000 kg/hr W
Stream 3 carries 0 kg/hr D and 50% W into the washer
Stream 4 carries 0 kg/hr P
Stream 5 carries 0 kg/hr P and 0 kg/hr D
Stream 6 carries 0 kg/hr D and 0 kg/hr W
Stream 7 carries 0 kg/hr P, 95% of D into the decanter, 5% of W into the decanter
Stream 8 carries 0 kg/hr P
Stream 9 carries 0 kg/hr P

Does a solution exist? Is it unique? 7

 Vector Spaces
• N
R is an example of a vector space
• A vectors space is a “special” set of vectors

• Properties of a vector space:

• closed under addition:

x, y 2 S ) x + y 2 S
• closed under scalar multiplication:
x 2 S ) cx 2 S

• contains the null vector:

02S
• has an additive inverse:
x 2 S ) (-x) 2 S : x + (-x) = 0
8
 Vector Spaces

• Is this a vector space?

{(1, 0), (0, 1)}

• Is this a vector space?

{y : y = 1 (1, 0) + 2 (0, 1); 1, 2 2 R}

• Is this a vector space?

{y : y = 1 (1, 1, 0) + 2 (1, 0, 1); 1, 2 2 R}

9
 Vector Spaces

• Is this a vector space?

{(1, 0), (0, 1)}

• Is this a vector space?

{y : y = 1 (1, 0) + 2 (0, 1); 1, 2 2 R}

• Is this a vector space?

{y : y = 1 (1, 1, 0) + 2 (1, 0, 1); 1, 2 2 R}

10
 Vector Spaces
• A “subspace” is a subset of a vector space

• It is still closed under addition and scalar multiplication

• It still contains the null vector

• 2
For example, R is a subspace of R 3

• Is this a subspace?
{y : y = A((3, 0) + (0, 1)); A1 , A2 2 R}
• The linear combination of a set of vectors:
y=
M
X
Ai x i
i=1

• The set of all possible linear combinations of a set of

vectors is a subspace:
span{x1 , x2 , . . . , xM }
M
X
= {y 2 RN : y = Ai xi ; Ai 2 R, i = 1, . . . , M }
11
i=1
 Linear Dependence

• If at least one non-trivial linear combination of a set of

vectors is equal to the null vector, the set is said to be
linearly dependent.

• The set {x1 , x2 , . . . , xM } with xi 2 RN is

linearly dependent if there exists at least one Ai 6= 0
such that:
M
X
Ai x i = 0
i=1

• If M > N, then the set of vectors is always dependent

12
 Linear Dependence
• Example: are the columns of I linearly dependent?
0 1 0 1 0 1 0 1
1 0 0 1
@ 0 A 1+
@ 1 A 2+
@ 0 A 3 =@ 2
A=0
0 0 1 3

• Example: are these vectors linearly dependent?

0 1 0 1 0 1
2 -1 0
@ -1 A , @ 2 A , @ -1 A
0 -1 2

• In general, if Ax = 0 has a non-trivial solution, then the

vectors (Ac1 Ac2 . . . AcM ) are linearly dependent.

13
 Linear Dependence
• Uniqueness of solutions to: Ax = b

• If we can find one vector for which: Ax = 0 , then a

unique solution cannot exist.

• Proof:

• Let x = x H P
+ x , and Ax H P
= 0 while Ax = b
• If x H
6= 0 , x = cxH + xP is another solution.
• Therefore, x cannot be unique.

• Uniqueness of solutions requires the columns of a matrix

be linearly independent!

• 1 2
(A c
A c
. . . A c
M )x H
= 0 H
only if x = 0

• If a system has more variables than equations, then a

unique solution cannot exist. It is under constrained.
14
 Linear Dependence
• The dimension of a subspace is the minimum number of
linearly independent vectors required to describe the
span:
S = span{(1, 0, 0), (0, 1, 0), (0, 0, 1)}, dim S = 3
S = span{(1, 0, 0), (0, 1, 0), (0, 0, 1), (0, 0, 2)}, dim S = 3

• Example: can Ax = b have a unique solution?

0 1
1 4 0
B 2 5 7 C
A=B C
@ 3 6 8 A
0 7 9

15
 Linear Dependence
• The dimension of a subspace is the minimum number of
linearly independent vectors required to describe the
span:
S = span{(1, 0, 0), (0, 1, 0), (0, 0, 1)}, dim S = 3
S = span{(1, 0, 0), (0, 1, 0), (0, 0, 1), (0, 0, 2)}, dim S = 3

• Example: can Ax = b have a unique solution?

0 1
1 4 0
B 2 5 7 C
A=B C
@ 3 6 8 A
0 7 9

16
 Four Fundamental Subspaces
N ⇥M
A2R

• Column space (range space):

c c c
R(A) = span{A1 , A2 , . . . , AM }

• Null space:
M
N (A) = {x 2 R : Ax = 0}
• Row space:
R(AT ) = span{Ar1 , Ar2 , . . . , ArN }
• Left null space:
T N T
N (A ) = {x 2 R : A x = 0}

17
 Column Space
A2R N ⇥M R(A) = span{A1c , A2c , . . . , AcM }

• The column space of A is a subspace of RN

• Vectors in R(A) are linear combinations of the

columns of A

• Existence of solutions:

• Consider: Ax = b
M
X
c
xi Ai =b
i=1

• If x exists, then b is a linear combination of the

columns of A . b 2 R(A)

• Converse: if b 2
/ R(A), then x cannot exist
18
 Existence of Solutions
A2R N ⇥M R(A) = span{A1c , A2c , . . . , AcM }

• Solutions to Ax = b exist only if b 2 R(A)

• Example: 0
1 0 0
1

• Does a solution exist with A = @ 0 0 0 A

0 0 0
0 1
1
• If b = @ 0 A?
0
0 1
0
• If b = @ 1 A?
0 19
 Existence of Solutions
A2R N ⇥M R(A) = span{A1c , A2c , . . . , AcM }

• Solutions to Ax = b exist only if b 2 R(A)

• Example: 0
1 0 0
1

• Does a solution exist with A = @ 0 0 0 A

0 0 0
0 1
1
• If b = @ 0 A?
0
0 1
0
• If b = @ 1 A?
0 20
 Existence of Solutions
? 1.1 kg/s
3 kg/s
• Example: separator, 2:1
?
• Does a solution exist?
0 1 0 1
1 1 ✓ ◆ 3
@ 2 1 A x 1
=@ 0 A
x2
1 0 1.1

• What is the column space?

• Is b 2 R(A) ?

21
 Null Space
A 2 RN ⇥M
• The set of all vectors that are transformed into the
null vector by A is called the null space of A
N (A) = {x 2 RM : Ax = 0}
• The null space is a subset of RM

• Not the same as R(A)

• 0 is in the null space of all matrices but is trivial

• Uniqueness:

• Consider two solutions Ax = b, Ay = b

• Such that A(x y) = 0
• If dim N (A) = 0 , then x y = 0, x=y
• A unique solution exists
23
 An
N ⇥ N For a matrix A 2 C N ⇥ N , det(A) = l1 l2 . . . l N .
•example
• For a matrix A 2 , tr(A) = l1 + l2 + . . . l N .

Null Space
C
Consider the reaction network:
• For a matrix A 2 C N ⇥ N , tr(A) = l1 + l2 + . . . l N .
k1 k2 k4
An example A ! B ! C ! D.

•
k3 k5
Consider the reaction network: An example
Example: 42 10.34: numerical methods, lecture notes
Consider
If thek1reaction the reaction in anetwork:
k2 takes k4 place well mixed, batch vessel, then the rate of
A ! B ! C ! D.
• A series of chemical
change of concentration

If the reaction takes place in a well [mixed,

k3
0a matrix
A]
k5
reactions:
batch
of eachk1species
A ! B ! C ! D.
1 is0equal to a product
vessel,
k1 0 then
k2 is k4
k3
the 0 of
rate of 0
10
k5 its eigenvalues,
[ A]
1 rate
the

•
change Conservation
of concentration ofequation:
0 1 0
d B
Ifeach
the
B
singular.
dt @mass
change of
[ B] C
species
Breaction
[C ]
Cis
relates
Physically,
takes
concentration
A
B place
@ 0 of1
this must
B k1 in ka2well mixed,
C = B individual
each
0
k2 species
species
k
k3be the
1 rateis
k4 k5
case
batch Csince
0 vessel, B Bconservatio
C B [ then
C B They
equations.
A @ [C ]
] C Cthe rat
. no
C are
A
[ A] k1 0 independent. 0 0 [ A] 3
B C B [0D] 1 00 C 0B k4
C k5 1 [0D] 1
d B [ B] C B k1 k2 To[ Athe k]3 eigenvalue 0 kC 0[ B] C
B zero,
1 of there 0 is a corresponding
0 [ A]eigenv
B C=B B [kB] C C B
Bk k A @ [kC ] A k C . C B [ B] C
dt @ [C ] A @ When 0 k2ddescribes
the reaction
B 3 C kthe
reaches
4 B its steady-state
chemical engineer’s intuition: C
5 1 2 (d3 / dt[ ⇤ ] 0
= 0), the
at rate equa
Bsteady Cstat
B C=B C B C .
[D] 0 is simply:
tion 0dtas@it [is Ck]4depleted
A @ k5by 0 only k[2Dforward
] k3 reactions, k4 k5 Awhile @ [C[B], ] A [C
[ Dnon-zero
] 0 00 kequilibrium k5 of0 [D ] 1
•
0
When the reaction reaches its ksteady-state obtain values set
1
(d0/dt[⇤] = 00), the rate by the 41 the reaction
Steady state: 1 0 [ A]equa- [ A]
tion is simply: BWhen these species. This eigenvector C B [ Bis] aCset of non-zero B theconcentraC eq
B 1 k the reaction
k 2 k reaches
3 0its steady-state
CB C (d / dt[ ⇤ ] = 0),
B [ B]rate C
B belongs to the null space C of
B the reaction
C = 0 rate matrix.
B C, s
This
0 @tion0is simply:
1k0 2 k 1 k k A0 @ [ C ]1 A @ [ C ] A
k1 0 0 0 determined [ A] 3 by assuming
4 5
A
[D] = 1 and performing back subs
[ ]
B k 0 0 C 0B k
C 4 k 5 B 1 [0 D ]C 1 [0D] 1
B 1 k2 k3 0 determine
kC 1 B 0[ B ] C 0
the concentrations 0 B [ Bof ] C A] other components.
[the [ AThe
]
B C B C = 0 or 0 B C ,
@ 0 k2 k4 B
k3 There k5 knon-zero
Bmust 1A be@ a [kC vectors
A
non-trivial
2 ] k 3 satisfying
solution 0 @ C[CB
steady-state
for
C Bthe
] A [ B C
steady
] C condition is
concentrations
B [ B] C
B – an C
B CB C = 0 or 0 B C
0 0 k4 eigenvector
@ k5 0 corresponding
k[2D ] k3 to 0k4an eigenvalue
k51 A [ D@]0 [C of] zero.
A The eigenvalues 1@ [C ] of A

• Null space ofthetherate rate 0

matrix
There must be a non-trivial solution for the steadyB
0
matrix:
0 givenk4by the
are
B
[ Asecular
] k5
concentrations
B [ B] C
C
C = c
B
[D]
equation:
– an
0
B (k3 /k2 ) (k5 /k4 ) 1
B
C
C
C.
[D]

eigenvector correspondingThere to an eigenvalue

must be of
l zero.@0The
k1 a non-trivial [C ]eigenvalues
solution
A for 0 the
@ ofsteady
k5 /k40concentrations
A –
the rate matrix are given by eigenvector B corresponding
the secular equation: k2 to l an C
B k1 [D ] eigenvalue k3 of zero.
1 0 The C eigenvalue
0 0 = rate
the det B matrix are given by the secular
1 equation: C (2.52)

•
@ 0 k2 k3 k4 l k5 A
What B is kthis subspace
k 1 l 0 where 0 geometrically?
0
c0is an arbitrary 0
0 scalar kthat is determined
C 4 k5 l by1a total
B 1 k2 l ance onk3the kh1 steady-state.
l 00 CBecause 0 the stoichiometric0 i 24rela
0 = det B B 2 C (2.52) C
 dimension of its column space:

Matrix Rank r = dim R(A).

The dimension of the null space is necessarily r M:

(2.46)

N ⇥M
R N (A) = r
A 2dim M.

• Rank ofrank
If the a matrix is the
of a matrix dimension
equals the number of its column
of columns in the space
matrix,
r = M, the dimension of the null space is zero. Such a matrix is said
r = dim R(A)
to be full rank. The rank of a matrix can be determined by reducing it
to row-echelon form: A ! U . This is done by following the Gaussian
• Finding the procedure
elimination rank: transform
will have the structure:
with pivoting. toThe
upper triangular
row-echelon formU ,
form matrix,
A!U 0 1
U11 U12 . . . U1r U1(r+1) . . . U1M
B 0
B U22 . . . U2r U2(r+1) . . . U2M C C
B .
B . .. .. .. .. .. .. CC
B . . . . . . . C
B C
U =B 0 0 . . . U rr Ur(r+1) . . . U rM
C. (2.47)
B C
B 0
B 0 ... 0 0 ... 0 C C
B C
@ 0 0 ... 0 0 ... 0 A
0 0 ... 0 0 ... 0

A square, non-singular matrix in C N ⇥ N , has rank r = N. The matrix

• is singular if r < N – the columns of the matrix are linearly dependent.

Rank nullity theorem:
A matrix in C N ⇥ M is termed rank deficient if r  min(N, M). Likewise,
when r < M, solutions, if any exist, are not unique.
dim N (A) = M r
25
Row space and left null space
Existence and Uniqueness
N ⇥M
A2R
• Existence:

• For any b in Ax = b

• A solution exists only if r = dim R(A) = N

• Uniqueness:

• A solution is unique only if dim N (A) = 0

• Equivalently when r = dim R(A) = M

26
MIT OpenCourseWare
httpV://ocw.mit.edu

10.34 Numerical Methods Applied to Chemical Engineering

Fall 2015

For information about citing these materials or our Terms of Use, visit: httpV://ocw.mit.edu/terms.