Systems of Linear Equations and Matrices

Bowens Mathematics Chapter 1, 2

MATRIX: Definitions
Matrix: a rectangular array of numbers which we

treat as a single (collective) object

Demarcated within brackets, parentheses, or

double lines Denoted by bold capital letters Matrix has m rows and n columns

Definitions
Matrix: a rectangular array of numbers which we

treat as a single (collective) object Order of matrix = row X column Element of a matrix = element row column

Notation
Matrices are usually denoted using upper-case

letters, while the corresponding lower-case letters, with two subscript indices, represent the entries. In addition to using upper-case letters to symbolize matrices, many authors use a special typographical style, commonly boldface upright (non-italic), to further distinguish matrices from other mathematical objects. An alternative notation involves the use of a double-underline B name, with or without boldface with the variable style, e.g.,.
4

Definitions
Matrix Short notation

a11

a12

a13 a23 a33

This is the 3,2 element, or element32

A3 X 3 = a21 a22 a31 a32

Order = 3x3

Types of Matrices
Square Matrix; Row Matrix; Column Matrix;

Diagonal Matrix; Unit Matrix; Zero Matrix; Scalar Matrix; Sub Matrix; Symmetric Matrix

Definition
A square matrix is a matrix with the same number

of rows and columns. An n-by-n matrix is known as a square matrix of order n. Any two square matrices of the same order can be added and multiplied. A square matrix A is called invertible or non-singular if there exists a matrix B such that AB = In.

Definitions
Vector: a matrix containing a single row (row

vector) or single column (column vector) Square matrix: same # rows and columns

Definitions
Diagonal matrix: one or more non-zero values on

main diagonal (top-left to bottom-right); all zeros on off-diagonal

1 0 0 0 5 0 0 0 8
9

Definitions
Scalar matrix: diagonal matrix with same non-

zero values on entire main diagonal

5 0 0 0 5 0 0 0 5
10

Definitions
Unity or identity matrix is special case of scalar

matrix where diagonal values = 1

1 0 0 0 1 0 0 0 1
11

Definitions
Symmetric Matrix: matrix remains unchanged

when rows and columns are interchanged

5 2 1 2 6 1 1 1 5

12

Definitions

Equality: two matrices of same order and with all equal elements

AssumeA =

1 2 3 2

and B =

y 2

Then A = B implies that x=2 and y=3

13

Matrix Operations
In matrix algebra, an ordinary number is called a scalar.

In scalar multiplication, multiply all entries in matrix by the

scalar.
The scalar often known as scaling factor

12
14

3 -1 6 4

36 - 12 72 48

Addition and Subtraction

4 3 2 4 6 =? 4 5 3 7 2 4 3 2 4 6 2 =? 4 5 3 7 2 1
15

Some Laws of Matrix

Commutative Law

A+B=B+A Associative Law A + (B + C) = (A + B) + C Identity Law A+O=A AO=A O A = -A

O is zero matrix. In a zero matrix all entries are 0s.

16

Matrix Multiplication
Scalar can be used to multiply a matrix of any

dimension Multiplication of two matrices is contingent upon the satisfaction of different dimensional requirement
Given two matrices A and B and we want to find AB.

The conformability condition for multiplication is that the column of A (lead matrix) must be equal to the row dimension of B (lag matrix)

17

The Rule of Multiplication

Amn B pq = Cmq [m = p] A = a11 a12 b13 b23 a11b12 + a12b22 a11b13 + a12b23

B=

b11 b12 b21 b22

C = AB = a11b11 + a12b21

18

Example
1 3 A= 2 8 4 0 B= 5 2 9 3 A B = ?

19

Laws
A commutative law for matrix multiplication

does not hold.

AB BA
An associative law for matrix multiplication does

hold
A (B C) = (A B) C = A B C
The product of any matrix and the zero matrix is

zero matrix
A O = O A= O

20

Identity Matrix
A I = A and

I A=A The role of identity (I) matrix is analogous to the role of the number 1.

1 0 0 1

2 1 0 5

(1x2) + (0 x0) (1x1) + (0 x5) (0 x2) + (1x0) (0 x1) + (1x5)

2 1 0 5

21

Problem Set 2-4

2 1 -1 1 0 0 1 3 -2 0 1 1 1 0 1 0 0 1 5 0 1 2 2 4 0 1 2 3 4 5 6 6 1 3 -1

15.

17.

22

Problem 21
Interest at the rates 0.06, 0.07, and 0.08 is

earned on respective investments of $3000, $2000 and $4000. Compute the total interest by matrix multiplication.

23

Problem 22
Two canned meat spreads, Regular and Superior, are made by

grinding beef, pork and lamb together. The numbers of pounds of each meat in a 15-pound batch of each brand are in the following table. Suppose we wish to make 10 batches of Superior and 20 of Regular. Multiply the meat matrix in the table and the batch vector ( 10 20) and interpret the result. Suppose that the per pound prices of beef, pork, and lamb are $2.50, $2.00, and $3.00, respectively. Multiply the price vector and the meat matrix and interpret the results.
Brand Beef Superior Regular
24

Pounds of Pork 2 8 Lamb 5 3 8 4

Basic Operations
Transposition of vector or matrix from A to A': a11

a'11; a21 a'12; a12 a'21; etc.

From A to A'

a c
25

b d

b d

Transpose
Transpose: Swap rows with columns
a M = d g b e h c f i

a MT = b c
V T = [x

d e f

g h i
z]

V=

x y z

26

Basic Operations
Transposition of vector or matrix from A to A': a11

a'11; a21 a'12; a12 a'21; etc.

From A to A'

a c
27

b d

b d

Transpose
Transpose: Swap rows with columns
a M = d g b e h c f i

a MT = b c
V T = [x

d e f

g h i
z]

V=

x y z

28

Transpose of a Matrix
A matrix obtained by interchanging rows and columns.

29

Inverse of a Matrix
a(a ) = 1 a -1 is the multiplica tive inverse of a because the product of a and a -1 is the identity. Two square matrices are inverses of each other if their productis the identity matrix I. AA = I
30

-1

-1

How to find Inverse of a Matrix?

Gauss-Jordan Inversion

The Gauss-Jordan method finds A. It transforms the

augmented matrix (A|I) into the augmented matrix (I|A).

1 0 0 1

a b
31

Example: Find Inverse

7 3 A= 2 1 1 -3 A = -2 7
-1

3 2 -1 A = 1 -1 0 2 0 -1
32

1 2 -1 1 -1 A = 1 -1 -1 3 2 4 -5

Conditions of Inverse
It is a square matrix

It has independent rows and columns

Determinant of the matrix is not zero

33

What is determinants?
A useful number associated with an n n matrix

(1) If A = [a11 ] , det ( A) = a11

(2) If A =
a11 a21 a12 a22 , det ( A) = a11a22 - a12 a21

(3) For n > 2 : det ( A) = ai1Ci1 + ai 2Ci 2 + + ain Cin

or det ( A) = a1 j C1 j + a2 j C2 j + + a2 j C2 j
34

det( A) = ai1Ci1 + ai 2Ci 2 + + ain Cin det( A) = a1 j C1 j + a2 j C2 j + + a2 j C2 j

cofactor minor
i j

Cij (1)

M ij

Mij is the determinant obtained by deleting the ith row and jth column of A.
35

(- 1)

i+ j

+
+ + +

+ +

+ + -

+ +

+ +

+ i+ j

The cofactor is the minor with a sign change considered

Cij = (-1) M ij
36

Adjoint of a Square Matrix

Let A=[aij]nxn be a square matrix of order n, then

adjoint of A is defined to be transpose of matrix [Aij]nxn, where Aij is co-factor of aij in |A|.

37

Example
4 A= 0 3 4 B = -2 3
38

1 3 0 1 3 -1

-1 2 7 -5 1 4

Inverse of a Matrix
Inverse of a square

matrix is:

1 A AdjA A
1

An n n matrix is invertible if and only if

det( A) 0
39

Solution of Equation
Transform the equations in matrices Ax = d A = coefficient matrix x = variable matrix d = RHS values Calculate the determinant of A .|A| Calculate the cofactor matrix Calculate the Adj. A (Transpose of cofactor matrix) 8. Find the inverse of a matrix A-1= Adj A/|A| 9. Solve x = A-1d
1. 2. 3. 4. 5. 6. 7.

40

Example
A manufacturer produces three products: P, Q and R which he sells in two markets. Annual sales volume are indicated as follows:
Market . I II . P 10,000 6,000 Product Q 2,000 20,000 R 18,000 8,000 . . .

(i) If unit sale prices of P, Q, and R are Tk. 2.50, 1.25 and 1.50 respectively, find the total revenue in each market with the help of matrix algebra. (ii)If the unit costs of the above 3 commodities are Tk. 1.80, 1.20 and 0.80 respectively, find the gross profit.

41

Basic Applications
Consider 2 equations, 2 unknowns:
2x + 3y = 5 3x + 2y = 5

Now define the following column vectors:

Then the equations can be written as

xa + yb = c

a=
42

2 3

,b =

3 2

,c =

5 5

Basic Applications
Why this works:
Recall matrix (vector) addition rule

2x

3x 2y (3x + 2 y) 5 And matrix (vector) equality rule

2x + 3y = 5 3x + 2y = 5

3y

(2 x + 3 y )

43

Equation System to Matrix

x+ y+z =6 4x - 2 y + z = 9 3x - y + z = 4
44

Inverse of a Matrix
Inverse of a square

matrix is:

1 A = adj A A
1

An n n matrix is invertible if and only if

45

Solution of Equation
Ax = d x = dA
1
1.
2. 3. 4. 5. 6. 7. 8. 9.
46

Transform the equations in matrices Ax = d A = coefficient matrix x = variable matrix d = RHS values Calculate the determinant of A .|A| Calculate the cofactor matrix Calculate the Adj. A (Transpose of cofactor matrix) Find the inverse of a matrix A-1= Adj A/|A| Solve x = A-1d

Example

2 x1 - 3x 2 = 6 x1 + 5 x2 = 29 3x1 - 4x 2 = 11

47

Markov Process
Markov Property: The state of the system at time t+1 depends only on the

state of the system at time t

Pr [X t +1 = xt +1 | X 1 X t = x1 xt ] = Pr [X t +1 = xt +1 | X t = xt
X1 X2 X3 X4 X5

Stationary Assumption: Transition probabilities are independent of time (t)

48

Markov Process
Simple Example

Weather: raining today 40% rain tomorrow 60% no rain tomorrow not raining today

20% rain tomorrow 80% no rain tomorrow

Stochastic FSM: 0.4

0.6

0.8

rain
0.2

no rain

49

Markov Process
Simple Example

Weather: raining today 40% rain tomorrow 60% no rain tomorrow not raining today

20% rain tomorrow 80% no rain tomorrow

The transition matrix:

P=

0.4 0.6 0.2 0.8

Stochastic matrix: Rows sum up to 1 Double stochastic matrix: Rows and columns sum up to 1

50

Markov Process
Gamblers Example Gambler - At

starts with $10

each play we have one of the following:

Gambler wins $1 with probability p

Gambler looses $1 with probability 1-p

Game

ends when gambler goes broke, or gains a fortune of $100

p p

(Both 0 and 100 are absorbing states)

p p

0
1-p

1
1-p

2
1-p 1-p

99

100

Start (10$) 51

Markov Process
Markov process - described by a stochastic FSM

Markov chain - a random walk on this graph

(distribution over paths)

We can ask more complex questions, like

Pr [X t + 2 = a | X t = b ] = ?
p

0
1-p

1
1-p

2
1-p 1-p

99

100

Start (10$)
52

Markov Process
Coke vs. Pepsi Example Given that a persons last cola purchase was Coke, there is a 90% chance that his next cola purchase will also be Coke. If a persons last cola purchase was Pepsi, there is an 80% chance that his next cola purchase will also be Pepsi.

transition matrix:

0.9 coke

0.1 pepsi 0.2

0.8

P=

0.9 0.1 0.2 0.8

53

Markov Process
Coke vs. Pepsi Example (cont)

Given that a person is currently a Pepsi purchaser, what is the probability that he will purchase Coke two purchases from now?

Pr[ Pepsi?Coke ] = Pr[ PepsiCokeCoke ] + Pr[ Pepsi Pepsi Coke ] =

0.2 * 0.9

0.8 *

0.2

= 0.34

0 . 9 0 . 1 2 P = P 0.2 0.8
Pepsi ?

0.9 0.1 0.2 0.8

? Coke
54

0.83 0.17 0.34 0.66

Markov Process
Coke vs. Pepsi Example (cont)

Given that a person is currently a Coke purchaser, what is the probability that he will purchase Pepsi three purchases from now?

P =

0.9 0.1 0.2 0.8

0.83 0.17 0.34 0.66

0.781 0.219 0.438 0.562

55

Markov Process
Coke vs. Pepsi Example (cont)
Assume each person makes one cola purchase per week Suppose 60% of all people now drink Coke, and 40% drink Pepsi What fraction of people will be drinking Coke three weeks from now?

P=

0.9 0.1 0.2 0.8

P =

0.781 0.219 0.438 0.562

Pr[X3=Coke] = 0.6 * 0.781 + 0.4 * 0.438 = 0.6438

Qi - the distribution in week i Q0=(0.6,0.4) - initial distribution Q3= Q0 * P3 =(0.6438,0.3562)

56

Markov Process
Coke vs. Pepsi Example (cont)
Simulation:
2/3

[
Pr[Xi = Coke]

0.9 0.1 0.2 0.8

= [2 3

stationary distribution

0.9

0.1

0.8

coke
0.2

pepsi

week - i

57

Steady State
A steady state vector is the state vector that

remains unchanged by the transition matrix

[v1 v2] X [T] = [v1 v2]

58

Example
What is steady state

matrix?

v1 v2

0 .8 0.2 0.6 0.4

v1 + v2 = 1
59

Example
Absorbing Markov

Stationary Markov

.6 .4
60