Gaussian Elimination in Matrix Algebra
Gaussian Elimination in Matrix Algebra
WEEK 5
LEARNING OUTCOMES
At the end of this section students should be able to:
Matrix and determinant methods for solving simultaneous equations are generally inefficient.
Another approach, known as reduction or elimination methods, is more efficient, and can handle
those cases where the previous matrix and determinant methods fail.
Well, matrix and determinant methods for solving simultaneous equations rely on det A not
equalling zero. If det A = 0, then other methods (that is, reduction methods) must be used to
solve the system of linear equations.
Gaussian Reduction
Reduction methods are a systemisation of the method of elimination. Consider for example the
following system of linear equations:
x 2y z 1 (1)
2x 3y z 3 (2)
3x y 2 z 2 (3)
To solve using the elimination method, we can begin by multiplying equation (1) by 2 and
subtracting equation (2) to eliminate x. Similar steps are then taken to eliminate either y or z,
leaving one equation in one unknown to solve. Using substitution, we can then find the values of
the other variable(s).
These calculations can be carried out by working with the array of coefficients and constants,
known as an augmented matrix. That is, we consider
1 2 1 1
2 3 1 3
3 1 2 2
The procedure is to use row operations on the array to reduce it to one in which all the elements
below the main diagonal are zero. The eliminations are carried out using row operations
according to the following rules for what is allowed:
The sequence of eliminations can now be written out as follows: begin with the array
1 2 1 1
2 3 1 3
3 1 2 2
1 2 1 1
0 1 1 1 R2 : R2 2 R1
0 7 5 1 R3 : R3 3R1
1 2 1 1
0 1 1 1
0 0 2 8 R3 : R3 7 R2
This array can now be used to find z, then y, and finally x. That is:
From Row 3: 2 z 8 z 4 .
From Row 2: y z 1 y 4 1 y 3 .
From Row 1: x 2 y z 1 x 6 4 1 x 2 1 x 1
In summary, the method is to use row operations on the array of coefficients so as to reduce the
array to one in which all elements below the main diagonal are zero. This method is referred to
as Gaussian elimination.
Basic Method
For a 3 3 system:
1 . . .
. . . .
. . . .
Step 2 : Get zeros as the first term in Rows 2 and 3, using Row 1.
1 . . .
0 . . .
0 . . .
Step 3 : Get a zero as the second element of Row 3, using Rows 2 and 3.
1 . . .
0 . . .
0 0 . .
Step 4 : Use Row 3 to find z, then Row 2 to find y, and Row 1 to find x.
2 x 3 y 4 z 19
6x 4 y 2z 8
x 5 y 4 z 23
Solution
x 5 y 4 z 23
6x 4 y 2z 8
2 x 3 y 4 z 19
When we now write the first array, a “1” appears as the first element in Row 1; that is, we get
1 5 4 23
6 4 2 8
2 3 4 19
We now proceed to “get zeros” as the first elements in Rows 2 and 3:
NOTE: The first element in Row 2 is 6 times the first element in Row 1, and the first element in
Row 3 is twice the first element in Row 1; so the appropriate row operations will be:
1 5 4 23
0 26 26 130 R2 : R2 6 R1
0 13 4 65 R3 : R3 2 R1
To get a zero as the second element in Row 3, two further row operations are needed.
1 5 4 23
0 1 1 5 R2 : R2 26
0 13 4 65
NOTE: The second element in Row 3 is – 13 times the second element in Row 2, so:
1 5 4 23
0 1 1 5
0 0 9 0 R3 : R3 13R2
4 x 2 y 3z 1
3x 2 y 2 z 5
x 3y 2z 1
Solution
x 3y 2z 1
3x 2 y 2 z 5
4 x 2 y 3z 1
When we now write down the first table, a “1” appears as the first element in Row 1; that is,
1 3 2 1
3 2 2 5
4 2 3 1
1 3 2 1
0 7 4 2 R2 R2 3R1
0 14 11 5 R3 R3 4 R1
Now to get a zero as the second element in Row 3, only one further row operation is needed:
1 3 2 1
0 7 4 2
0 0 3 9 R3 R3 2 R2
From Row 3, 3 z 9 , so z 3 .
From Row 2, 7 y 4z 2
7y 12 2
7y 14
y 2
From Row 1, x 3 y 2 z 1
x 6 6 1
x 1
A set of equations where the determinant of the coefficient matrix is equal to zero will have a
non-unique solution. This means that when we solve the set of equations, we will either have an
infinite number of solutions or no solution at all.
Examples
x y z 4
2x 3y z 3
3x 5 y 3z 2
Solution
1 1 1 1 1 1
Here A 2 3 1 , and det A 2 3 1 1 9 5 1 6 3 1 10 9 0 .
3 5 3 3 5 3
Hence we know that there is a non-unique solution for this set of equations. In fact this set of
equations reduces to (each student should check that they can do this):
1 1 1 4
0 1 3 5
0 0 0 0
From R3 : 0 x 0 y 0 z 0 0 0 .
This is a true statement, but not useful in determining a solution! This means that the set of
equations are redundant, meaning that there is no unique solution for x, y and z. In fact a
redundancy tells us that there are an infinite number of solutions that can be generated that will
satisfy the given equations.
To solve, let one of the unknowns be a parameter (for example, let z a ), and express the
others in terms of this.
From Row 2, y 3 z 5
y 3a 5
y 3a 5 y 5 3a
From Row 1, x y z 4
x 5 3a a 4
x 5 3a a 4 x 5 4a 4 x 9 4a
A particular solution is found by substituting any value for a. For example, we could let a 0 :
then x 9 , y 5 , z 0 .
x 2 y 3z 6
2x y 4z 2
4 x 3 y 2 z 14
Solution
1 2 3 6
2 1 4 2
4 3 2 14
Now we perform row operations on the augmented matrix as usual:
1 2 3 6
0 5 10 10 R2 R2 2 R1
0 5 10 10 R3 R3 4 R1
1 2 3 6
0 5 10 10
0 0 0 0 R3 R3 R2
5 y 10 z 10
5 y 10a 10
y 2a 2 y 2 2 a
But now from Row 1:
x 2 y 3z 6
x 2 2 2a 3a 6
x 4 4a 3a 6 x 4 a 6 x 2 a
Hence the infinite number of solutions can be generated by letting a equal any number in
x 2a
y 2 2a
z a
Example
Solution
1 2 3 1 0
1 3 1 1 2
2 1 1 3 3
4 4 1 5 5
1 2 3 1 0
0 1 4 0 2 R2 R2 R1
0 5 7 1 3 R3 R3 2 R1
0 4 11 1 5 R4 R4 4 R1
1 2 3 1 0
0 1 4 0 2
0 0 27 1 13 R3 R3 5 R2
0 0 27 1 13 R4 R4 4 R2
1 2 3 1 0
0 1 4 0 2
0 0 27 1 13
0 0 0 0 0 R4 R4 R3
Examples
x y z 4
2x 3y z 3
x 2 y 2z 2
Solution
1 1 1
Here det A 2 3 1 1 6 2 1 4 1 1 4 3 0 : non-unique solution! The
1 2 2
augmented matrix is
1 1 1 4
2 3 1 3 .
1 2 2 2
The last row gives 0x + 0y + 0z = – 1, i.e. 0 = – 1, which is impossible. This means the
equations are inconsistent, and there is no solution.
x 2 y 3z 1
3x y 2 z 7
5x 3 y 4 z 2
Answer
First we write down the augmented matrix, then we row reduce it as always.
The augmented matrix and relevant row operations are written below.
1 2 3 1
3 1 2 7
5 3 4 2
1 2 3 1
0 7 11 10 R2 R2 3R1
0 7 11 7 R3 R3 5 R1
1 2 3 1
0 7 11 10
0 0 0 3 R3 R3 R2
Unlike the determinant and matrix methods of solving systems of equations, the reduction
methods can be used when the number of unknowns and equations are not equal. By using row
reductions to produce zeroes below the diagonal, we can find the solutions of the set of equations
as before. When there are more equations than unknowns, the equations are usually inconsistent
and so there will be no solution. When there are more unknowns than equations the equations
will usually be redundant, and there will be an infinite number of solutions.
x 2y z 4
2x y z 3
x 3y 2
x y 2z 5
Solution
1 2 1 4
2 1 1 3
1 3 0 2
1 1 2 5
1 2 1 4
0 5 1 5 R2 R2 2 R1
0 5 1 6 R3 R3 R1
0 1 1 1 R4 R4 4 R1
1 2 1 4
0 0 0 1
R2 R2 R3
0 5 1 6
0 1 1 1
x y 2 z 2t 1
2x y z t 1
3x y z 2t 2
Solution
NOTE: This system of equations has four unknowns and three equations. There will be an
infinite number of solutions for this system, even though it turns out that the reduction does not
lead to a redundancy. Here is the augmented matrix:
1 1 2 2 1
2 1 1 1 1 .
3 1 1 2 2
1 1 2 2 1
0 3 5 3 1 R2 R2 2 R1
0 4 7 4 1 R3 R3 3R1
1 1 2 2 1
0 3 5 3 1
0 1 2 1 0 R3 R3 R2
1 1 2 2 1
0 1 2 1 0
0 3 5 3 1 R2 R3
1 1 2 2 1
0 1 2 1 0
0 0 1 0 1 R3 R3 3R2
From Row 3, z 1 .
From Row 1:
x y 2 z 2t 1
x 2 a 2 2a 1
x 2 a 2 2a 1
x a 1
x a 1
y 2 a
z 1
t a
Definition: A system of linear equations is said to be homogeneous if all the constants on the
right-hand side are zero.
x 2y z t 0
2x y z t 1
x y 3z t 0
2x 3y 2z t 0
A homogeneous system of equations always has one solution, namely the solution with all
unknowns equal to zero (known as the trivial solution). Thus a homogeneous system of
equations cannot be inconsistent. When solving homogeneous linear systems, there are only two
possibilities:
It can be shown that for n homogeneous linear equations to have a non-trivial solution, we must
have det A A 0 .
a1 b1 c1
a2 b2 c2 0 .
a3 b3 c3
Example
3x y 3z 0
x 4 y 2z 0
3x 10 y 12 z 0
First, check to see if this homogeneous system of equations will have a non-trivial solution; that
is, find the value of det A.
3 1 3
4 2 1 2 1 4
det A 1 4 2 3 1 3
10 12 3 12 3 10
3 10 12
3 48 20 1 12 6 3 10 12
0
As det A = 0, the equations will have a non-trivial solution. Therefore, we now know that an
infinite set of solutions exists, and so using reduction we should get a redundancy.
Now we can begin the row operations. Remember to rearrange the equations to get a “1” as the
first element in row 1; that is,
x 4 y 2z 0
3x y 3z 0
3x 10 y 12 z 0
1 4 2 0
3 1 3 0
3 10 12 0
1 4 2 0
0 11 9 0 R2 R2 3R1
0 22 18 0 R3 R3 3R1
1 4 2 0
0 11 9 0
0 0 0 0 R3 R3 2 R2
As previously, we know that this system will have an infinite number of solutions, as the
reduction has led to a redundancy.
11y 9a 0
9a
y
11
14a
x
11
9a
y
11
z a
We can use reduction methods to find matrix inverses. Essentially the method is as follows:
where the b 's are the elements in the inverse A 1 of matrix A.
That is,
b11 b12 b13
1
A b21 b22 b23 .
b b33
31 b32
2 1
Find the inverse of A .
5 3
Solution
2 1 1 0
5 3 0 1
1 1
1 0
R1
R1
2 2
5 3 0 1
2
1 1
1 0
2 2 R2 R2 5R1
0 11
5
1
2 2
1
1 1 2
0
2 2
0 5 2 R2 R2
1 11
11 11
1 0 3 1
1
11 11
R1 R1 R2
5 2 2
0 1
11 11
3 1
11 11
Hence A1 .
5 2
11 11
x1
x2 .
xn
The values of are called the eigenvalues of the matrix A, and the corresponding solutions (X)
are called the eigenvectors of A.
Example
a b x1
In the 2 2 case, suppose A and X . Then the equation becomes:
c d x2
a b x1 x1
c d x2 x2
or
ax1 bx2 x1
cx1 dx2 x2
In other words,
a x1 bx2 0
cx1 d x2 0
As a matrix equation, this is
a b x1 0
c d x2 0
or
A I X 0.
This is then a set of homogeneous equations, which has only the trivial solution of X 0 , or an
infinite number of solutions including the trivial solution.
det A I 0 .
Remember, X will have an infinite number of solutions. Can you say why?
Examples
1 5
1. Find the eigenvalues and corresponding eigenvectors for the matrix A .
2 4
Solution
To find the eigenvalues , solve the characteristic equation for ; so solve A I 0 , i.e.
1 5
0.
2 4
1 4 10 0
4 5 2 10 0
2 5 6 0
6 1 0
Hence 6 or 1 .
1 6 5 0
2 46 0
We know that there will be an infinite number of solutions; so using row reductions as before,
we should obtain a redundancy from
5 5 0
.
2 2 0
x1 x1 1
Hence X x1 is the general form of the eigenvector corresponding to the
x2 x1 1
eigenvalue 6 .
The eigenvector is then often given as just the vector without the constant multiple out the front.
1
In other words, we often say the eigenvector corresponding to the eigenvalue of 6 is X .
1
Now, letting 1 : A I X 0 becomes
11 5 0 2 5 0
.
2 4 1 0 2 5 0
5
Thus 2 x1 5 x2 0 x1 x2 ; and the corresponding eigenvector is
2
5 x2 5
x1
X 2 x 2 ,
x2 x
2
2 1
5
or just X 2 , or even X 5 (multiplying throughout by 2).
1
2
6 2 5
A 1 1 1 .
8 2 7
Solution
6 2 5
1 1 1 0.
8 2 7
Now substitute values for (remember they must be factors of the constant term 2); for example,
try 1 . Here 1 2 1 1 2 0 . So 1 is a solution.
3 2
Now to find the corresponding eigenvectors, solve A I X 0 for the three (3) values of .
Letting 1 , A I X 0 becomes
6 1 2 5 x1 0
1 11 1 x2 0
8 2 7 1 x3 0
7 2 5 x1 0
1 0 1 x2 0
8 2 6 x3 0
Now do row operations on the augmented matrix below to solve this system of equations.
Remember that you should always end up with a zero row!
7 2 5 0 0 2 2 0 R1 R1 7 R2
1 0 1 0 1 0 1 0
8 2 6 0 0 2 2 0 R3 R3 8 R2
0 2 2 0
1 0 1 0
0 0 0 0 R3 R3 R1
x1 x3 1
So X x2 x3 x3 1 is the general form of the eigenvector corresponding to the
x x 1
3 3
eigenvalue 1 .
6 2 2 5 x1 0
1 1 2 1 x2 0
8 2 7 2 x3 0
8 2 5 x1 0
1 1 1 x2 0
8 2 5 x3 0
Now do row operations on the augmented matrix below to solve this system of equations.
8 2 5 0
1 1 1 0
8 2 5 0
0 6 3 0 R1 R1 8 R2
1 1 1 0
0 6 3 0 R3 R3 8 R2
0 6 3 0
1 1 1 0
0 0 R3 R3 R1
0 0
x1 x2 1
So X x2 x2 x2 1 is the general form of the eigenvector corresponding to the
x 2x 2
3 2
eigenvalue 2 .
6 1 2 5 x1 0
1 11 1 x2 0
8 2 7 1 x3 0
5 2 5 x1 0
1 2 1 x2 0
8 2 8 x3 0
Now do row operations on the augmented matrix below to solve this system of equations.
5 2 5 0
1 2 1 0
8 2 8 0
0 12 0 0 R1 R1 5 R2
1 2 1 0
0 18 0 R3 R3 8 R2
0
0 12 0 0
1 2 1 0
0 0 R3 R3 1.5 R1
0 0
x1 x3 1
So X x2 0 x3 0 is the general form of the eigenvector corresponding to the
x x 1
3 3
eigenvalue 1 .
1 3
1. Find the inverse of A using reduction methods.
2 4
2. Solve the following simultaneous equations by reduction.
x 2y z 5
x 3y 2
(a) (b) 2x y z 0
2x y 3
x y 3z 8
x y 4z 1 x1 2 x2 3 x3 2
(c) 2x 7 y 6z 2 (d) 4 x1 5 x2 6 x3 8
x 9 y 8z 3 7 x1 8 x2 9 x3 14
3. Find the eigenvalues and corresponding eigenvectors for the following matrices.
0 3 0 1 2 7
1 3
(a) (b) 1 2 0 (c) 0 1 3
2 4 1 1 0 2
1 0
3 2 1
4. Show that 1 4 1 has an eigenvalue of 2 , and find the corresponding
4 2
0
eigenvector.
3 2 1
5. Given that 3 is an eigenvalue of 1 4 1 , find a corresponding eigenvector.
4 2
0
ANSWERS
1 4 3
Q1.
10 2 1
x1 3a 3
Q3. (a) Eigenvalue 1 , eigenvector a
x2 2a 2
x1 x2 1
Eigenvalue 2 , eigenvector x2
x2 x2 1
0 0 x1 1
Q4. x2 x2 1 Q5. 5 x1 x1 5
2x 2 4x 4
2 1
Extra Questions
More Questions 6
Do you understand this topic?
1. Can we use reduction to solve every set of simultaneous equations?
2. If det A = 0, what type of solution(s) can you expect to a set of simultaneous equations?
3. What is: (i) a redundancy (ii) an inconsistency?
4. What is a homogeneous set of equations? How do you tell if that set of homogeneous
equations has a nontrivial set of solutions?
5. How do you find an eigenvalue and an eigenvector?
6. Any other questions from the lecture notes?
2. Three machines together produce 650 parts per hour. Twice the production of the second
machine is 10 parts per hour more than the sum of the production of the other two
machines. If the first machine operates for 3 hours and the other two for 2 hours, 1550 parts
are produced. Let x, y and z be the number of parts produced by the first, second and third
machines respectively. Write down a system of equations relating x, y and z, and solve this
system to determine how many parts are produced by each machine.
4 6 6
4. (i) Determine the eigenvalues of the matrix A 0 1
0 .
1 1
1
(ii) Find ONE corresponding eigenvector for the matrix given in (i).
Answers
13a 6
1. (i) x=y=z=0 (ii) det A = 0, so an infinite no. of solutions (c) a 6
a
x y z 650
c1b2 c2b1 c a c1a2
2. 2 y x z 10 x 250, y 220, z 180 3. x , y 2 1
a1b2 a2b1 a1b2 a2b1
3x 2 y 2 z 1550
3 2 6
4. 2, X 0 ; 1, X 0 ; 1, X 6
1 1 1