Linear Algebra Operations in Python

Ahmet Goncu

September 28, 2024

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 1 / 12

Introduction

Python is a powerful tool for linear algebra computations

We’ll use NumPy, the fundamental package for scientific computing
in Python
Key topics: vectors, matrices, and common operations

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 2 / 12

 Setting Up

First, import NumPy:

1 import numpy as np

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 3 / 12

 Creating Vectors

1 # Create a vector
2 v = np . array ([1 , 2 , 3])
3 print ( v ) # Output : [1 2 3]
4
5 # Create a vector of zeros
6 zero_vector = np . zeros (3)
7 print ( zero_vector ) # Output : [0. 0. 0.]
8
9 # Create a vector of ones
10 one_vector = np . ones (3)
11 print ( one_vector ) # Output : [1. 1. 1.]

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 4 / 12

 Vector Operations

1 v1 = np . array ([1 , 2 , 3])

2 v2 = np . array ([4 , 5 , 6])
3
4 # Vector addition
5 print ( v1 + v2 ) # Output : [5 7 9]
6
7 # Scalar multiplication
8 print (2 * v1 ) # Output : [2 4 6]
9
10 # Dot product
11 print ( np . dot ( v1 , v2 ) ) # Output : 32
12
13 # Vector norm
14 print ( np . linalg . norm ( v1 ) ) # Output : 3.7416573867739413

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 5 / 12

 Creating Matrices
1 # Create a matrix
2 A = np . array ([[1 , 2] , [3 , 4]])
3 print ( A )
4 # Output :
5 # [[1 2]
6 # [3 4]]
7
8 # Create a matrix of zeros
9 zero_matrix = np . zeros ((2 , 2) )
10 print ( zero_matrix )
11 # Output :
12 # [[0. 0.]
13 # [0. 0.]]
14
15 # Create an identity matrix
16 I = np . eye (2)
17 print ( I )
18 # Output :
19 # [[1. 0.]
20 # [0. 1.]]
Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 6 / 12
 Matrix Operations (1/2)
1 A = np . array ([[1 , 2] , [3 , 4]])
2 B = np . array ([[5 , 6] , [7 , 8]])
3
4 # Matrix addition
5 print ( A + B )
6 # Output :
7 # [[ 6 8]
8 # [10 12]]
9
10 # Matrix multiplication
11 print ( np . dot (A , B ) )
12 # Output :
13 # [[19 22]
14 # [43 50]]
15
16 # Element - wise multiplication
17 print ( A * B )
18 # Output :
19 # [[ 5 12]
20 # [21 32]]
Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 7 / 12
 Matrix Operations (2/2)

1 # Transpose
2 print ( A . T )
3 # Output :
4 # [[1 3]
5 # [2 4]]
6
7 # Determinant
8 print ( np . linalg . det ( A ) ) # Output : -2.0
9
10 # Inverse
11 print ( np . linalg . inv ( A ) )
12 # Output :
13 # [[ -2. 1. ]
14 # [ 1.5 -0.5]]
15
16 # Eigenvalues and eigenvectors
17 eigenvalues , eigenvectors = np . linalg . eig ( A )
18 print ( " Eigenvalues : " , eigenvalues )
19 print ( " Eigenvectors : " , eigenvectors )

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 8 / 12

 Solving Linear Systems

Solve the system Ax = b:

1 A = np . array ([[1 , 2] , [3 , 4]])
2 b = np . array ([5 , 11])
3
4 x = np . linalg . solve (A , b )
5 print ( x ) # Output : [1. 2.]
6
7 # Verify the solution
8 print ( np . dot (A , x ) ) # Output : [5. 11.]

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 9 / 12

 Matrix Decompositions

LU Decomposition:
1 A = np . array ([[1 , 2] , [3 , 4]])
2 P , L , U = scipy . linalg . lu ( A )
3
4 print ( " P : " , P )
5 print ( " L : " , L )
6 print ( " U : " , U )

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 10 / 12

 Singular Value Decomposition (SVD)

1 A = np . array ([[1 , 2] , [3 , 4] , [5 , 6]])

2 U , s , Vt = np . linalg . svd ( A )
3
4 print ( " U : " , U )
5 print ( " s : " , s )
6 print ( " V ^ T : " , Vt )

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 11 / 12

Conclusion

Python with NumPy provides powerful tools for linear algebra

Easy to create and manipulate vectors and matrices
Efficient computation of complex operations
Useful for various applications: data science, machine learning,
scientific computing, etc.

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 12 / 12

Solving Systems of Equations in Python

NumPy and SciPy provide several methods for solving systems of

equations
We’ll explore:
Direct solution using NumPy
Least squares solution
Solving over-determined and under-determined systems
Iterative methods for large systems

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 13 / 12

 Direct Solution: 2x2 System

Solve the system:

2x + y = 5
3x + 2y = 8

1 import numpy as np
2
3 A = np . array ([[2 , 1] , [3 , 2]])
4 b = np . array ([5 , 8])
5
6 x = np . linalg . solve (A , b )
7 print ( " Solution : " , x )
8 # Output : Solution : [1. 3.]
9
10 # Verify the solution
11 print ( " Verification : " , np . allclose ( np . dot (A , x ) , b ) )
12 # Output : Verification : True

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 14 / 12

 Direct Solution: 3x3 System

Solve the system:

x + 2y + 3z = 14
2x − y + z = 4
3x + y − 2z = 2

1 A = np . array ([[1 , 2 , 3] , [2 , -1 , 1] , [3 , 1 , -2]])

2 b = np . array ([14 , 4 , 2])
3
4 x = np . linalg . solve (A , b )
5 print ( " Solution : " , x )
6 # Output : Solution : [1. 2. 3.]
7
8 print ( " Verification : " , np . allclose ( np . dot (A , x ) , b ) )
9 # Output : Verification : True

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 15 / 12

 Least Squares Solution

For over-determined systems (more equations than unknowns):

1 A = np . array ([[1 , 1] , [1 , 2] , [1 , 3]])
2 b = np . array ([2 , 4 , 5])
3
4 x , residuals , rank , s = np . linalg . lstsq (A , b , rcond = None )
5 print ( " Least squares solution : " , x )
6 # Output : Least squares solution : [1.07142857 1.32142857]
7
8 # Verify the solution
9 print ( " Verification : " , np . allclose ( np . dot (A , x ) , b ) )
10 # Output : Verification : False ( as expected for over -
determined systems )

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 16 / 12

 Under-determined System

For under-determined systems (fewer equations than unknowns):

1 A = np . array ([[1 , 2 , 3] , [4 , 5 , 6]])
2 b = np . array ([7 , 8])
3
4 # Using pseudo - inverse for a minimum norm solution
5 x = np . linalg . pinv ( A ) . dot ( b )
6 print ( " Solution : " , x )
7 # Output : Solution : [ -0.94444444 0.11111111 1.16666667]
8
9 print ( " Verification : " , np . allclose ( np . dot (A , x ) , b ) )
10 # Output : Verification : True

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 17 / 12

 Iterative Methods: Conjugate Gradient

For large, sparse systems:

1 from scipy . sparse import csc_matrix
2 from scipy . sparse . linalg import cg
3
4 A = csc_matrix ([[3 , 2 , 0] , [1 , -1 , 0] , [0 , 5 , 1]])
5 b = np . array ([2 , 4 , -1])
6
7 x , info = cg (A , b )
8 print ( " Solution : " , x )
9 # Output : Solution : [ 2. -2. 9.]
10
11 print ( " Verification : " , np . allclose ( A . dot ( x ) , b ) )
12 # Output : Verification : True

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 18 / 12

 Solving Nonlinear Systems

Using SciPy’s root finding:

1 from scipy . optimize import root
2
3 def equations ( vars ) :
4 x , y = vars
5 eq1 = x **2 + y **2 - 1
6 eq2 = x - y
7 return [ eq1 , eq2 ]
8
9 initial_guess = [1 , 1]
10 solution = root ( equations , initial_guess )
11
12 print ( " Solution : " , solution . x )
13 # Output : Solution : [ 0.70710678 0.70710678]

Ahmet Goncu Linear Algebra Operations in Python September 28, 2024 19 / 12