Mathematical Foundation of ML

AI&DS
Sem VI
Machine Learning,
UNIT – II
By,
Dr. Himani Deshpande
Dr.Himani Deshpande (TSEC)
Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

System of Linear Equations

AI&DS
Sem VI
Machine Learning,
System of linear equation is a collection of
two or more linear equations using the same variables.

Dr.Himani Deshpande (TSEC)

System of Linear Equations

AI&DS
Sem VI
Machine Learning,
"Independent" means that each equation gives new information.
Otherwise they are "Dependent".

Also called "Linear Independence" and "Linear Dependence"

Dr.Himani Deshpande (TSEC)
 System of Linear Equations

• In mathematics, the system of linear equation is the set of two or more

linear equations involving the same variables.

AI&DS
• Here, linear equations can be defined as the equations of the first order,
i.e., the highest power of the variable is 1

Sem VI
• Linear equations can have one variable, two variables, or three variables.

Machine Learning,
Only simple variables are allowed in linear equations. No x2, y3, √x, etc:

Dr.Himani Deshpande (TSEC)

 System of Linear equations example
• It's a race!
• You can run 0.2 km every minute.
• The Horse can run 0.5 km every minute. But it takes 6 minutes to saddle the horse.
• How far can you get before the horse catches you?

AI&DS
Sem VI
d=distance in km, t=time in minutes

So we have a system of linear equations :

d = 0.2t

Machine Learning,
d = 0.5(t−6)

Do you see how the horse starts at 6 minutes, but then runs faster?
It seems you get caught after 10 minutes ...
you only got 2 km away.
Dr.Himani Deshpande (TSEC)
 Solving linear equations

AI&DS
Sem VI
Using Graphs
Matrix
(For 2
Method
variables)

Machine Learning,
Addition
Substitution or
Method Subtraction
Method

Dr.Himani Deshpande (TSEC)

Substitution Method
3x+2y=16

7x+y=19

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Substitution
method
3x+2y=16

7x+y=19

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Addition or Subtraction method
• 4x+3y=−2
• 8x−2y=12

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Addition or Subtraction method
• 4x+3y=−2
• 8x−2y=12

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Matrix Method
2x – y = 4
3x +2y = 13

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Matrix Method
2x – y = 4
3x +2y = 13

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Matrix Method
2x – y = 4
3x +2y = 13

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Practice Questions

AI&DS
Sem VI
Machine Learning,
Solution :

Dr.Himani Deshpande (TSEC)

Scalar

AI&DS
Sem VI
Machine Learning,
Scalar means magnitude

Dr.Himani Deshpande (TSEC)

Vectors

AI&DS
Sem VI
Machine Learning,
Vector is magnitude with direction

Dr.Himani Deshpande (TSEC)

Vector

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Vectors

AI&DS
Sem VI
Machine Learning,
A vector has magnitude (size) and direction

We can also think of vectors as representing a point in space.

Dr.Himani Deshpande (TSEC)

NORMS

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Identifying data
Given a dataset with age and hair length

AI&DS
• Identify gender

Sem VI
• Categorize as adult , teenager, child or infant.
• We need to understand the magnitude of data

Machine Learning,
We need to quantify each record in terms of age and hair length

Dr.Himani Deshpande (TSEC)

Norms

AI&DS
• It is defined as the length or magnitude of the vector.
• This concept of norms is important in Machine Learning and Deep

Sem VI
Learning.
• The norm is generally used to evaluate the error of the model.

Machine Learning,
Dr.Himani Deshpande (TSEC)
Vector Norms

AI&DS
• Vector Norms are defined as a set of functions that take a
vector as an input and output a positive value against it.

Sem VI
• This is called the magnitude of a vector.

Machine Learning,
Dr.Himani Deshpande (TSEC)
 Norms
Norms are any functions that are characterized by the following
properties:

AI&DS
• Vector Norms are non-negative values. If you think of the norms as a

Sem VI
length, you easily see why they can’t be negative.
• Vector Norms are 0 if and only if the vector is itself a zero vector.

Machine Learning,
• Norms follow the triangular inequality(The norm of the sum of some
vectors is less than or equal to the sum of the norms of these vectors).

Dr.Himani Deshpande (TSEC)

Norms

✓L1 Norm

AI&DS
✓L2 Norm

Sem VI
✓L3 Norm
✓…
✓….

Machine Learning,
✓ Max Norm

Dr.Himani Deshpande (TSEC)

 L1 Norm
• The length of a vector can be calculated using the L1 norm.
• The notation for the L1 norm of a vector is ||v||1 and this type of norm is sometimes
also referred to as Manhattan Norm(since this uses Manhattan distance ).

AI&DS
• The L1 norm is calculated as the sum of the absolute vector values(which means add
the absolute value of vector components in different directions) and the absolute

Sem VI
value of a scalar is representing using the notation as |x1|.
• Manhattan distance from the origin to the vector space.

Machine Learning,
||v||1 = |x1| + |x2| + |x3|
where xi is the components of a given vector in different directions.

The L1 norm of a vector can be calculated using the norm(arr, 1 ) function present in the Linear Algebra
module inside the Numpy library.
Dr.Himani Deshpande (TSEC)
L1 Norm example

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
 L2 or Euclidean Norm
• The notation for the L2 norm of a vector is ||v|| 2 and this type of norm is also
known as Euclidean Norm(since this uses Euclidean distance ).
• The L2 norm is calculated as the square root of the sum of the squared vector

AI&DS
values. So, this norm finds the distance of the vector coordinate from the
origin of the vector space.

Sem VI
||v||2= sqrt [ (x1)2 + (x2)2 + (x3)2 ]

Machine Learning,
• The L2 norm of a vector can be calculated using the norm( ) function with
default parameters(means default order in the norm function is 2).
• This type of norm is most commonly used in Machine Learning.

Dr.Himani Deshpande (TSEC)

L2 Norm example

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Squared L2 Norm

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Unit Vector ||v||2 = 1

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
 Vector Max Norm
• This norm is referred to as L ∞ and also can be represented with
the infinity symbol. The notation for max norm is ||v|| ∞ .

AI&DS
• The max norm is calculated as returning the max value of the
vector.

Sem VI
||v||∞ = max(|b1| , |b2| , |b3|)

Machine Learning,
The max norm of a vector can be calculated using the norm( )
function along with set the parameter as ∞ (infinite) which is a order
for norm).

Dr.Himani Deshpande (TSEC)

Max Norm

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Lp Norm (Generalized)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Norms

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Why Norms are important

AI&DS
For instance, it is used to calculate the error between the output of a
Neural network and what is expected(the actual value or label) or can

Sem VI
be used in defining a regularization term which includes the magnitude
of the weights, to encourage small weights

Machine Learning,
The norm is generally used to evaluate the error of the model.

Dr.Himani Deshpande (TSEC)

Inner or Dot Product of Vectors

• The Dot product is a way to multiply two equal-length

AI&DS
vectors together.

Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Dot Product

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Inner or Dot Product of Vectors

• Dot product measures how similar are two vectors.

AI&DS
Sem VI
Machine Learning,
• Finding Dot product of two vectors is one of the most common
Tensor operation in Machine learning(Deep learning).

Dr.Himani Deshpande (TSEC)

Dot product

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Vector Dot Product

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Vector reshape to facilitate dot product

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Dot Product example

AI&DS
Sem VI
Machine Learning,
= -3

Dr.Himani Deshpande (TSEC)

Orthogonal Vectors

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
DOT PRODUCT OF VECTORS

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Distance between Vectors

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Distance between Vectors

• Euclidean Distance N2 Norm

AI&DS
• Manhattan Distance N1 Norm

Sem VI
Manhattan Euclidean

Machine Learning,
• Minkowski Distance
• Hamming Distance

Dr.Himani Deshpande (TSEC)

 Euclidean Distance
Euclidean distance between the two points using Pythagora's Theorem
(c2 = a2 + b2).

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
 Manhattan Distance
ManhattanDistance [{a, b, c}, {x, y, z}]

= Abs [a − x] + Abs [b − y] + Abs [c − z]

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Hamming Distance
• Hamming Distance measures the similarity between two strings of the
same length. The Hamming Distance between two strings of the same
length is the number of positions at which the corresponding characters are

AI&DS
different.

Sem VI
“euclidean” and “manhattan” -->
Since the length of these strings is equal, we can calculate the Hamming Distance.

Machine Learning,
Here seven characters are different whereas two characters are
similar
Hamming Distance here will be 7.
Dr.Himani Deshpande (TSEC)
Minkowski Distance

• Minkowski Distance is the generalized form of Euclidean and

Manhattan Distance.

AI&DS
Sem VI
Machine Learning,
• Here, p represents the order of the norm.

Dr.Himani Deshpande (TSEC)

 Distances between Vectors
• Point_1 = (1, 2, 3)
point_2 = (4, 5, 6)

Euclidean Distance = ?

AI&DS
Sem VI
• Manhattan Distance = ?

Machine Learning,
• Minkowski distance(p =3) = ?

Dr.Himani Deshpande (TSEC)

 AI&DS
Determinant of a Matrix

Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Determinant of a Matrix

AI&DS
Sem VI
Machine Learning,
Determinant = (8x-3) – (15x 7)
Determinant of 2X2 matrix = - 24 - 105
= - 129

Dr.Himani Deshpande (TSEC)

Determinant of a Matrix

AI&DS
Sem VI
Machine Learning,
Determinant of 3X3 matrix

Dr.Himani Deshpande (TSEC)

Determinant (Example 1)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Solution Determinant (Example 2)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Solution Determinant (Example 2)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
EIGEN VECTORS

AI&DS
&

Sem VI
EIGENVALUES

Machine Learning,
Dr.Himani Deshpande (TSEC)
EIGEN VALUES
• The word, Eigen is perhaps most usefully translated from German
which means Characteristic.

AI&DS
• when we talk about Eigenvalues and eigenvectors of a Matrix, we’re
talking about finding the characteristics of the matrix.

Sem VI
• Eigenvalues are a special set of scalars associated with a

Machine Learning,
linear system of equations (i.e., a matrix equation) that are
sometimes also known as characteristic roots, characteristic
values, proper values, or latent roots.

Dr.Himani Deshpande (TSEC)

Eigenvalues & Vectors
• Eigenvalues are associated with eigenvectors in Linear
algebra.

AI&DS
• Both terms are used in the analysis of linear transformations.
• Eigenvalues are the special set of scalar values that is

Sem VI
associated with the set of linear equations most probably in the
matrix equations.

Machine Learning,
• the corresponding factor which scales the eigenvectors is
called an eigenvalue.

Dr.Himani Deshpande (TSEC)

 EIGEN VECTORS

For a square matrix A, an Eigenvector and Eigenvalue make this equation true:

AI&DS
Sem VI
Matrix A Eigen Vector

Machine Learning,
with a matching
eigenvalue of 6
Find the eigenvalues of the 2 x 2 matrix
Matrix A

AI&DS
Sem VI
Calculating that determinant gets:

Machine Learning,
(−6−λ)(5−λ) − 3×4 = 0
Simplifying Quadratic Equation
λ2 + λ − 42 = 0
Solving the Equation,
λ = −7 or 6
Dr.Himani Deshpande (TSEC)
 Find the Eigenvector for the
Eigenvalue λ = 6:

After multiplying we get these two equations:

AI&DS
−6x + 3y = 6x
4x + 5y = 6y

Sem VI
Bringing all to left hand side −12x + 3y = 0
4x − 1y = 0

Machine Learning,
Either equation reveals that y = 4x, so the eigenvector is any non-zero
multiple of

So, eigenvector is multiple of this:

Dr.Himani Deshpande (TSEC)

Eigenvalue and Vector
Av = λv

AI&DS
Sem VI
Machine Learning,
So Av = λv, and we have success!

Dr.Himani Deshpande (TSEC)

Eigenvalues of the 3 x 3 matrix

AI&DS
determinant should equal zero:

Sem VI
Machine Learning,
(2−λ) [ (4−λ)(3−λ) − 5×4 ] = 0

So the Eigenvalues are −1, 2 and 8

Dr.Himani Deshpande (TSEC)

Eigenvalues of the 3 x 3 matrix

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Eigenvalues of the 3 x 3 matrix (cont..)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Eigenvalues of the 3 x 3 matrix (cont..)
Hence,
det (A-λI) = (1)+(2)+(3)
det (A-λI) = -λ3 + 6λ2 – 6λ – 8 – 12 +18λ +20-20λ

AI&DS
=-λ3+6λ2-8λ+0

Sem VI
Therefore, the Eigenvalues of the matrix A can be found by
-λ3+6λ2-8λ =0
Now, multiply the above equation by (-1) on both sides, we get

Machine Learning,
λ3-6λ2+8λ =0
On factoring the above equation, we get
λ(λ2-6λ+8)=0

Dr.Himani Deshpande (TSEC)

Eigenvalues of the 3 x 3 matrix (cont..)
Thus,
λ= 0, and (λ2-6λ+8)=0
Use the quadratic equation formula to find the roots of the equation

AI&DS
(λ2-6λ+8)=0
Here, a=1, b=-6, c=8

Sem VI
Now, the values in the quadratic formula,
=6±36−4(1)(8)2(1)
=6±36−322

Machine Learning,
=6±22
Hence, λ= 2 and λ=4
Therefore, the Eigenvalues of matrix A are 0, 2, 4.

Dr.Himani Deshpande (TSEC)

Find the Eigen Vector

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Practice problems

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
 Application of Eigen Vector in ML
• Principal Component Analysis(PCA)
• PCA is a very popular classical dimensionality reduction technique which uses this
concept to compress your data by reducing its dimensionality

AI&DS
• Spectral Clustering

Sem VI
• Spectral clustering is a family of methods to find K clusters using the eigenvectors
of a matrix.
• Interest Point Detection in Computer Vision

Machine Learning,
• Corners are useful interest points along with other more complex image features
such as SIFT, SURF, and HOG, etc.

Dr.Himani Deshpande (TSEC)

Trace of Matrix
• In a square matrix, the sum of elements of the principal diagonal is
called the ‘trace of a matrix’. We denote the trace of the matrix A by
‘tr A’.

AI&DS
Sem VI
Then,
tr A = 1 + 6 + 5 = 12.

Machine Learning,
Dr.Himani Deshpande (TSEC)
 PDM
• A positive definite matrix is a special type of matrix with nice
properties — it's symmetric and all its eigenvalues are positive.

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
How to check or calculate whether a matrix is
positive definite:

1. Check All Eigenvalues > 0

•If all eigenvalues of A are positive, then A is positive definite.

AI&DS
Sem VI
2. Check All Leading Principal Minors > 0
• Compute determinants of top-left submatrices:
• det(A1)>0

Machine Learning,
• det(A2))>0
• ...
• det(AN) >0

• This is called Sylvester’s Criterion.

• Works for symmetric matrices only.
Dr.Himani Deshpande (TSEC)
How to check or calculate whether a
matrix is positive definite (Cont.)

3. Try a Cholesky Decomposition

AI&DS
Sem VI
• If you can compute a Cholesky decomposition (i.e. A=LLT , where
L is lower triangular), then A is positive definite.

Machine Learning,
• If it fails, the matrix is not positive definite.

Dr.Himani Deshpande (TSEC)

Positive Definite Matrix
A square matrix is positive definite if pre-multiplying and post-multiplying it by the same vector
always gives a positive number as a result, independently of how we choose the vector.

• A square matrix is positive definite if A is symmetric matrix and

AI&DS
any one of the following condition is true.

Sem VI
• All the eigen values > 0 i.e. +ve
OR

Machine Learning,
• All it’s pivotes are +ve
OR
• All upper left detereminants of order 1,2,….n of an nXn matrix A are +ve.

Dr.Himani Deshpande (TSEC)

Symmetric Matrix
• Symmetric

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Row-Echelon form for checking pivot
values

AI&DS
• CONDITION 1
• All rows of zero at bottom.

Sem VI
• CONDITION 2

Machine Learning,
• Staircase pattern of first non-zero entries in each row.

Dr.Himani Deshpande (TSEC)

Row-Echelon form
Condition 1
All rows of zero at bottom.

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
 Row-Echelon form
Condition 2
Staircase Pattern

AI&DS
Sem VI
Machine Learning,
REF REF
proper staircase patten Not REF proper staircase patten

Dr.Himani Deshpande (TSEC)

Pivots
• Pivots are the first non-zero element in each row of a matrix
that is in Row-Echelon form.

AI&DS
• Row-Echelon form of a matrix is the final resultant matrix of
Gaussian Elimination technique.

Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Upper left determinants of matrix

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Examples of Positive Definite Matrix
• Taken on board

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Diagonalization of a matrix
• The conversion of a matrix into diagonal form is
called diagonalization.
• The eigenvalues of a matrix are clearly represented by

AI&DS
diagonal matrices.

Sem VI
• A Diagonal Matrix is a square matrix in which all of the
elements are zero except the principal diagonal
elements.

Machine Learning,
• Numerical on Diagonalization

Dr.Himani Deshpande (TSEC)

Projection of a Vector

AI&DS
Sem VI
Machine Learning,
While Projecting a vector, the magnitude of vector doesn’t matter ,
we change the direction of the vector
Dr.Himani Deshpande (TSEC)
Orthogonal Projection

The notation for the orthogonal

AI&DS
projection of v onto u (i.e. onto the
line determined by u) is proju(v).

Sem VI
Machine Learning,
orthogonal — essentially the same thing as being perpendicular
but in multi-dimensional space

Dr.Himani Deshpande (TSEC)

SVD

• The Singular Value Decomposition (SVD) of a matrix is a

AI&DS
factorization of that matrix into three matrices.

Sem VI
• It has some interesting algebraic properties and conveys
important geometrical and theoretical insights about linear

Machine Learning,
transformations.

Dr.Himani Deshpande (TSEC)

SVD
SVD is a way to break down a big matrix (a grid of numbers) into
smaller, easier-to-understand parts.

AI&DS
• It’s kind of like taking a complex object apart to see how it

Sem VI
works—like disassembling a machine into gears, levers, and
screws.

Machine Learning,
Dr.Himani Deshpande (TSEC)
 SVD Explanation
features to identify …
1.The amount of chocolate on the inside
2.The curvature of the m&m

AI&DS
3.The size in mms of the m inscribed onto the m&m
4.microscopic dents and their position on the m&m

Sem VI
surface
What SVD is doing is trying 5.The shade of the particular color of the m&m
to identify a relevant sub-

Machine Learning,
space of all of the
dimensions created by the All the features
noise in the data. except for color
create noise in the
data.
Dr.Himani Deshpande (TSEC)
SVD Explanation

Singular value decomposition is nothing but

decomposing a given matrix of data to understand how

AI&DS
to best approximate the data to eliminate noise!

Sem VI
These “approximations” are calculated by the SVD

Machine Learning,
algorithm to form what are known as “singular vectors”
and “singular values.”

Dr.Himani Deshpande (TSEC)

SVD application
Let’s say you have a matrix A that represents a grayscale image:
• Rows = pixels down
• Columns = pixels across

AI&DS
• Values = brightness
Using SVD:

Sem VI
1.You decompose A into U, Σ, and VT
2.Keep only the top k singular values (biggest ones)
3.Reconstruct the image using just those parts

Machine Learning,
Result: compressed image, still looks good!

Dr.Himani Deshpande (TSEC)

Applications

• Data reduction (cutting down size without losing much info)

AI&DS
• Noise filtering

Sem VI
• Recommender systems (like Netflix or Spotify)

• Solving equations when there's no perfect solution

Machine Learning,
Dr.Himani Deshpande (TSEC)
 AI&DS
Sem VI
SVD can factories any matrix into a product of three matrices

Machine Learning,
Dr.Himani Deshpande (TSEC)
Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

U & V are rotations of Matrix A

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

SVD
• SVD of matrix M is M=UΣVᵗ
U and V are two matrices which are orthogonal is nature

AI&DS
Sem VI
Where,
M-is original matrix we want to decompose
U-is left singular matrix (columns are left singular vectors)

Machine Learning,
U columns contain eigenvectors of matrix MMᵗ
Σ-is a diagonal matrix containing singular (eigen)values
V-is right singular matrix (columns are right singular vectors).
V columns contain eigenvectors of matrix MᵗM

Dr.Himani Deshpande (TSEC)

SVD

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD

AI&DS
Sem VI
Machine Learning,
Unitary matrices have significant importance in quantum mechanics because they
preserve norms, and thus, probability amplitudes.

Dr.Himani Deshpande (TSEC)

Singular Value Decomposition (SVM)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Singular Value Decomposition (SVM)

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

 Find the SVD for the matrix A (On Board)
A = 3 1 1
-1 3 1

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD Numerical

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD Numerical

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD Numerical

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD Numerical

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

Dr.Himani Deshpande (TSEC)

Machine Learning, Sem VI AI&DS

SVD Applications

• As SVD helps in the perfect representation of any matrix and

AI&DS
it’s quite easy to eliminate data that is not that important in a
matrix for the production of low-dimensional approximation.

Sem VI
• That’s why its role becomes important in the application of

Machine Learning,
image compression and recommendation system

Dr.Himani Deshpande (TSEC)

SVD based recommendation systems

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD in recommendation system

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
SVM Movie recommendation

AI&DS
Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)
Application of SVD --> PCA
• Principal component analysis (PCA).
The objective is to reduce the dimensionality of a dataset in order to
plot the data or use it in machine learning algorithms that might not

AI&DS
work well with high-dimensional data. The idea here is that you

Sem VI
imagine that your data is drawn from a multivariate normal
distribution.

Machine Learning,
Dr.Himani Deshpande (TSEC)
Application of SVD --> Image Compression
• In SVD the image is decomposed into three elements by the Eigen
values of the image.

AI&DS
• This computation allows us to retain the important singular values that the

Sem VI
image requires while also releasing the values that are not as necessary in
retaining the quality of the image.

Machine Learning,
Dr.Himani Deshpande (TSEC)
SVD Applications
• Image compression.
• Market data analysis.

AI&DS
• Latent Semantic Indexing (LSI) for web document search

Sem VI
• Political spectrum analysis.
• 3D image deformation using moving least-squares.
• SVD and PCA for gene expression data.

Machine Learning,
Dr.Himani Deshpande (TSEC)
 AI&DS
END OF UNIT-II

Sem VI
Machine Learning,
Dr.Himani Deshpande (TSEC)