Scribd
Upload
26 views10 pages

Mean Vector and Covariance Calculation

The document outlines the process of calculating the mean vector and the covariance matrix for a set of feature vectors in a machine learning context. It details the steps to subtract the mean vector from the feature vectors and compute the eigenvalues and eigenvectors of the covariance matrix. The principal component is identified as the eigenvector corresponding to the greatest eigenvalue, indicating its significance in the dataset analysis.

Uploaded by

zeyad.amgad.za
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
26 views10 pages

Mean Vector and Covariance Calculation

The document outlines the process of calculating the mean vector and the covariance matrix for a set of feature vectors in a machine learning context. It details the steps to subtract the mean vector from the feature vectors and compute the eigenvalues and eigenvectors of the covariance matrix. The principal component is identified as the eigenvector corresponding to the greatest eigenvalue, indicating its significance in the dataset analysis.

Uploaded by

zeyad.amgad.za
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

FALL 2024

Intro To Machine
Learning

Note 8
Get data.
The given feature vectors are-
x1 = (2, 1)
x2 = (3, 5)
x3 = (4, 3)
x4 = (5, 6)
x5 = (6, 7)
x6 = (7, 8)

Calculate the mean vector (µ).

Mean vector (µ)


= ((2 + 3 + 4 + 5 + 6 + 7) / 6, (1 + 5 + 3 + 6 + 7 + 8) / 6)
= (4.5, 5)
Thus,
Subtract mean vector (µ) from the given feature vectors.

x1 µ = (2 4.5, 1 5) = (-2.5, -4)


x2 µ = (3 4.5, 5 5) = (-1.5, 0)
x3 µ = (4 4.5, 3 5) = (-0.5, -2)
x4 µ = (5 4.5, 6 5) = (0.5, 1)
x5 µ = (6 4.5, 7 5) = (1.5, 2)
x6 µ = (7 4.5, 8 5) = (2.5, 3)

Feature vectors (xi) after subtracting mean vector (µ) are-

Calculate the covariance matrix.


Covariance matrix is given by-
Now,

Covariance matrix
= (m1 + m2 + m3 + m4 + m5 + m6) / 6

On adding the above matrices and dividing by 6, we get-


Calculate the eigen values and eigen vectors of the covariance matrix.

So, we have-

From here,
(2.92 (3.67 x 3.67) = 0

16.56 2 13.47 = 0
2

1 2 = 0.38.

Clearly, the second eigen value is very small compared to the first eigen value.
So, the second eigen vector can be left out.

Eigen vector corresponding to the greatest eigen value is the principal component for the given data
set.
1.
We use the following equation to find the eigen vector-

where-
M = Covariance Matrix
X = Eigen vector

Solving these, we get-


2.92X1 + 3.67X2 = 8.22X1
3.67X1 + 5.67X2 = 8.22X2

On simplification, we get-
5.3X1 = 3.67X2
3.67X1 = 2.55X2

From (1) and (2), X1 = 0.69X2


From (2), the eigen vector is-
Thus, principal component for the given data set is-

You might also like