Face Recognition using PCA (Eigenfaces)

A Linear Algebra mini-project showing how eigenvalues, eigenvectors, orthogonal projection, and distance in vector spaces can be used for dimensionality reduction and face recognition.

Abstract

This project applies Principal Component Analysis (PCA) to face images from the Olivetti dataset. Each image is treated as a vector in a high-dimensional space, and PCA is used to find a lower-dimensional subspace that captures the most important facial variations. The basis vectors of this subspace are called eigenfaces.

The project demonstrates that core linear algebra ideas are not only theoretical. They directly support:

• dimensionality reduction
• image approximation
• data compression
• noise reduction
• pattern recognition

Objective

The goal is to show how linear algebra can solve a real problem:

1. Represent each face image as a vector.
2. Find the principal directions of variation using eigenvectors of the covariance matrix.
3. Project faces into a smaller subspace.
4. Compare projected faces using Euclidean distance for recognition.

Why This Fits a Linear Algebra Course

This project is a strong application of linear algebra because it uses:

• Vector spaces: each image is a point in R^4096
• Mean centering: shifts the dataset so PCA works around the origin
• Covariance matrices: capture how pixel values vary together
• Eigenvalues and eigenvectors: identify the most informative directions
• Orthonormal bases: the eigenfaces form a basis for the PCA subspace
• Projection: every face is mapped onto a lower-dimensional subspace
• Norms and distance: recognition uses Euclidean distance in eigenspace

Mathematical Model

1. Image as a Vector

Each grayscale face image has size 64 x 64, so it is flattened into a vector in R^4096.

2. Mean Face

If the training faces are x_1, x_2, ..., x_n, then the mean face is

mu = (1/n) sum x_i

3. Centered Data Matrix

We subtract the mean face from every image:

A = X - mu

Each row of A is a centered face vector.

4. Covariance Matrix and the PCA Trick

The full covariance matrix is

C = A^T A

Since this matrix is very large (4096 x 4096), the project uses the standard PCA trick:

C' = A A^T

This smaller matrix has size n x n, making the computation much more efficient.

5. Eigenfaces

If v is an eigenvector of A A^T, then A^T v gives an eigenvector of A^T A. These transformed eigenvectors become the eigenfaces.

The top eigenfaces correspond to the largest eigenvalues, which means they capture the most variance in the dataset.

6. Projection into the PCA Subspace

If U_k contains the top k eigenfaces, then the coordinates of a face x in eigenspace are:

w = U_k^T (x - mu)

This reduces a 4096-dimensional vector to only k values.

7. Reconstruction

A face can be approximated from its projection by:

x_hat = mu + sum (w_i * u_i)

or equivalently,

x_hat = mu + U_k w

8. Recognition

Recognition is performed by nearest-neighbor comparison in eigenspace:

distance = ||w_1 - w_2||_2

The training face with minimum distance to the test face is chosen as the predicted identity.

Real-World Applications Demonstrated

This project shows several applications of linear algebra:

• Compression: storing a face using k coefficients instead of 4096 raw pixels
• Approximation: reconstructing an image from a low-dimensional basis
• Feature extraction: using principal components as meaningful features
• Classification: comparing faces using distances in a reduced space
• Noise filtering: ignoring small variations not captured by leading components

Project Structure

miniproject/
├── README.md
├── requirements.txt
├── eigenfaces.py
├── fastapi_app.py
├── utils.py
├── input/
│   ├── heldout_1_george_w_bush.png
│   ├── heldout_2_colin_powell.png
│   ├── heldout_3_ariel_sharon.png
│   ├── heldout_4_junichiro_koizumi.png
│   ├── heldout_5_hugo_chavez.png
│   └── ...
└── output/
    ├── mean_face.png
    ├── eigenfaces.png
    ├── reconstruction.png
    ├── variance_capture.png
    ├── projection_breakdown.png
    └── accuracy_curve.png

Installation and Run

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python eigenfaces.py

To run the interactive FastAPI web app:

uvicorn fastapi_app:app --reload

Then open the local URL shown by Uvicorn, usually:

http://127.0.0.1:8000

The web app uses the larger LFW People dataset from scikit-learn instead of the smaller Olivetti dataset. It lets you upload a face image from the browser, converts it to grayscale, crops/resizes it to the LFW face shape, projects it into the eigenface subspace, and returns the closest matching LFW training-set person. The first run may download the LFW dataset if it is not already cached.

Sample upload images are available in the input/ folder. Open the web app, click Choose File, and select one of those PNG files. These examples are held-out test images, so the exact photos are not used to train the eigenface model, but the people are present in the known LFW classes.

Important limitation: if the uploaded person is not part of the LFW training set, the app still returns the closest dataset person. That means the result is a nearest mathematical match, not a guaranteed real identity.

What the Program Produces

When the demo runs, it:

1. Loads the Olivetti face dataset.
2. Splits it into training and testing sets.
3. Computes the mean face.
4. Computes eigenfaces using PCA.
5. Projects training faces into eigenspace.
6. Saves visualizations in the output/ folder.
7. Evaluates recognition accuracy for different numbers of components.
8. Prints linear algebra diagnostics for presentation.

Output Figures and Their Meaning

mean_face.png

Shows the average of all training faces. Geometrically, this is the center of the dataset in face space.

eigenfaces.png

Shows the principal basis vectors of the face subspace. These are the eigenvectors associated with the largest eigenvalues.

reconstruction.png

Shows how a face is reconstructed with different numbers of components. This demonstrates approximation in a subspace.

variance_capture.png

Shows cumulative explained variance. This justifies why a small number of eigenfaces is enough to preserve most information.

projection_breakdown.png

Shows:

• original face
• mean face
• projected reconstruction
• residual error

This is one of the best figures to explain orthogonal projection and approximation error.

accuracy_curve.png

Shows how recognition accuracy changes with the number of principal components.

Key Results

Typical results on the Olivetti dataset are:

• around 87% accuracy with 10 components
• around 94% accuracy with 50 components
• around 94% accuracy with 100 components
• about 94% explained variance with 100 components

These values show that a much smaller subspace can still preserve most useful facial information.

Presentation Flow

If you want a clear classroom explanation, present it in this order:

1. A face image is converted into a vector in R^4096.
2. The dataset is centered by subtracting the mean face.
3. PCA finds the most important directions of variation.
4. These directions become eigenfaces.
5. Each face is projected into a low-dimensional eigenspace.
6. Similar faces stay close together in this reduced space.
7. Reconstruction shows what information the subspace preserves.
8. Residual error shows what information is lost.

Short Viva Explanation

If someone asks, "Where is linear algebra used here?", a concise answer is:

Each face image is treated as a vector, PCA computes eigenvectors of the covariance structure, those eigenvectors form an orthogonal basis called eigenfaces, and each face is projected onto that basis for compression, reconstruction, and recognition using Euclidean distance.

Conclusion

This project demonstrates that linear algebra is a practical tool for data analysis and machine learning. Using eigenvalues, eigenvectors, covariance matrices, and subspace projection, it becomes possible to represent high-dimensional images compactly and perform accurate face recognition in a mathematically elegant way.

References

• Turk, M., and Pentland, A. "Eigenfaces for Recognition." Journal of Cognitive Neuroscience, 1991.
• Belhumeur, P. N., Hespanha, J. P., and Kriegman, D. J. "Eigenfaces vs. Fisherfaces." IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997.
• scikit-learn documentation for the Olivetti faces dataset.
• scikit-learn documentation for the LFW People faces dataset.

Author

Rohit Pujari
Linear Algebra Mini-Project, 4th Semester