Pattern Recognition and Computer

Vision (ETCS-411)
Executive Summary and Strategic Examination
Analysis
This research report serves as a definitive, expert-level study resource for the "Pattern
Recognition and Computer Vision" (PRCV) course (Paper Code: ETCS-411/ML-411T),
specifically tailored for the end-semester examinations conducted by Guru Gobind Singh
Indraprastha University (IPU). The objective is to synthesize the provided syllabus , academic
notes, and extensive research materials into a singular, high-density document that functions as
both a theoretical textbook and a strategic exam guide.
The IPU examination for this subject typically carries 75 marks and assesses students on a
blend of algorithmic derivation, mathematical problem-solving, and conceptual application. The
syllabus is structured to transition from the logical foundations of machine learning (Unit I) to
statistical methodologies (Unit II), followed by pixel-level image processing (Unit III), and
culminating in high-level shape analysis (Unit IV).
A nuanced analysis of previous question trends and syllabus weightage suggests a distinct
hierarchy of importance. Derivations (e.g., Backpropagation, Fisher’s Linear Discriminant) and
algorithmic traces (e.g., Canny Edge Detection, ID3 Decision Tree construction) are high-yield
areas. This report addresses these areas with exhaustive detail, ensuring that students can
reproduce the necessary mathematical rigor required for full marks.

Unit I: Induction Algorithms and Classification

Frameworks
1.1 Strategic Unit Overview
Unit I introduces the paradigm of Supervised Learning, where the system learns a mapping from
inputs x to outputs y based on a labeled training set D = \{(x_1, y_1),..., (x_n, y_n)\}. In the
context of the IPU exam, this unit is fundamental. It frequently contributes 20-25% of the total
marks, often featuring a mix of numerical problems (Entropy calculation) and long-answer
theoretical questions (Neural Network derivation).
Exam Topic Ranking (Priority High to Low):
1.​ Decision Trees: Numerical problems on Entropy and Information Gain (ID3 Algorithm).
2.​ Neural Networks: Derivation of the Backpropagation algorithm.
3.​ Bayesian Methods: Naive Bayes Classifier (Categorical and Gaussian), Laplace
Smoothing.
4.​ Support Vector Machines (SVM): Concept of Margin (Hard vs. Soft).
5.​ Genetic Algorithms: Steps and biological analogy.
6.​ Rule Induction: Sequential Covering algorithms.
1.2 Induction Algorithms and Decision Trees
Induction is the process of reasoning from specific observed cases to general rules. Unlike
deduction, which guarantees truth if premises are true, induction provides probabilistic
guarantees.

1.2.1 Rule Induction and Sequential Covering

Rule induction algorithms extract classification models in the form of IF-THEN rules. These rules
are highly interpretable, making them valuable in domains where "explainability" is crucial (e.g.,
medical diagnosis).
The primary method for learning sets of rules is the Sequential Covering Algorithm. This is an
iterative process:
1.​ Learn-One-Rule: Select a class (e.g., "Positive"). Construct a single rule that covers as
many positive examples as possible while covering few or no negative examples.
2.​ Remove Data: Remove the positive examples covered by this rule from the training set.
3.​ Repeat: Repeat the process with the remaining data until all positive examples are
covered or a quality threshold is met.
This "separate and conquer" strategy differs from the "divide and conquer" strategy of decision
trees, as it removes data sequentially rather than splitting it recursively.

1.2.2 Decision Trees: The ID3 Algorithm

A decision tree represents a function that takes a vector of attribute values as input and returns
a "decision" (class label). The tree consists of internal nodes (attribute tests), branches (attribute
values), and leaf nodes (class labels).
The ID3 (Iterative Dichotomiser 3) algorithm constructs the tree top-down by selecting the
"best" attribute to split the data at each step. The metric for "best" is Information Gain, which
relies on the concept of Entropy.
Mathematical Foundation: Entropy (H) Entropy, in information theory, measures the impurity
or uncertainty of a dataset S. For a binary classification problem (Positives p_+, Negatives p_-),
the entropy is defined as:
●​ If the set is pure (e.g., 100% Positive), p_+=1, p_-=0, and Entropy = 0.
●​ If the set is maximally impure (50% Positive, 50% Negative), Entropy = 1.
Mathematical Foundation: Information Gain (IG) Information Gain measures the expected
reduction in entropy caused by partitioning the examples according to an attribute A.
Where S_v is the subset of S for which attribute A has value v. The algorithm calculates IG for
all available attributes and selects the one with the maximum value as the root of the current
subtree.
Comprehensive Numerical Example (Exam Style): Consider a dataset deciding whether to
"Play Tennis" (n=14: 9 Yes, 5 No). Attributes: Outlook (Sunny, Overcast, Rain), Wind (Weak,
Strong).
1.​ Step 1: Calculate System Entropy H(S)
2.​ Step 2: Calculate Gain for Attribute 'Wind'
○​ Weak Wind: 8 examples (6 Yes, 2 No).
○​ Strong Wind: 6 examples (3 Yes, 3 No).
○​ Weighted Average Entropy:
 ○​ Information Gain:
3.​ Step 3: Comparison You would repeat this for 'Outlook', 'Humidity', etc. If IG(S, Outlook)
= 0.246, then Outlook (0.246) > Wind (0.049), so Outlook is chosen as the root node.
Exam Tip: Always write the log values explicitly (e.g., \log_2(0.5) = -1). Showing the
step-by-step substitution is critical for partial marking if the final calculation is slightly off.

1.3 Bayesian Methods

1.3.1 Foundations and Naive Bayes Classifier

Bayesian classification is based on Bayes' Theorem, which provides a way to calculate the
probability of a hypothesis (class C) given prior knowledge and observed evidence (data X).
●​ Posterior P(C|X): Probability of class C given data X.
●​ Likelihood P(X|C): Probability of observing data X given that the class is C.
●​ Prior P(C): Initial probability of class C.
●​ Evidence P(X): Probability of data X occurring (normalization constant).
The Naive Bayes Classifier simplifies the computation of P(X|C) by assuming that all attributes
in X = (x_1, x_2,..., x_n) are conditionally independent given the class C.
The classification rule becomes:
This assumption is "naive" because features in real-world data (e.g., height and weight) are
often correlated. However, the classifier performs surprisingly well in practice.

1.3.2 Gaussian Naive Bayes for Continuous Attributes

Standard Naive Bayes works with categorical counts. For continuous attributes (e.g.,
Temperature = 25.3), we cannot use frequency tables. Instead, we assume the continuous
values for each class follow a Gaussian (Normal) Distribution.
For a feature x_i and class c, we calculate the mean \mu_c and variance \sigma_c^2 from the
training data. The likelihood is then computed using the Probability Density Function (PDF):
Exam Numerical Strategy: If given a table of continuous data (e.g., "Height"):
1.​ Separate data by class (e.g., Male heights, Female heights).
2.​ Calculate \mu_{male}, \sigma_{male}^2 and \mu_{female}, \sigma_{female}^2.
3.​ For a test instance (e.g., height = 6.0), plug 6.0 into both Gaussian equations to get
P(h=6.0|Male) and P(h=6.0|Female).
4.​ Multiply by priors and compare.

1.3.3 The Zero-Frequency Problem and Laplace Smoothing

If a specific feature value (e.g., "Outlook=Foggy") never appears in the training data for a
specific class (e.g., "Play=Yes"), then P(Foggy|Yes) = 0. Since Naive Bayes multiplies
probabilities, this zero annihilates the entire probability score, making the prediction 0
regardless of other strong evidence.
Solution: Laplace Correction (Smoothing) We add a small smoothing parameter \alpha
(usually 1) to the count of every feature value.
where k is the number of possible values the attribute can take. This ensures no probability is
ever strictly zero.
1.4 Neural Networks and Backpropagation
1.4.1 Architecture and Perceptrons

Neural networks are composed of layers of nodes (neurons). A single perceptron computes a
weighted sum of inputs plus a bias, passed through an activation function \phi:
Single perceptrons can only classify linearly separable data. For non-linear patterns, we use
Multi-Layer Perceptrons (MLP) with hidden layers.

1.4.2 The Backpropagation Algorithm (Deep Derivation)

Backpropagation is the gradient descent method used to train MLPs. It minimizes the Error
function E by iteratively adjusting weights. The derivation of the weight update rules is a critical
exam topic.
Setup:
●​ Error function (Sum of Squared Errors): E = \frac{1}{2} \sum_{k} (t_k - o_k)^2, where t_k is
target and o_k is actual output for unit k.
●​ Activation function (Sigmoid): \sigma(z) = \frac{1}{1+e^{-z}}. Derivative: \sigma'(z) =
\sigma(z)(1-\sigma(z)).
Goal: Find \Delta w_{ji}, the change in weight from unit i to unit j. Using the Chain Rule:
where \text{net}_j = \sum_i w_{ji} o_i. Since \frac{\partial \text{net}_j}{\partial w_{ji}} = o_i, we
focus on the error term \delta_j = - \frac{\partial E}{\partial \text{net}_j}.
Case 1: Output Layer Weights (w_{kj}) For an output unit k:
So, the error signal \delta_k is:
The weight update is: \Delta w_{kj} = \eta \delta_k o_j (where \eta is learning rate).
Case 2: Hidden Layer Weights (w_{ji}) For a hidden unit j, the error comes from all
downstream output units k connected to it.
The weight update is: \Delta w_{ji} = \eta \delta_j o_i.
Algorithm Summary:
1.​ Forward Pass: Compute outputs o_j and o_k for all layers.
2.​ Backward Pass: Compute \delta_k for output layer.
3.​ Backpropagate: Compute \delta_j for hidden layers using \delta_k.
4.​ Update: Adjust weights w_{new} = w_{old} + \Delta w..

1.5 Support Vector Machines (SVM)

SVM is a supervised learning model that classifies data by finding the optimal hyperplane that
maximizes the margin between classes.
●​ Support Vectors: The data points closest to the hyperplane. These are the "difficult"
points that define the boundary.
●​ Hard Margin SVM: Assumes data is perfectly linearly separable. It solves the
optimization problem: This is fragile; one outlier can prevent convergence or drastically
skew the boundary.
●​ Soft Margin SVM: Introduces slack variables \xi_i \ge 0 to allow some misclassification
(data points can be inside the margin or on the wrong side). The objective changes to: C
is the regularization parameter.
○​ High C: Penalizes errors heavily (approximates Hard Margin). Risk of overfitting.
 ○​ Low C: Allows more errors for a wider margin. Smoother boundary.

1.6 Genetic Algorithms (GA)

GAs are adaptive heuristic search algorithms based on the evolutionary ideas of natural
selection. They are robust for finding global optima in complex spaces.
Key Steps:
1.​ Population Initialization: Randomly generate a set of candidate solutions
(chromosomes).
2.​ Fitness Evaluation: Test each solution against the problem (e.g., classification accuracy)
and assign a fitness score.
3.​ Selection: Choose parents for the next generation. Fitter individuals have a higher
probability of selection (Roulette Wheel Selection).
4.​ Crossover (Recombination): Combine genetic material from two parents to create
offspring (e.g., swap bits at a random crossover point). This exploits known good partial
solutions.
5.​ Mutation: Randomly flip bits in the offspring's chromosome. This maintains diversity and
prevents premature convergence to local optima.
6.​ Termination: Repeat until convergence or max generations.

Unit II: Statistical Pattern Recognition

2.1 Strategic Unit Overview
Unit II shifts from algorithmic rules to statistical density estimation. This unit requires a strong
grasp of probability theory and linear algebra. The "Curse of Dimensionality" and "Fisher's
Linear Discriminant" are the cornerstones of this section.
Exam Topic Ranking (Priority High to Low):
1.​ Linear Discriminant Analysis (LDA): Derivation of Fisher's Criterion and the projection
vector.
2.​ Parameter Estimation: Maximum Likelihood (MLE) vs. Bayesian Estimation. Sequential
Estimation derivation.
3.​ Dimensionality: Curse of Dimensionality, Overfitting, Polynomial Curve Fitting.
4.​ Parzen Windows: Non-parametric density estimation.

2.2 The Curse of Dimensionality and Model Complexity

2.2.1 The Curse of Dimensionality

This term refers to the various phenomena that arise when analyzing data in high-dimensional
spaces (hundreds or thousands of features).
●​ Sparsity: As dimensions increase, the volume of the space increases exponentially.
Available data becomes sparse. To maintain the same data density as in 1D, the number
of data points needed grows exponentially with dimensions.
●​ Distance Concentration: In high dimensions, the difference between the maximum and
minimum distance to any point tends to zero. Euclidean distance becomes meaningless,
which breaks algorithms like k-NN and K-Means.
 ●​ Hughes Effect: For a fixed sample size, adding more features initially improves classifier
performance, but beyond a peak, performance degrades due to estimation errors.

2.2.2 Polynomial Curve Fitting and Overfitting

This concept illustrates the trade-off between bias and variance.

●​ Underfitting (High Bias): A model (e.g., linear) is too simple to capture the data
structure.
●​ Overfitting (High Variance): A model (e.g., high-degree polynomial) fits the training data
perfectly, including the noise. It oscillates wildly and fails to generalize to new data.
●​ Regularization: A technique to control overfitting by adding a penalty term to the error
function (e.g., \lambda ||w||^2, Ridge Regression). This forces the model to keep
coefficients small, resulting in a smoother curve.

2.3 Linear Discriminant Analysis (Fisher's LDA)

Fisher's LDA is a supervised dimensionality reduction technique. Unlike PCA, which maximizes
total variance (unsupervised), LDA maximizes class separability.
Derivation of Fisher's Linear Discriminant (Critical Exam Question): We seek a projection
vector w that transforms a d-dimensional input x into a scalar y = w^T x. We want to choose w
so that the projected class means are far apart, and the projected class variances are small.
Step 1: Definitions Let m_1, m_2 be the mean vectors of two classes in the original space.
Projected means: \tilde{m}_1 = w^T m_1, \quad \tilde{m}_2 = w^T m_2. Distance between
projected means: |\tilde{m}_1 - \tilde{m}_2|^2.
Step 2: Scatter Matrices
●​ Within-Class Scatter Matrix (S_W): Represents the spread of data around their
respective class means. The projected within-class variance is proportional to w^T S_W
w.
●​ Between-Class Scatter Matrix (S_B): Represents the spread between the class means.
The projected between-class variance is proportional to w^T S_B w.
Step 3: Fisher Criterion (J(w)) We define the objective function J(w) as the ratio of
between-class variance to within-class variance:
Step 4: Maximization To maximize J(w), we take the derivative with respect to w and set it to
zero. Using matrix calculus properties, the solution is the eigenvector corresponding to the
largest eigenvalue of the generalized eigenvalue problem:
Since S_B w is always in the direction of (m_1 - m_2), the optimal projection vector is:
This vector w defines the line that best separates the two classes.

2.4 Parameter Estimation

Pattern recognition often involves estimating the parameters (e.g., mean \mu, variance
\sigma^2) of a probability density function from data.

2.4.1 Maximum Likelihood Estimation (MLE) vs. Bayesian Estimation

●​ MLE: Assumes the parameter \theta is a fixed, unknown constant. We find the \theta that
maximizes the likelihood function L(\theta) = P(Data|\theta). It uses only the data.
●​ Bayesian Estimation: Assumes \theta is a random variable with a prior distribution
 P(\theta). We use Bayes' theorem to compute the posterior distribution P(\theta|Data).
This incorporates prior knowledge (e.g., "the mean is likely close to 0").
Table: MLE vs. Bayesian Estimation
Feature Maximum Likelihood (MLE) Bayesian Estimation
View of \theta Fixed, unknown constant Random variable
Input Data only Data + Prior Belief
Output Single point estimate Probability distribution
Sample Size Needs large data for accuracy robust with small data (if prior is
good)
Complexity Optimization problem (simple) Integration problem (complex)
2.4.2 Sequential Parameter Estimation

In online learning, data points x_1, x_2,... arrive one by one. We want to update our estimate of
the mean \mu without storing all previous data points.
Recursive Mean Formula Derivation: The sample mean of n points is \mu_n = \frac{1}{n}
\sum_{i=1}^n x_i. Separate the last term:
Since \mu_{n-1} = \frac{1}{n-1} \sum_{i=1}^{n-1} x_i, we substitute \sum x_i = (n-1)\mu_{n-1}:
This formula updates the mean using only the old mean, the new data point, and the count n.
This is the foundation of adaptive filters like the Kalman Filter.

Unit III: Image Processing Techniques

3.1 Strategic Unit Overview
Unit III moves from abstract data to visual data (images). This unit is heavily algorithmic.
Questions frequently ask for "steps of algorithm X" or "compare X and Y".
Exam Topic Ranking (Priority High to Low):
1.​ Canny Edge Detection: The 5-step process (memorize this).
2.​ Thresholding: Otsu's Method (Variance calculation).
3.​ Hough Transform: Line detection logic (Accumulator array).
4.​ Morphology: Basic operations (Dilation/Erosion) and applications (Thinning).
5.​ Texture: GLCM features.

3.2 Edge Detection and The Canny Algorithm

The Canny Edge Detector is considered the optimal edge detector because it satisfies three
criteria: good detection (low error), good localization (edges are exactly where they should be),
and minimal response (one edge is detected only once).
The 5-Step Canny Algorithm :
1.​ Noise Reduction (Gaussian Smoothing): Since edge detection relies on derivatives, it
is highly sensitive to noise. The image is convolved with a Gaussian kernel (typically 5
\times 5) to smooth it.
2.​ Gradient Calculation: Apply Sobel filters to find the intensity gradient in horizontal (G_x)
and vertical (G_y) directions.
○​ Magnitude: G = \sqrt{G_x^2 + G_y^2}
○​ Direction: \theta = \tan^{-1}(G_y / G_x) (rounded to 0, 45, 90, 135 degrees).
3.​ Non-Maximum Suppression (NMS): The gradient magnitude produces "thick" edges.
 NMS thins them. For each pixel, compare its gradient magnitude to its neighbors in the
direction of the gradient \theta. If the pixel is not the local maximum, set it to zero. This
results in single-pixel wide edges.
4.​ Double Thresholding: Classify pixels into three categories using two thresholds,
T_{high} and T_{low}:
○​ Strong Edge: G > T_{high}.
○​ Weak Edge: T_{low} \le G \le T_{high}.
○​ Non-Edge: G < T_{low} (discarded).
5.​ Edge Tracking by Hysteresis: The final step transforms weak edges. A weak edge is
preserved only if it is connected (8-connectivity) to a strong edge. This allows faint parts
of a true contour to be kept while isolated noise (which forms unconnected weak edges) is
removed.

3.3 Otsu's Method for Thresholding

Thresholding converts a grayscale image into a binary image. Otsu's Method is an
unsupervised, non-parametric algorithm that automatically finds the optimal threshold T. It
assumes the image histogram is bi-modal (foreground and background).
Concept: Otsu's method searches for the threshold t that minimizes the Within-Class Variance
(\sigma_w^2) of the two classes (black and white pixels). Mathematically, this is equivalent to
maximizing the Between-Class Variance (\sigma_B^2).
Algorithm Steps:
1.​ Compute the histogram and probabilities p_i of each intensity level i.
2.​ Iterate through all possible thresholds t = 0 to 255.
3.​ Update weight \omega_b(t) (probability of background) and \omega_f(t) (probability of
foreground).
4.​ Update mean \mu_b(t) and \mu_f(t).
5.​ Calculate Between-Class Variance:
6.​ The optimal threshold T is the t that yields the maximum \sigma_B^2.

3.4 Mathematical Morphology

Morphology is a tool for extracting image components that are useful in the representation and
description of region shape. It uses a Structuring Element (SE), a small matrix (like a kernel) to
probe the image.
Fundamental Operations :
Operation Symbol Description Mathematical Def. Application
Dilation A \oplus B Expands/thickens ${z (\hat{B})_z \cap A
objects. A pixel is \neq \emptyset}$
1 if any pixel under
SE is 1.
Erosion A \ominus B Shrinks/thins ${z (B)_z \subseteq
objects. A pixel is A}$
1 if all pixels under
SE are 1.
Opening A \circ B Erosion followed (A \ominus B) Smoothes
by Dilation. \oplus B contours, breaks
Operation Symbol Description Mathematical Def. Application
narrow isthmuses.
Closing A \bullet B Dilation followed (A \oplus B) Fills small
by Erosion. \ominus B holes/gaps without
expanding size.
Thinning: Thinning reduces a shape to a skeleton (1-pixel width) while preserving the
topological structure (holes and connectivity). It is achieved by iteratively applying a sequence of
morphological "Hit-or-Miss" transforms until convergence.

3.5 Hough Transform

The Hough Transform is a feature extraction technique used to detect simple shapes (lines,
circles) in an image.
Line Detection Logic: In the image space (x,y), a line is y = mx + c. However, vertical lines
have infinite slope m. Hough uses the polar representation:
●​ \rho: Perpendicular distance from origin to the line.
●​ \theta: Angle of the normal.
Algorithm:
1.​ Edge Detection: Apply Canny to get edge pixels.
2.​ Accumulator: Create a 2D array (Accumulator) with axes \rho and \theta, initialized to 0.
3.​ Voting: For every edge pixel (x_i, y_i), iterate through all \theta. Calculate \rho. Increment
the Accumulator cell (\rho, \theta).
○​ This maps a point in image space to a curve in Hough space.
4.​ Peak Finding: Find local maxima in the Accumulator. A peak at (\rho', \theta') indicates a
line exists with those parameters. High votes mean many edge pixels lie on that line.

Unit IV: Shape Analysis and Description

4.1 Strategic Unit Overview
Unit IV deals with "post-processing." After we have edges or regions (from Unit III), how do we
describe them to a computer? Is it a circle? Is it a letter 'A'?
Exam Topic Ranking (Priority High to Low):
1.​ Connected Component Labeling: The 2-pass algorithm steps and trace.
2.​ Active Contours (Snakes): Energy minimization concept.
3.​ Shape Descriptors: Chain Codes, Hu Moments (Invariance), Fourier Descriptors.
4.​ Distance Transforms: Algorithm and utility.

4.2 Connected Component Labeling (CCL)

CCL scans a binary image and groups pixels into connected regions (blobs), assigning a unique
label to each.
The Two-Pass Algorithm :
●​ Pass 1 (Labeling & Equivalence): Iterate through pixels. For a foreground pixel:
○​ Check neighbors (Top and Left).
○​ If no neighbors are labeled: Assign a new label.
○​ If one neighbor is labeled: Copy that label.
 ○​ If neighbors have different labels (e.g., Top=3, Left=5): Assign the smaller label (3)
and record an Equivalence (3 is equivalent to 5) in a table. This handles U-shaped
objects where two arms meet.
●​ Pass 2 (Resolving): Iterate through the image again. Check the equivalence table.
Replace every label with its "root" equivalent label (e.g., replace all 5s with 3s). Now, each
object has exactly one unique label.

4.3 Deformable Models: Active Contours (Snakes)

A "Snake" is an energy-minimizing spline guided by external constraint forces and influenced by
image forces. It "slithers" from an initial position to lock onto nearby edges.
Energy Functional:
1.​ Internal Energy (E_{int}): Determines the physical properties of the snake.
○​ Elasticity (\alpha): Penalizes stretching. Keeps the snake continuous (like a rubber
band).
○​ Stiffness (\beta): Penalizes bending. Keeps the snake smooth (like a thin metal
rod).
2.​ External Energy (E_{ext}): Derived from the image I(x,y).
○​ The snake seeks areas of high gradient (edges). We use the negative magnitude so
that edges represent "valleys" of low energy. The snake slides downhill into these
valleys.

4.4 Shape Descriptors

4.4.1 Chain Codes

A boundary representation that stores the direction of traversal rather than coordinates.
●​ 4-Connectivity: Directions 0 (East), 1 (North), 2 (West), 3 (South).
●​ 8-Connectivity: Adds diagonals (0 to 7).
●​ Normalization: To make the code invariant to starting point, we treat the sequence as a
circular number and find the minimum integer permutation. To make it invariant to rotation,
we use the first difference (derivative) of the chain code.

4.4.2 Hu Moments (Moment Invariants)

Moments are statistical averages of image intensity.

●​ m_{00} is the Area.
●​ (m_{10}/m_{00}, m_{01}/m_{00}) is the Centroid.
Hu Moments are a set of 7 non-linear combinations of normalized central moments. They are
famous for being invariant to:
1.​ Translation (Position)
2.​ Scale (Size)
3.​ Rotation (Orientation) The 7th moment is skew-invariant, useful for distinguishing
mirrored shapes.

Exam Strategy and Differentiation Tables

Q: Differentiate between Supervised and Unsupervised Learning.
Feature Supervised Learning Unsupervised Learning
Data Labeled (X, Y) Unlabeled (X)
Goal Prediction / Classification Pattern Discovery / Clustering
Feedback Explicit (Error between target Implicit (Internal
and output) consistency/structure)
Complexity Generally simpler training Computationally complex
Examples SVM, Decision Trees, K-Means, PCA, Otsu's Method
Backprop
Q: Parametric vs. Non-Parametric Methods.
Feature Parametric Methods Non-Parametric Methods
Assumptions Assumes data follows a specific No strong assumptions about
distribution (e.g., Gaussian) data distribution
Parameters Fixed set (\mu, \sigma) Parameters grow with dataset
independent of dataset size size
Efficiency Fast training and inference Slow inference (lazy learning)
Data Need Works well with smaller data Requires large data to
generalize
Examples LDA, Gaussian Naive Bayes k-NN, Parzen Windows,
Decision Trees
This report covers the theoretical breadth and practical depth required for the IPU PRCV
examination. Focus on the derivations in Units I and II, and the algorithmic steps in Units III and
IV for maximum scoring potential.

Works cited

1. Pattern Recogintion and Computer Vision | PDF | Applied Mathematics | Algorithms - Scribd,
https://www.scribd.com/document/863270405/Pattern-Recogintion-and-Computer-Vision 2.
SCHEME OF EXAMINATION & SYLLABI - Guru Gobind Singh Indraprastha University,
http://www.ipu.ac.in/Pubinfo2024/syllbtechcs2023aff.pdf 3. Automatically Evolving Rule
Induction Algorithms - UFMG,
https://homepages.dcc.ufmg.br/~glpappa/papers/Pappaetal-2006-PKDD.pdf 4. Learning Sets of
Rules, http://www.cs.cmu.edu/afs/cs/project/theo-20/www/mlbook/ch10.pdf 5. Sequential
Covering Algorithm - GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/sequential-covering-algorithm/ 6. Decision
Tree for classification using ID3 algorithm with example - Gyan Sanchay,
https://gyansanchay.csjmu.ac.in/wp-content/uploads/2022/02/Decision-Tree-by-using-ID3-algorit
hm.pdf 7. Iterative Dichotomiser 3 (ID3) Algorithm From Scratch - GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/iterative-dichotomiser-3-id3-algorithm-from-scr
atch/ 8. Decision Trees Example Problem,
https://www.cs.cmu.edu/~aarti/Class/10315_Fall20/recs/DecisionTreesBoostingExampleProble
m.pdf 9. Decision Trees Explained - Entropy, Information Gain, Gini Index, CCP Pruning..,
https://towardsdatascience.com/decision-trees-explained-entropy-information-gain-gini-index-cc
p-pruning-4d78070db36c/ 10. A Step by Step ID3 Decision Tree Example - Sefik Ilkin Serengil,
https://sefiks.com/2017/11/20/a-step-by-step-id3-decision-tree-example/ 11. 1. Decision Tree |
ID3 Algorithm | Solved Numerical Example | by Mahesh Huddar,
https://www.youtube.com/watch?v=coOTEc-0OGw 12. Naive Bayes Classifier Explained With
Practical Problems - Analytics Vidhya,
https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/ 13. Naive Bayes
classifier - Wikipedia, https://en.wikipedia.org/wiki/Naive_Bayes_classifier 14. Gaussian Naive
Bayes - GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/gaussian-naive-bayes/ 15. Gaussian Naive
Bayes, Clearly Explained!!! - YouTube, https://www.youtube.com/watch?v=H3EjCKtlVog 16.
Pattern Recognition and Computer Vision Unit-1 | PDF | Bayesian Inference - Scribd,
https://www.scribd.com/document/822006386/Pattern-Recognition-and-Computer-Vision-Unit-1
17. Mastering Backpropagation: A Comprehensive Guide for Neural Networks - DataCamp,
https://www.datacamp.com/tutorial/mastering-backpropagation 18. A Step by Step
Backpropagation Example - Matt Mazur,
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ 19. Derivation of
Backpropagation, https://www.cs.swarthmore.edu/~meeden/cs81/s10/BackPropDeriv.pdf 20.
Using a Hard Margin vs Soft Margin in SVM - GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/using-a-hard-margin-vs-soft-margin-in-svm/
21. Support Vector Machines: From Hard Margin to Soft Margin - DEV Community,
https://dev.to/harsimranjit_singh_0133dc/support-vector-machines-from-hard-margin-to-soft-mar
gin-1bj1 22. Using a Hard Margin vs. Soft Margin in SVM | Baeldung on Computer Science,
https://www.baeldung.com/cs/svm-hard-margin-vs-soft-margin 23. Pattern recognition Using
Genetic Algorithm - SciSpace,
https://scispace.com/pdf/pattern-recognition-using-genetic-algorithm-4ej0ws7qys.pdf 24.
Genetic Algorithms in Pattern Recognition | by Ravindrababu T | inspiringbrilliance | Medium,
https://medium.com/inspiredbrilliance/genetic-algorithms-in-pattern-recognition-574a7292cae0
25. The Curse of Dimensionality in Machine Learning: Challenges, Impacts, and Solutions,
https://www.datacamp.com/blog/curse-of-dimensionality-machine-learning 26. ML Series: Day
50 — Curse of Dimensionality | by Ebrahim Mousavi - Medium,
https://medium.com/@ebimsv/ml-series-day-50-curse-of-dimensionality-463f250fc8a4 27.
Underfitting vs. Overfitting - Scikit-learn,
https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html
28. Introduction - Polynomial Curve Fitting-Amit Rajan Blog,
https://amitrajan012.github.io/post/pattern-recognition-chapter-1-introduction_1/ 29. What Is
Linear Discriminant Analysis? - IBM,
https://www.ibm.com/think/topics/linear-discriminant-analysis 30. Lecture 8 - Computer Science,
UWO, https://www.csd.uwo.ca/~oveksler/Courses/CS434a_541a/Lecture8.pdf 31. Fisher's
Linear Discriminant Classification,
https://cseweb.ucsd.edu/~x4liu/notebooks/fisher-discriminant.html 32. Fisher Linear
Discriminant Analysis, https://statweb.rutgers.edu/yhung/HW586/LDA/Fisher-LDA.pdf 33.
Maximum likelihood estimation - Wikipedia,
https://en.wikipedia.org/wiki/Maximum_likelihood_estimation 34. An Overview of Sequential
Monte Carlo Methods for Parameter Estimation in General State-Space Models,
https://wwwf.imperial.ac.uk/~nkantas/sysid09_final_normal_format.pdf 35. Direct MLE vs
Estimating a parameter and then doing MLE - Cross Validated,
https://stats.stackexchange.com/questions/342006/direct-mle-vs-estimating-a-parameter-and-th
en-doing-mle 36. A Bayesian Adaptive Ensemble Kalman Filter for Sequential State and
Parameter Estimation in - AMS Journals,
https://journals.ametsoc.org/view/journals/mwre/146/1/mwr-d-16-0427.1.xml 37. Sequential
Estimation - GeeksforGeeks,
https://www.geeksforgeeks.org/artificial-intelligence/sequential-estimation/ 38. Canny in detail:.
Edge detection is a crucial process in… | by Helen K Joy - Medium,
https://medium.com/@helenjoy88/canny-in-detail-b7015fa9e688 39. Canny edge detector -
Wikipedia, https://en.wikipedia.org/wiki/Canny_edge_detector 40. Canny Edge Detector - Justin
Liang, https://justin-liang.com/tutorials/canny/ 41. Understanding Otsu's Method for Image
Segmentation | Baeldung on Computer Science,
https://www.baeldung.com/cs/otsu-segmentation 42. Otsu's method for image thresholding
explained and implemented - Muthukrishnan,
https://muthu.co/otsus-method-for-image-thresholding-explained-and-implemented/ 43. Otsu
Thresholding - The Lab Book Pages,
http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html 44. Binary Image
Analysis, https://courses.cs.washington.edu/courses/cse576/19sp/notes/Basics1_white.pdf 45.
Types of Morphological Operations - MATLAB & Simulink - MathWorks,
https://www.mathworks.com/help/images/morphological-dilation-and-erosion.html 46.
Morphology - Thinning - People @EECS,
https://people.eecs.berkeley.edu/~fateman/294-9/lectures/thin.html 47. Hough Transform:
Visualizing Line Detection Step by Step | PythonAlchemist,
https://www.pythonalchemist.com/blog/hough-transform-line-detection 48. Hough transform -
Wikipedia, https://en.wikipedia.org/wiki/Hough_transform 49. Connected-component labeling -
Wikipedia, https://en.wikipedia.org/wiki/Connected-component_labeling 50. Implementing a
Connected Component Labeling algorithm from scratch | by Xavier Weber,
https://medium.com/data-science/implementing-a-connected-component-labeling-algorithm-from
-scratch-94e1636554f 51. Snakes: Active Contour Models - UCLA Computer Science
Department, https://web.cs.ucla.edu/~dt/papers/ijcv88/ijcv88.pdf 52. Active contour models
(snakes), https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/YOUNG/vision7.html
53. Connectivity - NI - National Instruments,
https://www.ni.com/docs/en-US/bundle/ni-vision/page/connectivity.html 54. Chapter 11
Representation & Description,
https://www.ee.nthu.edu.tw/clhuang/09420EE368000DIP/chapter11.pdf 55. An Empirical
Analysis of Invariance Hu's Moment Feature over a Digital Image - IEEE Xplore,
https://ieeexplore.ieee.org/document/10522255/ 56. Shape Matching using Hu Moments (C++ /
Python) | LearnOpenCV, https://learnopencv.com/shape-matching-using-hu-moments-c-python/
57. Difference between Supervised and Unsupervised Learning - GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/difference-between-supervised-and-unsupervi
sed-learning/ 58. Difference between Parametric and Non-Parametric Methods -
GeeksforGeeks,
https://www.geeksforgeeks.org/machine-learning/difference-between-parametric-and-non-param
etric-methods/