Clustering
Machine
Learning
Algorithms
Outline
01 ML Categories 03 K-Means
02 What is clustering? 04 Hierarchical Clustering
Machine Learning Catgeories
(1) Supervised
● Used to train machines using
labeled data
● Takes labeled inputs and maps it
to known outputs (you already Classification
know the target variable)
&
Regression Problems
Machine Learning Catgeories
Clustering
&
(2) Unsupervised Association Problems
● Uses unlabeled data to discover
patterns and features in the data
● Understands patterns and trends in
the data and discovers the output
Machine Learning Catgeories
Reward Based
(3) Reinforcement Problems
● Uses an agent and an environment
to produce actions and rewards
● Follows trial and error method to
arrive at final solution
● Agent receives award after
finishing task
Clustering
Grouping similar objects together
Clusters
According to some predefined similarity or dissimilarity measures
Clustering Methods
01 02
Partitional Hierarchical
Partitional Clustering
Database
‘k’
partitions
‘n’ Objects of data
Satisfying: Process:
- Each group contains at least one object - Create Initial partitioning
- Each object belongs to exactly one cluster - Use an iterative relocation technique to improve
partitioning
K-Means
K-Means
Stop Condition
- Define a maximum number of iterations
- Inertia doesn’t decrease or only
decreases insignificantly
(Inertia is the sum of squared distances. It
keeps decreasing throughout the iterations,
thus improving the data compactness)
K-Means
Avoiding Local Optima
By minimizing initialization bias:
We perform 10 different random
initializations and run the K-Means
algorithm and calculate inertia.
We then choose the best option
(lowest inertia)
This can be performed in R language
via n_init parameter
K-Means
Advantages
Fast
Can serve as a data reduction
technique
K-Means
Disadvantages
It has a tendency to identify clusters
with same size ad volume (spherical
shapes)
Unable to identify elongated or non-
convex clusters
K-Means
Practical Use
Text Mining
Predictive Marketing
Clustering Methods
01 02
Partitional Hierarchical
Hierarchical Clustering
Database
Dendrogram
‘n’ Objects
Hierarchical Clustering
Hierarchical Clustering
Top-Bottom
Clustering continues until small
groups of similar clusters are
obtained
Hierarchical Clustering
Bottom-Up
Clustering continues until a
single cluster is obtained
Hierarchical Clustering
Algorithm
Step1: Consider every data point as
an individual cluster
Step 2: Calculate Proximity Matrix
for each cluster
Step 3:Merge the clusters which are
highly similar or close to each other.
Step 4: Recalculate the proximity
matrix for each cluster
Step 5: Repeat Steps 3 and 4 until
only a single cluster remains.
Hierarchical Clustering
Linkage Methods
A B C D E
A 0
B 1 0
C 2 2 0
D 2 5 3 0
E 3 4 6 6 0
Hierarchical Clustering
Linkage Methods
A B C D E
A 0
B 1 0
C 2 2 0
D 2 5 3 0
E 3 4 6 6 0
Hierarchical Clustering
Linkage Methods
A,B C D E
A,B 0
C 0
D 3 0
E 6 6 0
Hierarchical Clustering
Linkage Methods
A,B C D E
A,B 0
Single C 2 0
Link D 2 3 0
E 3 6 6 0
Hierarchical Clustering
Linkage Methods
A,B C D E
A,B 0
Complete C 2 0
Link D 5 3 0
E 4 6 6 0
Hierarchical Clustering
Linkage Methods
A,B C D E
A,B 0
C 2 0
Average D 3.5 3 0
E 3.5 6 6 0
Hierarchical Clustering
Linkage Methods
A,B C D E
A,B 0
C 2 0
Average D 3.5 3 0
E 3.5 6 6 0
Hierarchical Clustering
Linkage Methods
(A,B),C D E
(A,B),C 0
D 0
Average E 6 0
Hierarchical Clustering
Linkage Methods
(A,B),C D E
(A,B),C 0
D 3.33 0
Average E 4.33 6 0
Hierarchical Clustering
Linkage Methods
Single Link criteria
Complete Link criteria
Distance between
Average criteria cluster means
Centeroid criteria
Ward’s criteria
Hierarchical Clustering
Linkage Methods
Single Link criteria
Complete Link criteria
Minimize total within
Average criteria cluster variance
Centeroid criteria
Ward’s criteria
Hierarchical Clustering
Advantages Disadvantages
It overcomes the spherical High complexity level because
shape problem of K-Means it makes a huge number of
calculations between groups
بالتوفيق
