Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Machine Learning by Bio-Inspired

Algorithms
A Research Approach

Dr. Suresh Dara

Professor
Department of Computer Science and Engineering
B V Raju Institute of Technology, Narsapur, Telangana
[email protected]
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Outline
1 Introduction
AI Components
AI Domains
2 Machine Learning
3 AI Vs Machine Learning
Some Popular Classifiers
Some Popular Clusters
4 Nature Inspired Computation (EC)
Most Popular EC Algorithms
5 Genetic Algorithm
6 PSO
7 Research Challenges
Domains Where we can apply EC
8 Our Work
9 Conclusion
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

AI Components

AI Components
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

AI Domains

Artificial Intelligence -Domains

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Machine Learning

What is Machine Learning

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

ML Types

Different types of ML
Association
Supervised Learning
Classification
Regression
Unsupervised Learning
Reinforcement Learning
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

AI Vs ML Vs DL
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Machine Learning(ML)

ML is a branch of artificial intelligence, gives computers the

ability to learn without being explicitly programmed. –Arthur
Samuel, 1959.
Training
Validation
Testing
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Some Popular Classifiers

Classifiers

Classification is the process of generating a description or a model

for each class of a given dataset.
Alternating Decision Tree(ADT)
K-nearest neighbor (k-NN)
Best-First decision Tree (BFTree)
Bayesian Logistic Regression
Decision Stump(DS)
Bayesian network classifier
Extra Tree (ET)
Naive Bayes
Functional Trees(FT)
LibLinear
J48
LibSVM
LADTree(LADT)
Logistic
Logistic Model Trees(LMT)
Multi Layer Perceptron (MLP)
Random tree(RT)
Stochastic Gradient Descent(SGD)
Random Forest(RF)
SPegasos
REPTree(REPT)
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Some Popular Clusters

Clusters

Clustering involves the grouping of similar objects into a set known

as cluster
K–means Clustering K-Means Cluster Family
Mean-Shift Clustering K-Means++ (only changes how to
initialize centroids)
DBSCAN – Density Based Spatial
Clustering of Applications with Online K-Means
Noise Spherical K-Means
EM using GMM – K-Medoids
Expectation-Maximization (EM) Kernel K-Means
Clustering using Gaussian Mixture
K-Modes
Models (GMM)
Bisecting K-Means
Agglomerative Hierarchical
Clustering Fuzzy C-Means
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Evolutionary Computation
Evolutionary computation is a family of algorithms for global
optimization inspired by biological evolution, and the subfield of AI
and soft computing studying these algorithms
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Natural Computing Paradigms

Three level of development:

Phylogeny: The development of a species over time
(Evolution)
Ontogeny: The adaption of (in own life time) a organism
(Specialization)
Epigenesis: The development of complex organism (ongoing
experimental learning)
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Idea of EC
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Nature Computation

Conventional computing paradigms often have difficulty

dealing with real world problems
such as those characterized by noisy or incomplete data or
multimodality, because of their inflexible construction
Natural systems have evolved over millennia to solve such
problems
They have inspired several natural computing paradigms that
can be used where conventional computing techniques perform
unsatisfactorily
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Most Popular EC Algorithms

EC Algorithms
Genetic Algorithm
Evolutionary Programming
Population/Swarm–Based Optimization Methods
Particle Swarm Optimization
Immune Algorithms
Ant Colony Optimization
Firefly Algorithm
Cuckoo Algorithms
Bee Algorithm
Animal Based (Bat, Wolf, Monkey)
Many. . .
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Basic Idea & Principle Of Natural Selection

“Select The Best, Discard The Rest"

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

General Scheme of EA’s

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Pseudo-code for typical EA

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Components of Evolutionary Algorithms

Representations

Evaluation (Fitness) Function

Population

Selection (Parent Selection , Survivor Selection)

Variation Operators

Initialisation / Termination
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Genetic Algorithm

Genetic algorithms are global stochastic optimization

techniques based on natural genetics.

Robust and non-problem specific.

GAs code the parameter set of the optimization problem as

finite-length string.

GAs start the searching from a population of random points,

improve the quality of the population over time by genetic
operations: selection, crossover, mutation;

The best fitted solution will be evolved toward objective

function.
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Genetic algorithm -Flow chart

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Initial population
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Select
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Crossover
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Another Crossover
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Mutation
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA -Another Mutation
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA- Old population + children

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA- New Population: Generation 2

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA- Generation 3
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA- Generation 4, etc.

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA Example
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA Example(cont..)
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA Example(cont..) Selection operator

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA Example(cont..)Crossover operator
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA Example(cont..) Mutation operator

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

GA with Python

https://pypi.org/project/geneticalgorithm
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Different Flavours of EAs

Historically different flavours of EAs have been associated with

different representations
Binary strings : Genetic Algorithms
Real-valued vectors : Evolution Strategies
Finite state Machines: Evolutionary Programming
LISP trees: Genetic Programming

These differences are largely irrelevant, best strategy

choose representation to suit problem
choose variation operators to suit representation

Selection operators only use fitness and so are independent of

representation
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Particle Swarm Optimization

Introduced by Kennedy Eberhart in 1995, Inspired by social

behavior and movement dynamics of insects, birds and fish
PSO is a robust stochastic optimization technique based on
the movement and intelligence of swarms.
It uses a number of agents (particles) that constitute a swarm
moving around in the search space looking for the best
solution.
Each particle is treated as a point in a N-dimensional space
which adjusts its âflyingâ according to its own flying
experience as well as the flying experience of other particles.
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO Components

Update Position:

xij (t + 1) = xij + vij (t + 1) (1)

Update Velocity:

vij (t +1) = w ∗vij (t)+[c1∗R1∗pij −xij (t)]+[c2∗R2∗pg (t)−xij (t)]

(2)
xit - Particle position, vij - Particle velocity,
pij - Best "remembered" individual particle position,
pg (t) - Best "remembered" swarm position,
c1, c2 - Cognitive and social parameters,
R1, R2 - Random numbers between 0 and 1,
w - inertia weight.
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO Architecture
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO inertia weight

Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO Parameters

cognition-only
social-only, swarm best
inertia value: could help a particle stuck in a poor region move
away by hopping onto a more productive wave.
Preventing swarm explosion through the control : R1 and R2
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Why PSO than GA

Simple implementation, very few algorithm parameters

Good exploration abilities, very efficient global search algorithm
Insensitive to scaling of design variables
Easily parallelized for concurrent processing
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

PSO with Python

https://pyswarms.readthedocs.io/en/latest/index.html
PySwarms Package
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Hybrids . . .

PSO + Genetic Algorithms

PSO + Differential Evolution (DE)
PSO + fuzzy logic controller
PSO + Genetic Programming
PSO + Other soft computing tools
Similarly any other EC Algorithm with any other combination
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Domains Where we can apply EC

Medical domain
Cognitive science
Computational biology
Speech recognition
Wireless sensor networks
Recommender systems
Agriculture
Many. . .
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Domains Where we can apply EC

How achieve solutions

What is Your Problem?

How you Encoding and Decoding
Which Algorithm choosing and How?
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Domains Where we can apply EC

Eg: Medical Domian

Medical
Images Analysis (X-Ray, MRI)
Disease Prediction
High Dimensional Genomic Data Analysis
Feature Subset Selection
Feature Reduction
Remove irrelevant Features
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Our Research Contribution

1 Cancer Microarray Data Feature Selection using Multi

Objective Binary Particle Swarm Optimization Algorithm

2 A Hamming Distance based Binary Particle Swarm
Optimization (HD-BPSO) Algorithm for Feature Selection in
High Dimensional Data
3 A Rough Based Hybrid Binary PSO Algorithm for Flat Feature
Selection and Classification in Gene Expression Data
4 Binary gravitational Search algorithm (BGSA) for solving
feature selection problem
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

Conclusions
1 EC can Apply any domain and any kind of problem solving

2 Choosing algorithm is critical task, its domain and problem

independent
3 Selecting Activation function
4 Design fitness function
5 Learn new frameworks and tune the parameters
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

My Google Scholar
https://scholar.google.co.in/citations?user=ldzq6M8AAAAJhl=enoi=ao
Introduction Machine Learning AI Vs Machine Learning Nature Inspired Computation (EC) Genetic Algorithm

[email protected], [email protected]