COMPUTATIONAL METHODS AND TECHNIQUES

Module I

Neural Networks

A neural network is a method in artificial intelligence that teaches computers to

process data in a way that is inspired by the human brain. It is a type of machine
learning process, called deep learning, that uses interconnected nodes or neurons in
a layered structure that resembles the human brain.
This article focuses on three important types of neural networks that form the basis for most pre-trained
models in deep learning: Artificial Neural Networks (ANN) Convolution Neural Networks (CNN) Recurrent
Neural Networks (RNN

Artificial neural networks are a branch of machine learning models that are built using principles of neuronal
organization discovered by connectionism in the biological neural networks constituting animal brains

5 algorithms to train a neural network

 Gradient descent.
 Newton method.
 Conjugate gradient.
 Quasi-Newton method.
 Levenberg-Marquardt algorithm.

Backpropagation is the most common training algorithm for neural networks. It makes gradient descent
feasible for multi-layer neural networks. TensorFlow handles backpropagation automatically, so you don't
need a deep understanding of the algorithm

The three main types of learning in neural networks are supervised learning, unsupervised learning, and
reinforcement learning

The Feed-Forward (FF) network is commonly used along with Back Propagation to train neural networks.

Hopfield neural network (HNN) is one of the ANN which is a single layer recursive neural network where
all neurons are both input and output and they are connected with equal weights . The energy function of
the HNN model is defined in such a way that its time derivative is negative

The Kohonen map performs a mapping from a continuous input space to a discrete output space,
preserving the topological properties of the input. This means that points close to each other in the input
space are mapped to the same neighboring neurons in the output space.
k-means clustering is a method of vector quantization, originally from signal processing, that aims to partition n
observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a
prototype of the cluster

Basically, ART network is a vector classifier which accepts an input vector and classifies it into one of the
categories depending upon which of the stored pattern it resembles the most.
In the field of mathematical modeling, a radial basis function network is an artificial neural network that uses radial
basis functions as activation functions. The output of the network is a linear combination of radial basis functions of
the inputs and neuron parameters. Radial basis function networks have many uses, including function
approximation, time series prediction, classification, and system control. They were first formulated in a 1988 paper
by Broomhead and Lowe, both researchers at the Royal Signals and Radar Establishment
Deep Learning

Neural networks are a part of deep learning, which comes under the
comprehensive term, artificial intelligence. Neural networks are a set of
algorithms that are modelled after the human brain. These networks are also
known as artificial neural networks (ANN).
Sensory neurons, motor neurons and interneurons form the human
brain. Artificial neurons, form the replica of the human brain (i.e. a neural
network).

Artificial Neural Network (ANN)

Artificial Neural Network (ANN) is a collection of connected units (nodes). These

connected units are known as artificial neurons. These units closely resemble
the original neurons of a human brain. Every node is built with a set of inputs,
weights, and a bias value. Weights of the neural network are held within the
hidden layers.

Weights and biases are learning parameters of machine learning models, they
are modified for training the neural networks.

Architecture of an artificial neural network

Applications of Neural Networks

Neural Networks are regulating some key sectors including finance, healthcare,
and automotive. As these artificial neurons function in a way similar to the
human brain. They can be used for image recognition, character recognition and
stock market predictions. Let’s understand the diverse applications of neural
networks

1. Facial Recognition

Facial Recognition Systems are serving as robust systems of surveillance.

Recognition Systems matches the human face and compares it with the digital
images. They are used in offices for selective entries. The systems thus
authenticate a human face and match it up with the list of IDs that are present
in its database.
(Must Check: Facial Recognition Work in Deep Learning?)

Convolutional Neural Networks (CNN) are used for facial recognition and
image processing. Large number of pictures are fed into the database for
training a neural network. The collected images are further processed for
training.

Sampling layers in CNN are used for proper evaluations. Models are optimized
for accurate recognition results.

(Related Blog: How does Basic Convolution Work for Image Processing? )

2. Stock Market Prediction

Investments are subject to market risks. It is nearly impossible to predict the

upcoming changes in the highly volatile stock market. The forever changing
bullish and bearish phases were unpredictable before the advent of neural
networks. But well what changed it all? Neural Networks of course…

To make a successful stock prediction in real time a Multilayer Perceptron

MLP (class of feedforward artificial intelligence algorithm) is employed. MLP
comprises multiple layers of nodes, each of these layers is fully connected to the
succeeding nodes. Stock’s past performances, annual returns, and non profit
ratios are considered for building the MLP model.

Check out this video to know how the LTSM model is built for making predictions
in the stock market.

3. Social Media
No matter how cliche it may sound, social media has altered the normal boring
course of life. Artificial Neural Networks are used to study the behaviours of
social media users. Data shared everyday via virtual conversations is tacked up
and analyzed for competitive analysis.

Neural networks duplicate the behaviours of social media users. Post analysis of
individuals' behaviours via social media networks the data can be linked to
people’s spending habits. Multilayer Perceptron ANN is used to mine data
from social media applications.

MLP forecasts social media trends, it uses different training methods like Mean
Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean
Squared Error (MSE). MLP takes into consideration several factors like user’s
favourite instagram pages, bookmarked choices etc. These factors are
considered as inputs for training the MLP model.

In the ever changing dynamics of social media applications, artificial neural

networks can definitely work as the best fit model for user data analysis.

(Related Blog: Detection of Fake and False News using CNNl)

4. Aerospace

Aerospace Engineering is an expansive term that covers developments in

spacecraft and aircraft. Fault diagnosis, high performance auto piloting, securing
the aircraft control systems, and modeling key dynamic simulations are some of
the key areas that neural networks have taken over. Time delay Neural
networks can be employed for modelling non linear time dynamic systems.

Time Delay Neural Networks are used for position independent feature
recognition. The algorithm thus built based on time delay neural networks can
recognize patterns. (Recognizing patterns are automatically built by neural
networks by copying the original data from feature units).

Other than this TNN are also used to provide stronger dynamics to the NN
models. As passenger safety is of utmost importance inside an aircraft,
algorithms built using the neural network systems ensures the accuracy in the
autopilot system. As most of the autopilot functions are automated, it is
important to ensure a way that maximizes the security.

Applications of neural networks

5. Defence

Defence is the backbone of every country. Every country’s state in the

international domain is assessed by its military operations. Neural Networks also
shape the defence operations of technologically advanced countries. The United
States of America, Britain, and Japan are some countries that use artificial
neural networks for developing an active defence strategy.

Neural networks are used in logistics, armed attack analysis, and for object
location. They are also used in air patrols, maritime patrol, and for controlling
automated drones. The defence sector is getting the much needed kick of
artificial intelligence to scale up its technologies.

Convolutional Neural Networks(CNN), are employed for determining the

presence of underwater mines. Underwater mines are the underpass that serve
as an illegal commute route between two countries. Unmanned Airborne
Vehicle (UAV), and Unmanned Undersea Vehicle (UUV) these autonomous
sea vehicles use convolutional neural networks for the image processing.
Convolutional layers form the basis of Convolutional Neural Networks. These
layers use different filters for differentiating between images. Layers also have
bigger filters that filter channels for image extraction.
6. Healthcare

The age old saying goes like “Health is Wealth”. Modern day individuals are
leveraging the advantages of technology in the healthcare
sector. Convolutional Neural Networks are actively employed in the
healthcare industry for X ray detection, CT Scan and ultrasound.

As CNN is used in image processing, the medical imaging data retrieved from
aforementioned tests is analyzed and assessed based on neural network
models. Recurrent Neural Network (RNN) is also being employed for the
development of voice recognition systems.

(Must Check: Learning Recurrent Neural Network and applications)

Voice recognition systems are used these days to keep track of the patient’s
data. Researchers are also employing Generative Neural Networks for drug
discovery. Matching different categories of drugs is a hefty task, but generative
neural networks have broken down the hefty task of drug discovery. They can
be used for combining different elements which forms the basis of drug
discovery.

7. Signature Verification and Handwriting Analysis

Signature Verification , as the self explanatory term goes, is used for verifying
an individual’s signature. Banks, and other financial institutions use signature
verification to cross check the identity of an individual.

Usually a signature verification software is used to examine the signatures. As

cases of forgery are pretty common in financial institutions, signature
verification is an important factor that seeks to closely examine the authenticity
of signed documents.

Artificial Neural Networks are used for verifying the signatures. ANN are
trained to recognize the difference between real and forged signatures. ANNs
can be used for the verification of both offline and online signatures.

For training an ANN model, varied datasets are fed in the database. The data
thus fed help the ANN model to differentiate. ANN model employs image
processing for extraction of features.

(Related Blog: Hand Gesture Classification using Deep Learning with Keras)

Handwriting analysis plays an integral role in forensics. The analysis is further

used to evaluate the variations in two handwritten documents. The process of
spilling words on a blank sheet is also used for behavioural
analysis. Convolutional Neural Networks (CNN) are used for handwriting
analysis and handwriting verification.

8. Weather Forecasting

The forecasts done by the meteorological department were never accurate

before artificial intelligence came into force. Weather Forecasting is primarily
undertaken to anticipate the upcoming weather conditions beforehand. In the
modern era, weather forecasts are even used to predict the possibilities of
natural disasters.

Multilayer Perceptron (MLP), Convolutional Neural Network (CNN) and

Recurrent Neural Networks (RNN) are used for weather forecasting.
Traditional ANN multilayer models can also be used to predict climatic
 conditions 15 days in advance. A combination of different types of neural
network architecture can be used to predict air temperatures.

(Must Check: Weather Forecasting Using Big Data Analytics)

Various inputs like air temperature, relative humidity, wind speed and solar
radiations were considered for training neural network based
models. Combination models (MLP+CNN), (CNN+RNN) usually works better
in the case of weather forecasting.

MODULE-II
Fuzzy logic is an approach to computing based on "degrees of truth" rather than the
usual "true or false" (1 or 0) Boolean logic on which the modern computer is based.
The idea of fuzzy logic was first advanced by Lotfi Zadeh of the University of
California at Berkeley in the 1960s.
Fuzzy logic is a form of many-valued logic in which the truth value of variables may be any real number between 0
and 1. It is employed to handle the concept of partial truth, where the truth value may range between completely true
and completely false.
Enter “fuzzy logic.”
 Here's some examples:
 Air Conditioning. Old air conditioners were set to a minimum and maximum room temperature. ...
 Cooking. If you've ever struggled with getting rice cooked to the right texture and moisture, you might
have purchased a rice cooker. ...
 Washing Machines. ...
 TVs. ...
 Law Enforcement.

MODULE-III
Every optimization problem has three components: an objective function, decision variables, and
constraints. When one talks about formulating an optimization problem, it means translating a “real-world”
problem into the mathematical equations and variables which comprise these three components
In this article, we made an introduction to mathematical optimization. First, we talked about the term
optimization. Then we discussed its three basic terms: objective function, decision variables, and
constraints
Optimization Theory is a branch of mathematics devoted to solving optimization problems. Optimization
problems are mathematical functions where we want to minimize or maximize the function value
The conversion optimization process has four main steps: research, testing, implementation, and
analysis. With all these steps, you must plan an effective content strategy to help people understand how
to use a product or market a new service.
Linear programming (LP) or Linear Optimisation may be defined as the problem of
maximizing or minimizing a linear function that is subjected to linear constraints. The
constraints may be equalities or inequalities. The optimisation problems involve the
calculation of profit and loss
The different types of linear programming are as follows:
 Solving linear programming by Simplex method.
 Solving linear programming using R.
 Solving linear programming by graphical method.
 Solving linear programming with the use of an open solver.
Linear programming (LP) uses many linear inequalities pertaining to a given scenario to determine the
“optimal” value one can obtain under those constraints. A classic example would be calculating the
“optimal” production levels to maximize profits, given the restrictions of supplies and personnel
Nonlinear programming is minimizing or maximizing a nonlinear objective function
subject to bound constraints, linear constraints, or nonlinear constraints, where the
constraints can be inequalities or equalities
Module IV
In computer science and operations research, a genetic algorithm is a metaheuristic inspired by the process of
natural selection that belongs to the larger class of evolutionary algorithms
The genetic algorithm is a method for solving both constrained and unconstrained
optimization problems that is based on natural selection, the process that drives
biological evolution. The genetic algorithm repeatedly modifies a population of
individual solutions.

Genetic Algorithms(GAs) are adaptive heuristic search algorithms that belong to the larger part of
evolutionary algorithms. Genetic algorithms are based on the ideas of natural selection and
genetics. These are intelligent exploitation of random search provided with historical data to direct
the search into the region of better performance in solution space. They are commonly used to
generate high-quality solutions for optimization problems and search problems.
Genetic algorithms simulate the process of natural selection which means those species who
can adapt to changes in their environment are able to survive and reproduce and go to next
generation. In simple words, they simulate “survival of the fittest” among individual of consecutive
generation for solving a problem. Each generation consist of a population of individuals and
each individual represents a point in search space and possible solution. Each individual is
represented as a string of character/integer/float/bits
1. Individual in population compete for resources and mate
2. Those individuals who are successful (fittest) then mate to create more offspring than others
3. Genes from “fittest” parent propagate throughout the generation, that is sometimes parents
create offspring which is better than either parent.
4. Thus each successive generation is more suited for their environment.
Search space
The population of individuals are maintained within search space. Each individual represents a
solution in search space for given problem. Each individual is coded as a finite length vector
(analogous to chromosome) of components. These variable components are analogous to Genes.
Thus a chromosome (individual) is composed of several genes (variable components).

Fitness Score
A Fitness Score is given to each individual which shows the ability of an individual to
“compete”. The individual having optimal fitness score (or near optimal) are sought.
The GAs maintains the population of n individuals (chromosome/solutions) along with their fitness
[Link] individuals having better fitness scores are given more chance to reproduce than
others. The individuals with better fitness scores are selected who mate and produce better
offspring by combining chromosomes of parents. The population size is static so the room has to
be created for new arrivals. So, some individuals die and get replaced by new arrivals eventually
creating new generation when all the mating opportunity of the old population is exhausted. It is
hoped that over successive generations better solutions will arrive while least fit die.
Each new generation has on average more “better genes” than the individual (solution) of previous
generations. Thus each new generations have better “partial solutions” than previous
generations. Once the offspring produced having no significant difference from offspring produced
by previous populations, the population is converged. The algorithm is said to be converged to a
set of solutions for the problem.
Operators of Genetic Algorithms
Once the initial generation is created, the algorithm evolves the generation using following
operators –
1) Selection Operator: The idea is to give preference to the individuals with good fitness scores
and allow them to pass their genes to successive generations.
2) Crossover Operator: This represents mating between individuals. Two individuals are selected
using selection operator and crossover sites are chosen randomly. Then the genes at these
crossover sites are exchanged thus creating a completely new individual (offspring). For example
–
3) Mutation Operator: The key idea is to insert random genes in offspring to maintain the
diversity in the population to avoid premature convergence. For example –

The whole algorithm can be summarized as –

1) Randomly initialize populations p
2) Determine fitness of population
3) Until convergence repeat:
a) Select parents from population
b) Crossover and generate new population
c) Perform mutation on new population
d) Calculate fitness for new population
Example problem and solution using Genetic Algorithms
Given a target string, the goal is to produce target string starting from a random string of the same
length. In the following implementation, following analogies are made –
 Characters A-Z, a-z, 0-9, and other special symbols are considered as genes
 A string generated by these characters is considered as chromosome/solution/Individual
Fitness score is the number of characters which differ from characters in target string at a
particular index. So individual having lower fitness value is given more preference.