See discussions, stats, and author profiles for this publication at: [Link]

net/publication/346391164

Study on image processing using deep learning techniques

Article in Materials Today Proceedings · October 2020

DOI: 10.1016/[Link].2020.09.536

CITATIONS READS

14 8,397

3 authors:

Kschandra Rao Y Srinivas

Jagannath Institute for Technology and Management GITAM University
5 PUBLICATIONS 23 CITATIONS 223 PUBLICATIONS 1,215 CITATIONS

SEE PROFILE SEE PROFILE

M Vamshi Krishna
Dhanekula institution of engineering and technology
37 PUBLICATIONS 60 CITATIONS

SEE PROFILE

All content following this page was uploaded by M Vamshi Krishna on 28 January 2021.

The user has requested enhancement of the downloaded file.

 Materials Today: Proceedings xxx (xxxx) xxx

Contents lists available at ScienceDirect

Materials Today: Proceedings

journal homepage: [Link]/locate/matpr

Study on image processing using deep learning techniques

Santoshachandra Rao Karanam a, Y. Srinivas b, M. Vamshi Krishna c
a
Department of CSE, Centurion University of Technology and Management, Odisha, India
b
Department of IT, GITAM University, Visakhapatnam, India
c
Centurion University of Technology and Management, Odisha, India

a r t i c l e i n f o a b s t r a c t

Article history: Image processing plays an important role in the fields like image mining, medical imaging, medical image
Received 13 September 2020 processing, web mining etc. Content Based Medical Image Retrieval (CBMIR) systems are vital to the
Accepted 21 September 2020 underlying operation of medical databases because they allow quick search and retrieval of medical
Available online xxxx
images. The present article gives complete understanding on usage of deep learning Algorithm in image
segmentation. This article presents an overview of various possibilities of deep learning which can be
Keywords: adopted into medical imaging and image mining. It also throws an insight of content based image retrie-
Image processing
val and identifying a flaw in the medical image.
Image analysis
Medical imaging
Ó 2020 Elsevier Ltd. All rights reserved.
Web mining and deep learning Selection and peer-review under responsibility of the scientific committee of the Emerging Trends in
Materials Science, Technology and Engineering.

1. Introduction nique, future based technique , depth based technique and model
based techniques are also highlighted in the Literature [11–14].
Deep learning is a core sub-field of Artificial Intelligence. The However it is assumed that model based techniques or generative
aim of Deep learning is study flexible computer algorithms that approaches are considered to be more effective since in this pro-
are able to learn. This learning that is being done is always based cess, the analysis is based on estimation of the parameters and
on some kind of experience based on known input data or instruc- whole analysis is bases on these parameters. Apart from these
tions. In particular medical image analysis is a field where Deep approaches with the latest insights in the field of machine learning,
Learning can improve the quality and speed of research. Image pro- deep learning techniques came into existence. These techniques
cessing deals with two main objectives namely enhancing the assume to be more promising in particular for analyzing the med-
images and analyzing the images. Among image enhancement ical tissues. In this article, a comprehensive study is carried out to
techniques, works are projected in the literature showcasing vari- present the various fields of image processing where deep learning
ous applications where the need of enhancement is clearly identi- algorithms are more suitable. Section2 of the article presents an
fied. It also helps to analyze the images to have better overview of convolutional neural networks. The applications of
understanding about the image and during this process techniques deep learning algorithms into content based image retrievals are
like clustering, classification are mostly considered. With the presented. The advantages of the current proposed deep learning
advancements of technologies, many applications of image pro- algorithms in various disciplines are highlighted in section4 of
cessing came into existence ranging from ordinary image process- the article. The concluding section5 summarizes the article with
ing to analyzing of medical images. As the number of medical cases various feature directions.
are increasing in proportion to number of individuals, to need a
better understanding about the diseases one need to have a com- 2. Machine learning
prehensive study about the images . In order to achieve this objec-
tive, many medical image segmentation techniques are evolved Computers need machine learning approaches to figure out
based on different methodologies such as edge based enhance- how tasks can be carried out without specific programming. To
ments, shape based enhancements, region based descriptors, tex- perform those functions, computers have to learn from the knowl-
ture based technique, color based technique [1–10]. The above edge received. When a necessary computer task is carried out,
said techniques are considered to be de-generative models and algorithms can be programmed to tell the system to take specific
apart from these models other models like pattern based tech- measures to solve the problem. In practical practice, it helps the

[Link]
2214-7853/Ó 2020 Elsevier Ltd. All rights reserved.
Selection and peer-review under responsibility of the scientific committee of the Emerging Trends in Materials Science, Technology and Engineering.

Please cite this article as: Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna, Study on image processing using deep learning techniques,
Materials Today: Proceedings, [Link]
Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna Materials Today: Proceedings xxx (xxxx) xxx

machine not to specify any required steps, but to construct its algo- because they bind to them. Usually, artificial neurons and edges
rithm. In the machine training field, computers are taught how to are calculated according to learning processes.
do tasks where an algorithm is not completely satisfactory. If there The frequency of the signal at contact is increased or reduced.
are several potential responses, one way is to identify some of the The threshold of artificial neurons can only be transmitted when
right choices as valid. This method is used to improve the algo- the threshold value is exceeded by the aggregate signal. These neu-
rithms used as computer training data to determine the right reac- rons are normally attached to layers. Various layers can transform
tions. For example, the MNIST handwritten numbers dataset was their inputs into different kinds. Signals transfer from the first (in-
often used to train a digital character recognition system. put layer) layer (output layer) to the last layer, probably after sev-
eral layer crossings. The ANN method was initially developed to
tackle problems as a human brain might solve them. Fig. 2 repre-
sents the multilayer ANN approach. However, over time emphasis
2.1. Methods of the Machine learning
on other behaviors has been transferred that lead to biological
changes. Computer vision and speech comprehension are some
The output of a machine analysis algorithm running on data is
important, profound learning applications.
referred to as ‘‘pattern”. A model is what was learned from an algo-
rithm. It is ‘‘the thing” that is saved after the execution of a training
data algorithm and represents the rules, numbers, and any data 2.2. Convolutional neural networks (CNN)
structure unique to the algorithm needed to make predictions.
Machine learning consists of a building model that will practice ANN is an information processing paradigm mainly inspired by
on some training data and will then process the additional infor- biological nervous systems. It is composed by a high number of
mation for predictions. Many approaches for machine learning sys- processing units, called neurons, working in unison to solve a
tems have been used and researched. specific task. Learning process in ANN involves, like in a biological
system, the adjustments of the connections between the process-
 Artificial Neural networks ing units. Convolutional neural networks is a part of machine
 Decision tree learning techniques which deals with knowledge representation
 Support vector Machine and deep learning methodology is a sub field of convolutional neu-
 Regression Analysis ral networks which is more adaptive for analyzing the visual
 Bayesian Networks images. This deep learning algorithms help in better understanding
of the parameters in the form of dividing the images into layers
Artificial Neural Network. An interconnected node population, such that each individual layer is scrutinized and can be more per-
like the extensive neuron network of the brain, is an artificial neu- fectly analyzed when compared to the conventional analysis pro-
ral network. The growing circular node is an artificial neuron, cedure. The number of layers that are formulated upon the
which is a relation between the output of a neuron and the input application of deep leaning approach includes of horizontal layer,
of a neuron. Remote-influenced software applications shape brain vertical layer, input layer and output layer. The number of hidden
anatomy in biological processes are neural networks or attachment layers can be identified by using the product of horizontal vertical
networks. Such processes typically ‘‘learn to perform tasks” with- and the number of channels being used.
out the tasks’ basic rules being configured.
A model ANN is constructed on artichokes of connected units or
knots linked to neurons that freely form the biological brain neu- 2.2.1. CNN neural network model
rons. A connection can relay information like the synapses inside Typically, fully linked neural networks aren’t working well on
a natural brain, a ‘‘signal” or ‘‘wave” from an artificial neuron photos. That’s because every single pixel Is an input, as we add
shown in Fig. 1 to another neuron. A signal-received from artificial more layers, the parameter quantity increases exponentially.
neuron may process it and indicate the relationship of new artifi- What’s This The special structure renders one image distinguish-
cial neurons. A signal at an artificial neuron connection is a real able from another. Areas within close proximity And areas in the
number in typical ANN implementations, and a particular function vicinity are highly important in photos. CNN can be used to reach
of the sum of its weighted inputs decides, the output of each arti- a low level and low Representation of contents of images and end
ficial neuron. ‘‘Edges” of the model are called artificial neurons to end structure.

Fig. 1. Biological neuron. Fig. 2. Artificial Neuron.

2
Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna Materials Today: Proceedings xxx (xxxx) xxx

a) Convolution 3. Content based medical image retrieval (CBMIR)

In CNN, one tile (3x3 or 5x5 pixels) of the input feature map is Content Based Medical Image Retrieval (CBMIR) is a rapidly
extracted and added by a convolution Filters. Filters slide over the developing field and has been one of the most promising applica-
input Grid of the function from left to right and from top to bottom, tions of Content Based Image Retrieval (CBIR) systems to date.
one pixel at a time, extracting each respective map. The CNN One advantage of modern health-care facilities is that they collect
‘‘learns” the optimal values for the filter matrices during training, massive quantities of image data, because of the sheer volume of
which allow it to extract Relevant features. As the number of filters patients which they have processed. Unfortunately, significant por-
used increases, so does the amount of features that the CNN does tions of the collected data are left underutilized because of scala-
remove. bility, privacy, and or integration restrictions/roadblocks. CBMIR
systems present a potential solution to the issues of privacy and
b) Activation Function scalability, because they can automatically search through millions
of medical image to find a list of relevant hits to a search query
An Activation function is a neural network component. The acti- with minimal human involvement. It is important to note that
vation function determines whether a Fires on nerves, or not. To CBMIR systems are software based systems and suffer from the
ensure that our network is not nonlinear, we need to make sure same risks and privacy concerns associated with any such system.
Because these are nonlinear activation functions. We can use a In the past a critical limitation of medical image retrieval systems
stage feature as our enabling Function which gives zero or one out- has been maintaining patient privacy because image data needed
put. If the output reaches a certain threshold then Neurons are to be heavily controlled and restricted. CBMIR systems have helped
shot, and we got one. If the output value is below the threshold overcome these roadblocks by using computer algorithms to
then it is not Shot, and we got a blank. review medical images instead of human evaluators, and are gen-
erally agnostic to a patients identity .
c) Rectified Linear Units In order to Retrieve the image effectively, features are most pre-
dominantly used .These features ‘‘ include low level features , high-
The ReLU is one of the Activation functions that is best known level features, semantic features, low level features and high level
[9]. The ReLU or ‘‘Set Linear” Unit ‘‘output, a zero for every x value features are assumed to lose some of the inherent meaningful pat-
less than zero. For any value of x greater than or equal to The func- ters which are of core necessity during the analysis of the images
tion returns x, then zero. The network applies after every CNN con- are retrieval of the images. Since CBIR is assumed to be a retrieval
volution process a Transformation of the rectified linear unit methodology based on the features effectual data is not promising
(ReLU) to the transformed element, to implement Nonlinearity when these features are considered since they are assumed to miss
inside pattern. sum of the visual information. To overcome this semantic based
approaches which maintains the semantic representations are
d) Pooling mostly preferred. Hence the paradigm is shifted towards retrieval
of content based on CBIR. Since deep learning approach are most
Pooling is the next step after ReLU, in which the CNN reduces promising in the consideration of deep learning approach, they
convolved sampling Feature, and number of feature map dimen- are mostly preferred. Using CBMIR systems also allows medical
sions, while retaining the most critical feature Information [10]. professionals to trace similar historical cases based on the similar-
It’s information and this method is called max pooling, and is ity of their medical images, allowing for the development of better
one of the Usual algorithms. Also, there are other pools, such as diagnostic models and protocols. The advancement of such sys-
the regular pool and min pool. Max pooling works exactly the same tems is critical to the development of data-based diagnostic tech-
Convolutionary style. It slides over the feature map, extracting tiles niques which potentially provides life-saving information to
of a given size and there is forming each extracted tile, output the medical professionals.
maximum value to a new function map from that tile and It dis- CBMIR systems share a common framework with conventional
cards all other beliefs. There are two parameters to Limit pooling CBIR approaches as shown in Fig When initially, an medical image
operations. Max-pooling size Typical filter is 2x2 pixels. The dis- database is created, and the CBMIR system is in its online phase, all
tance between each extracted tile is called stride, in pixel. It is dis- medical images stored have representative feature vectors
tinct from those of convolution, Where convolution filters slide extracted. Feature vectors are encoded with characteristic informa-
pixel by pixel over the feature map but in max pooling, step tion about the medical image and stored with the image. Database
Determines the locations from which each tile is drawn. For a queries are handled by the online component of the CBMIR system,
filter size of 2x2, specifies a phase of 2 That the max pooling which compare extracted feature vectors from a query image to all
operation extracts from the function map all non-overlapping feature vectors compared with the database via the similarity mea-
2x2 tiles. sure. A list of likely matches is then generated, stored, then dis-
played to the user for review or evaluation. Modern CBMIR
e) Fully Connected Layers systems go a step further, by asking the user to identify the rele-
vance of the displayed images to the query image to improve the
At the end of a convolutional neural network one or more com- accuracy of the CBMIR system. Medical images are especially chal-
pletely connected layers are. Four layers Is completely connected lenging for CBIR systems because there are many imaging modal-
when each node in the first layer connects to each node in the sec- ities, and the image size is often quite large. Calculating a query
ond layer Cap. Their task is to conduct classification based on the images similarity to an entire database, containing thousands if
characteristic that the convolutions collect. Mostly, neurons are not millions of images, can be computationally expensive. Conven-
fully connected to the end. This final layer is completely linked tional CBIR systems tackle this problem by reducing the size of the
and contains a Activation function SoftMax which gives a probabil- representative image feature vector to a manageable size, and
ity value of 0 to 1 for each of the model’s ranking labels attempt to using specialized filters to extract only the most salient features.
predict. However, in medical imaging, the size and resolution of the images

3
Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna Materials Today: Proceedings xxx (xxxx) xxx

vary based on the type of image device and method used. Practi- Classification, as we have more than two classifications Precision
cally this translates to creation longer feature vectors, which in alone may be deceptive if we have disproportionate experiences
turn are computationally more expensive to process. Exasperating of several different groups In Dataset. Calculating a uncertainty
the challenges mentioned above is the fact that misclassification in matrix to get a better understanding of what our classification
CBMIR systems can lead to potentially fatal consequences (i.e., model is forecasting And visual representation of it is a good
misdiagnosis) Figs. 3–5. choice. Maximum number of assumptions right and wrong Tests
are summarized and then broken down according to gender.
a) Training the Model The function (Confusion Matrix) calculates the confusion matrix
and returns the result as an Array showing how our classification
We’ve developed and compiled our model so far. This is the model is confused with which class when it comes to making Prog-
time for pattern preparation. In the computer field We need a huge noses. It provides insight not only into what errors are made by the
dataset of image and video files for learning and machine vision classifier but also more Different error forms. We can print this
and that’s one of the Dispute. Because when loading and process- sequence or use [Link] to plot it and An interpretation
ing lots of images or video data, We possibly encountered a situa- of the performance.
tion where our computer didn’t have enough memory [14].
Therefore, we need to create Data Generators when loading and
processing images in Keras library. 4. Image processing using deep learning

b) Image Data Generator Function The first outcome regards the difference between natural
images and medical images in Deep Learning. Natural images are
The class Image Data Generator is very useful in our application plenty of very different objects with very different structures. This
of image classification. We can use this data generator in many allows the network to learn very complex and different filters
ways, depending on the form we would like to use. The bigger especially in the deeper layers Deep learning is one of the promis-
the number of epochs, the better our model. But only up to a cer- ing field focusing on medical image analysis. Deep learning open
tain point will the model stop improving over each epoch. That is it possibilities in areas like bio-imaging , neuro-imaging and DNA
Why we use ’EarlyStopping(es)’ to stop training if there is no sub- sequencing deep learning algorithms help in automatic medical
stantial improvement in the model. We use the method flow from imaging segmentations with focus on various features extracted
directory in the Image Data generator class of the library keras. from the medical image dataset. It provides a new way of identify-
ing abnormalities and bring plausible out comes with better diag-
c) Prediction nosis. As disclosed previously Supervised Learning is an approach
to learning that require a known dataset. This set is provided of
Now let’s load the images for tests or predictions using the pre- both inputs and correct out- puts for the algorithm used. Starting
vious (load Image) (function). Original images had to be resized from this set of examples the program is guided to describe a
before they would be given to a predictive model trained. Applica- model able to predict the correct output. At this point the predic-
tions We can resize all test images to 128X128 using the CV2 com- tion model must be validated with another known dataset inde-
puter vision library. Then, we will Use the Number phy library to pendent from the training set. Only when the validation phase is
add all images to a single collection. satisfactory the algorithm can be considered reliable for use on
unknown data.
d) Load Trained model for Prediction Algorithm selection: The first step is to choose the supervised
algorithm to use. Every method has different strength and weak
To load the trained model we use Image Data Generator class point. The choice depends on the particular problem and on the
load model method. That will load from the valued location a qual- kind and amount of available data. Some of these algorithms are:
ified model. This method load model takes a path For the predic- Support Vector Machine (SVM), Decision Tree, Artificial Neural
tion algorithm and loads qualified software. Predicting process is Network and Deep Learning. In this work the focus will be on Deep
then used to estimate Using the given trained model every test Learning, extension of ANN, for reasons that will be explained in
picture. the next sections.
Training: The training phase is probably the most important
e) Confusion Matrix one, as the final performances depend on the predictive model
built.
To summarize the results of our classification algorithm for per-
formance and prediction, we used a Matrix confusion technique.  A known dataset is selected; must be as more representative of
the problem as possible. Using dataset not general enough can
lead to over fitting and to bad performances. This set, the train-
ing set, must provide an output (label) for each listed input.
 The algorithm is trained with the selected dataset. The aim of
this phase is trying to build a model able to t the data provided,
that is predict the correct output for each input provided as best
as possible.

Validation: The validation phase is important to test the perfor-

mances achieved by the prediction model built in the previous
phase. Another known dataset, called test set, is prepared. The
dataset must provide, as the training set, reliable input and output
for each example. An important property of this set is that it should
be as independent as possible from the training one. The previ-
Fig. 3. CNN Model. ously trained algorithm is here used to predict the input data of
4
Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna Materials Today: Proceedings xxx (xxxx) xxx

Fig. 4. Framework for content based medical image retrieval using deep convolutional neural network.

image quite processing and hence various areas that can be of

potential use for the application of deep learning algorithms are
highlighted.

CRediT authorship contribution statement

Santoshachandra Rao Karanam: Conceptualization, Methodol-

ogy, Software, Data curation. Y. Srinivas: Writing - original draft,
Validation, Visualization. M. Vamshi Krishna: Investigation,
Supervision, Software, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing finan-

Fig. 5. Supervised Learning flowchart. cial interests or personal relationships that could have appeared
to influence the work reported in this paper.

the test set. Only the input are used and the output are predicted References
by the algorithm and stored. The fundamental difference from
the train step is that, in this one, the output label are not used to [1] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016.
[2] A. Krizhevsky, I. Sutskever, G. Hinton, ImageNet classification with deep
improve the prediction capabilities of the model, but only to eval-
convolutional neural networks, NIPS (2012).
uate its performances. The predicted outputs are validated using [3] J. Deng, A. Berg, S. Satheesh, H. Su, A. Khosla, and L. FeiFei. ImageNet Large
the known outputs. The performance are hence evaluated and Scale Visual Recognition Competition 2012 (ILSVRC2012)..
analysed. If they are satisfactory it is possible to go to the final step, [4] S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: Towards real-time object
detection with region proposal networks, NIPS (2015).
otherwise the algorithm or the training phase must be reviewed [5] M. A. Abbas, ‘‘Improving deep learning performance using random forest HTM
with different precautions or parameters. Model Deployment Once cortical learning algorithm,” 2018 First International Workshop on Deep and
the algorithm is trained and validated, it is possible to use it as an Representation Learning (IWDRL), Cairo, 2018, pp. 13-18.
[6] J. Wu, Yinan Yu, Chang Huang and Kai Yu, ‘‘Deep multiple instance learning for
automatic system to solve the original problem on new data. image classification and auto-annotation,” 2015 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 3460-3469.
[7] L. Shao, D. Wu, X. Li, Learning Deep and Wide: A Spectral Method for Learning
5. Conclusion Deep Networks, IEEE Transactions on Neural Networks and Learning Systems
25 (12) (Dec. 2014) 2303–2308.
The first outcome regards the difference between natural [8] K. Nguyen, C. Fookes and S. Sridharan, ‘‘Improving deep convolutional neural
networks with unsupervised feature learning,” 2015 IEEE International
images and medical images in Deep Learning. Natural images are Conference on Image Processing (ICIP), Quebec City, QC, 2015, pp. 2270-2274.
plenty of very different objects with very different structures. This [9] Y. Wang, ‘‘Deep reasoning and thinking beyond deep learning by cognitive
allows the network to learn very complex and different filters robots and brain-inspired systems,” 2016 IEEE 15th International Conference
on Cognitive Informatics & Cognitive Computing (ICCI*CC), Palo Alto, CA, 2016,
especially in the deeper layers. Medical images, in particular fun- pp. 3-3.
dus retinal images, have a lower degree of very different patterns, [10] B. Akay, O. Kaynar and F. Demirkoparan, ‘‘Deep learning based recommender
but very small difference in few details can do the difference in systems,” 2017 International Conference on Computer Science and
Engineering (UBMK), Antalya, 2017, pp. 645-648.
classification. This is testified by the high difference in the deeper _ H. Ünlük, ‘‘Deep Learning researches in Turkey: An
[11] Z. Pala, V. Yamli and I.
convolutional layers filters. The ones belonging to a network academic approach,” 2017 XIIIth International Conference on Perspective
trained with retinal fundus images are much simpler than the ones Technologies and Methods in MEMS Design (MEMSTECH), Lviv, 2017, pp. 120-
obtained from Image net dataset. This article briefly brings out the 123.
[12] A. Hasan, O. Kalıpsız and S. Akyokusß, ‘‘Predicting financial market in big data:
various fields where deep learning can be considered. The advan- Deep learning,” 2017 International Conference on Computer Science and
tage of deep learning algorithm is particular in the field of medical Engineering (UBMK), Antalya, 2017, pp. 510-515.

5
Santoshachandra Rao Karanam, Y. Srinivas and M. Vamshi Krishna Materials Today: Proceedings xxx (xxxx) xxx

[13] J. Maggu and A. Majumdar, ‘‘Greedy deep transform learning,” 2017 IEEE Further Reading
International Conference on Image Processing (ICIP), Beijing, 2017, pp. 1822-
1826.
[1] Y. Yuan, L. Mou, X. Lu, Scene Recognition by Manifold Regularized Deep
[14] J. Wang and D. Pei, ‘‘Kernel-based deep learning for intelligent data analysis,”
Learning Architecture, IEEE Transactions on Neural Networks and Learning
2017 First International Conference on Electronics Instrumentation &
Systems 26 (10) (Oct. 2015) 2222–2233.
Information Systems (EIIS), Harbin, 2017, pp. 1-5.

View publication stats