Deep Learning and

Parallel Computing
Environment for
Bioengineering
Systems
Deep Learning and
Parallel Computing
Environment for
Bioengineering
Systems

Edited by
Arun Kumar Sangaiah
Elsevier
3251 Riverport Lane
St. Louis, Missouri 63043

Deep Learning and Parallel Computing Environment for Bioengineering Systems ISBN: 978-0-12-816718-2
Copyright © 2019 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photo-
copying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how
to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the
Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: [Link]/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted
herein).

Notices

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information,
methods, compounds or experiments described herein. Because of rapid advances in the medical sciences, in particular, independent
verification of diagnoses and drug dosages should be made. To the fullest extent of the law, no responsibility is assumed by Elsevier,
authors, editors or contributors for any injury and/or damage to persons or property as a matter of products liability, negligence or
otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Publisher: Mara Conner

Acquisition Editor: Sonnini R. Yura
Editorial Project Manager: Thomas Van Der Ploeg
Production Project Manager: Nirmala Arumugam
Designer: Mark Rogers
List of Contributors

S.P. Abirami, ME Shaik Abdul Khalandar Basha, MTech

Department of Computer Science and Engineering School of Information Technology and Engineering
Coimbatore Institute of Technology Vellore Institute of Technology (VIT)
Coimbatore, Tamil Nadu, India Vellore, Tamil Nadu, India
Sandhya Armoogum, PhD Syed Muzamil Basha, MTech
Department of Industrial Systems Engineering School of Computer Science and Engineering
School of Innovative Technologies & Engineering Vellore Institute of Technology (VIT)
University of Technology Mauritius Vellore, Tamil Nadu, India
Pointe-Aux-Sables, Mauritius
Pralhad Gavali, ME
R. Arun, ME RIT
Department of Computer Science Islampur, Maharashtra, India
Builders Engineering College
Kangayam, Tamil Nadu, India Alireza Goli, ME
Department of Industrial Engineering
Erfan Babaee Tirkolaee, ME Yazd University
Department of Industrial Engineering Yazd, Iran
Mazandaran University of Science & Technology
Babol, Iran R. Karthick, BOT
Steps Rehabilitation Center (I)
C. Bagavathi, MTech Steps Groups
Department of ECE Coimbatore, Tamil Nadu, India
Government College of Technology
Coimbatore, Tamil Nadu, India Ramgopal Kashyap, PhD

Mani Bakhshi, ME G. Kousalya, ME, PhD

Department of Industrial Engineering Department of Computer Science and Engineering
Isfahan University of Technology Coimbatore Institute of Technology
Isfahan, Iran Coimbatore, Tamil Nadu, India

K. Balaji, ME K. Lavanya, PhD

School of Computer Science and Engineering School of Computer Science and Engineering
Vellore Institute of Technology (VIT) Vellore Institute of Technology (VIT)
Vellore, Tamil Nadu, India Vellore, Tamil Nadu, India

Balakrishnan, ME, PhD XiaoMing Li, PhD

Department of Computer Science and Engineering School of Computer Science & Technology
Vellore Institute of Technology (VIT) Tianjin University
Vellore, Tamil Nadu, India Tianjin, China

J. Saira Banu, PhD M. Madiajagan, MS, PhD

Vellore Institute of Technology (VIT) Vellore Institute of Technology (VIT)
Vellore, Tamil Nadu, India Vellore, Tamil Nadu, India

v
vi LIST OF CONTRIBUTORS

Ankit Sandeep Malhotra, BE Dharmendra Singh Rajput, PhD

School of Electrical Engineering School of Information Technology and Engineering
Vellore Institute of Technology (VIT) Vellore Institute of Technology (VIT)
Vellore, Tamil Nadu, India Vellore, Tamil Nadu, India
Geethu Mohan, ME Arun Kumar Sangaiah, PhD
School of Electronics Engineering School of Computing Science and Engineering
Vellore Institute of Technology (VIT) Vellore Institute of Technology (VIT)
Vellore, Tamil Nadu, India Vellore, Tamil Nadu India
R. Mohanasundaram, PhD O. Saraniya, ME, PhD
School of Computing Science and Engineering Department of ECE
Vellore Institute of Technology (VIT) Government College of Technology
Vellore, Tamil Nadu, India Coimbatore, Tamil Nadu, India

T.M. Navamani, ME, PhD G. Sreeja, ME

School of Computer Science and Engineering Department of ECE
Vellore Institute of Technology (VIT) Government College of Technology
Vellore, Tamil Nadu, India Coimbatore, Tamil Nadu, India

P.S. Periasamy, PhD M. Monica Subashini, PhD

Department of ECE School of Electrical Engineering
K.S.R College of Engineering Vellore Institute of Technology (VIT)
Thiruchengode, Tamil Nadu, India Vellore, Tamil Nadu, India

S. Sridhar Raj, BTech, MTech Durai Raj Vincent, PhD

Pondicherry University School of Information Technology and Engineering
Kalapet, Puducherry, India Vellore Institute of Technology (VIT)
Vellore, Tamil Nadu, India
Preface

Deep machine learning is an emergent area in the field and applications of deep learning approaches applied
of computational intelligence (CI) research that is con- to parallel computing environment in bioengineering
cerned with the analysis and design of learning algo- systems. Presently, there are many noteworthy issues
rithms, representations of data, at multiple levels of ab- (health informatics, bio-image informatics energy effi-
straction. Deep learning is a technique for implement- ciency, etc.) that need to be addressed in the context
ing machine learning that provides an effective solution of deep machine learning, parallel computing and bio-
for parallel computing environment in bi-engineering engineering. For the aforementioned reasons, this book
problems. It encompasses artificial intelligence (AI), focuses on addressing a comprehensive nature of cogni-
artificial neural network, reasoning, natural language tive neural computing, parallel computing and on em-
processing that will be helpful to human intelligence phasizing its character in human intelligence and learn-
and decision making process. The heterogeneous par-
ing systems, complex analysis tasks mimicking human
allel computing architectures have been significant for
cognition and learning behavior, prediction and control
real-time bio-engineering applications that needed a de-
of bio-engineering systems. This book intends to give an
sign of a high-level operating system for matching the
processing tasks to the appropriate machine learning overview of state-of-the-art of issues and solution guide-
paradigm in a mixed-machine parallel system. This ef- lines in the new era of deep machine learning paradigm
fort is exerted to investigate the feasibility of a deep ma- and its recent trends of techniques for bioengineering.
chine learning technique for implementing a high-level
operating system for heterogeneous parallel computers.
The new frontier research era and convergence of ORGANIZATION OF THE BOOK
deep machine learning and parallel computing with ref- The volume is organized into 15 chapters. A brief de-
erence to bio-engineering has three main streams need- scription of each chapter is given as follows:
ing to be addressed in the current scenario: bioinfor- Chapter 1 illustrates the parallel processing basic
matics, medical imaging, and sustainable engineering. concepts with examples in order to highlight the sig-
This book is integrating machine learning, cognitive nificance of parallel deep learning. The types of par-
neural computing, parallel computing paradigms, ad- allelization technique are addressed, and the relation
vanced data analytics and optimization opportunities between computational intelligence and parallel deep
to bring more compute to the bio-engineering problems learning, the challenges in combining them together
and challenges. Further, it is important to make a note (parallel computing, graphics processing unit and new
that convergence of parallel computing architectures,
hardware for deep learning in computational intelli-
deep machine learning and its intelligence techniques
gence research) and benefits are discussed in this chap-
has not been adequately investigated from the perspec-
ter.
tive of bioengineering research streams (bioinformatics,
Chapter 2 presents the big data analytics with re-
medical imaging, and sustainable engineering) and its
related research issues. Obviously, these challenges also gards to the Hadoop Big Data framework for storing and
create immense opportunities for researchers. processing big data, described in the context of bioinfor-
The book will present novel, in-depth and funda- matics. The authors have highlighted the importance of
mental research contributions either from a method- the machine learning approach for performing predic-
ological or application perspective in understanding tive and prescriptive analytics. Thus, machine and deep
the fusion of deep machine learning paradigms and learning approaches currently being used in the context
their capabilities in solving a diverse range of problems of big data analytics in the Hadoop framework have also
in bio-engineering and its real-world applications. The been presented, as well as the current uses of such tech-
overall objective of the book is to illustrate the state- niques and tools in bioinformatics are illustrated in this
of-the-art and recent developments in the new theories chapter.

vii
viii PREFACE

Chapter 3 deals with the survey of image fusion learning algorithms for real time medical imaging data
algorithms based on deep convolutional neural net- sets and justified their profound impact on the medical
work, and the results obtained by these methods are field.
interpreted and discussed. The chapter authors have ad- Chapter 9 describes the role of machine learning al-
dressed the significance in combining the outcomes of gorithms on both linear and nonlinear data in address-
different modalities to utilize the complementary infor- ing regression and classification problems. The results
mation from each modality to form a better image. With obtained in this chapter are applicable to address the
image fusion, the multi-sensor data with complemen- real time problems like classification and regression.
tary information about the particular region are com- The chapter results state that support vector machine
paratively analyzed in this chapter. (SVM) performs better than all other classification al-
Chapter 4 illustrates the necessity of integrating ma- gorithms and the neural network (NN) approach gives
chine and deep learning methodology with the diagno- the lowest mean squared error (MSE) in the regression
sis of brain tumor, and recent segmentation and classifi- problem.
cation techniques on magnetic resonance images (MRI) The main objective of Chapter 10 is to consolidate
are reviewed. This chapter addressed the current trends the benefits of the classifications using singular value
in the grading of brain tumor with a focus on gliomas decomposition (SVD-QR) and limited memory sub-
which include astrocytoma. The current state-of-the-art, space optimization SVD (LMSVD-QR) calculations for
software packages, evaluation and validation metrics preprocessing of profound learning in multilayer neu-
used in different approaches are discussed, along with ral systems. This chapter has indicated why singular
integration into the clinical environment. value decomposition (SVD)-QR calculation is required
Chapter 5 provides the essentials of deep learning for preprocessing of profound learning for vast scale in-
methods with convolutional neural networks and an- formation input.
alyzes their achievements in medical image analysis Chapter 11 presents the challenges in storing and
such as deep feature representation, detection, segmen- processing big data using Hadoop and Spark. The au-
tation, classification, and prediction. This chapter re- thors have highlighted the new analytical platforms
views the different deep learning convolution neural such as Hadoop, Spark, along with MapReduce pro-
network methods. The features, benefits, and applica- gramming. The objective of this chapter is to make the
tions of convolutional neural network methods are also readers understand the challenges in storing and pro-
discussed in this chapter. cessing big data and how to use different big data frame-
Chapter 6 investigated how deep learning could be works effectively to store and process big data.
applied to the classification of images on the CIFAR- Chapter 12 presents a novel mixed-integer linear
10 database. The chapter authors have specified deep programming (MILP) model to consider a location
learning technologies that are becoming more acces- routing problem (LRP) for multiple perishable prod-
sible for corporations and individuals and give better ucts with vehicles having multiple trips, intermediate
results than the convolution neuron network. In this depots, and soft time windows. To cope with the solu-
chapter, deep convolutional neural networks are used tion complexity of the problem, an efficient biography
for classification and GPU technology is used for paral- based optimization algorithm (BBO) is investigated in
lel processing. this chapter.
Chapter 7 discusses the basic deep learning network Chapter 13 gives a brief overview of evolutionary
models and outlines some of the applications in health procedures, systolic arrays and methods to transform an
informatics. In this chapter, biomedical data can be ef- iterative algorithm into architecture. Significance of pa-
ficiently processed by deep learning networks, which in rameters derived from GTDM is mentioned, and the pa-
turn increase the predictive power for many specific ap- rameters involved in selecting the best of addressed al-
plications in the health informatics domain. Thus, this gorithms are clearly justified. The chapter authors have
chapter highlights that deep learning algorithms can revealed that ant colony optimization performed the
provide better outcomes and prediction in health infor- best among the selected evolutionary algorithms in ad-
matics with the integration of advanced parallel proces- dressing a systolic array mapping of grey tone distribu-
sors. tion matrix (GTDM) computation.
Chapter 8 illustrates the role of deep learning and The ultimate aim of Chapter 14 is to design a com-
semi-supervised and transfer learning algorithms for plete combinatorial model for the results from vari-
medical imaging. The chapter authors have used classifi- ous screening experiments involving multimodal deep
cation of supervised, semi-supervised and unsupervised learning technique that projects into better solution of
 PREFACE ix

autism identification. This chapter mainly focuses on plied to parallel computing environment in bioengi-
the emotional sequence identification of children who neering systems. The book aims to present concepts
are autism spectrum disorder (ASD) positive and ASD and technologies that are successfully used in the im-
negative (i.e., normal TD). plementation of today’s intelligent data-centric critical
Chapter 15 gives parallel machine learning and deep systems and multimedia cloud big data, having a good
learning approaches for bioinformatics. The authors chance to be used in future computing systems. The
have outlined the deep learning and other deep-based book will constitute teaching material for organizing
representative learning algorithms which have been the course titled Computational Intelligence for New
applied successfully in image understanding, speech Computing Environments, hence suitable for university
recognition, and text classification, etc. level courses as well as research scholars.

Arun Kumar Sangaiah

AUDIENCE School of Computing Science and Engineering
The overall objective of the book is to illustrate the Vellore Institute of Technology
state-of-the-art and recent developments in the new the- Vellore, Tamil Nadu, India
ories and applications of deep learning approaches ap-
Foreword

This book delivers a significant forum for the technical ploring the significance of deep learning systems and
advancement of deep learning in parallel computing en- bio-engineering in the next paradigm of computing.
vironment across bio-engineering diversified domains This book gives an intensive and in-depth cover-
and its applications. Pursuing an interdisciplinary ap- age of the use of deep learning in the field of bio-
proach, it focuses on methods used to identify and ac- engineering systems and various interesting findings.
quire valid, potentially useful knowledge sources. Man- This book is a significant step in this field’s maturation
aging the gathered knowledge and applying it to mul- and will serve to unify, advance, and challenge the scien-
tiple domains including health care, social networks, tific community in many important ways. In addition,
mining, recommendation systems, image processing, this book is well suited for researchers exploring the sig-
pattern recognition and predictions using deep learning nificance of deep-learning systems and bio-engineering.
This book integrates in fact the core ideas of deep learn-
paradigms is the major strength of this book. Effective
ing and its applications in bio-engineering application
data and knowledge management has become a key
domains, to be accessible to all scholars and academi-
to the success of engineering applications and business
cians. The proposed techniques and concepts in this
organizations that can offer a substantial competitive
book can be extended in the future to accommodate
edge. changing business organizations’ needs, as well as prac-
The book “Deep Learning and Parallel Computing titioners’ innovative ideas.
Environment for Bioengineering Systems” is focusing on I am pleased to appreciate the editors and authors on
domain experts and developers, who want to under- their accomplishment, and hope that the readers will
stand and explore the application of deep learning find the book worthwhile and a source of inspiration in
and computational intelligence aspects (opportunities their research and professional activity.
and challenges) for the design and development of
parallel computing environment in the context of bio- Prof. Vincenzo Piuri, PhD
engineering systems era and their related applications IEEE Fellow
such as smarter health care, homeland security, com- Department of Computer Science
putational biology, robotics, and intelligent assistance. University of Milan
This book is a significant collection of 15 chapters ex- Milan, Italy

xi
Acknowledgment

The editors would like to recognize the help of all the of the reviewers who helped us to refine the context of
people involved in this project and especially the au- this book. Most of the authors also served as referees;
thors and reviewers that took part in the peer review we highly appreciate their double task.
process. Without their support, this book would not Finally, our gratitude goes to all of our friends and
have become a reality. colleagues, who were so generous with their encourage-
First, the editors would like to thank each of the au- ment, advice and support.
thors for their contributions. Our sincere gratitude goes
to the chapter’s authors who contributed their time and
expertise to this book. Arun Kumar Sangaiah
Second, the editors wish to acknowledge the valu- School of Computing Science and Engineering
able contributions of the reviewers regarding the im- Vellore Institute of Technology
provement of quality, coherence, and content presen- Vellore, Tamil Nadu, India
tation of chapters. We deeply appreciate the comments

xiii
Contents

L I S T O F C O N T R I B U T O R S, v 1.3.1 Hardware Architecture of CPU

P R E F A C E, vii and GPU, 7
F O R E W O R D, xi 1.3.2 Suitability of GPU to Parallel
A C K N O W L E D G M E N T, xiii Deep Learning, 7
1.3.3 CPU vs. GPU, 8
1 Parallel Computing, 1.3.4 Advantages of Using GPU in
Graphics Processing Unit (GPU) and Parallel Deep Learning, 8
New Hardware for Deep Learning 1.3.5 Disadvantages of Using GPU
in Computational Intelligence in Parallel Deep Learning, 8
Research, 1 1.3.6 Famous GPUs on the
M. Madiajagan, MS, PhD, Market, 9
S. Sridhar Raj, BTech, MTech 1.4 GPU Based Parallel Deep Learning