Lecture Notes in Electrical Engineering 1009

Hariharan Muthusamy
János Botzheim
Richi Nayak Editors

Robotics,
Control and
Computer
Vision
Select Proceedings of ICRCCV 2022
Lecture Notes in Electrical Engineering

Volume 1009

Series Editors

Leopoldo Angrisani, Department of Electrical and Information Technologies Engineering, University of Napoli
Federico II, Naples, Italy
Marco Arteaga, Departament de Control y Robótica, Universidad Nacional Autónoma de México, Coyoacán,
Mexico
Bijaya Ketan Panigrahi, Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi,
Delhi, India
Samarjit Chakraborty, Fakultät für Elektrotechnik und Informationstechnik, TU München, Munich, Germany
Jiming Chen, Zhejiang University, Hangzhou, Zhejiang, China
Shanben Chen, School of Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai, China
Tan Kay Chen, Department of Electrical and Computer Engineering, National University of Singapore, Singapore,
Singapore
Rüdiger Dillmann, Humanoids and Intelligent Systems Laboratory, Karlsruhe Institute for Technology, Karlsruhe,
Germany
Haibin Duan, Beijing University of Aeronautics and Astronautics, Beijing, China
Gianluigi Ferrari, Università di Parma, Parma, Italy
Manuel Ferre, Centre for Automation and Robotics CAR (UPM-CSIC), Universidad Politécnica de Madrid,
Madrid, Spain
Sandra Hirche, Department of Electrical Engineering and Information Science, Technische Universität München,
Munich, Germany
Faryar Jabbari, Department of Mechanical and Aerospace Engineering, University of California, Irvine, CA, USA
Limin Jia, State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing, China
Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland
Alaa Khamis, German University in Egypt El Tagamoa El Khames, New Cairo City, Egypt
Torsten Kroeger, Stanford University, Stanford, CA, USA
Yong Li, Hunan University, Changsha, Hunan, China
Qilian Liang, Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX, USA
Ferran Martín, Departament d’Enginyeria Electrònica, Universitat Autònoma de Barcelona, Bellaterra, Barcelona,
Spain
Tan Cher Ming, College of Engineering, Nanyang Technological University, Singapore, Singapore
Wolfgang Minker, Institute of Information Technology, University of Ulm, Ulm, Germany
Pradeep Misra, Department of Electrical Engineering, Wright State University, Dayton, OH, USA
Sebastian Möller, Quality and Usability Laboratory, TU Berlin, Berlin, Germany
Subhas Mukhopadhyay, School of Engineering and Advanced Technology, Massey University,
Palmerston North, Manawatu-Wanganui, New Zealand
Cun-Zheng Ning, Department of Electrical Engineering, Arizona State University, Tempe, AZ, USA
Toyoaki Nishida, Graduate School of Informatics, Kyoto University, Kyoto, Japan
Luca Oneto, Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova,
Genova, Genova, Italy
Federica Pascucci, Dipartimento di Ingegneria, Università degli Studi Roma Tre, Roma, Italy
Yong Qin, State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing, China
Gan Woon Seng, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore,
Singapore
Joachim Speidel, Institute of Telecommunications, Universität Stuttgart, Stuttgart, Germany
Germano Veiga, Campus da FEUP, INESC Porto, Porto, Portugal
Haitao Wu, Academy of Opto-electronics, Chinese Academy of Sciences, Beijing, China
Walter Zamboni, DIEM—Università degli studi di Salerno, Fisciano, Salerno, Italy
Junjie James Zhang, Charlotte, NC, USA
The book series Lecture Notes in Electrical Engineering (LNEE) publishes the
latest developments in Electrical Engineering—quickly, informally and in high
quality. While original research reported in proceedings and monographs has
traditionally formed the core of LNEE, we also encourage authors to submit books
devoted to supporting student education and professional training in the various
fields and applications areas of electrical engineering. The series cover classical and
emerging topics concerning:
• Communication Engineering, Information Theory and Networks
• Electronics Engineering and Microelectronics
• Signal, Image and Speech Processing
• Wireless and Mobile Communication
• Circuits and Systems
• Energy Systems, Power Electronics and Electrical Machines
• Electro-optical Engineering
• Instrumentation Engineering
• Avionics Engineering
• Control Systems
• Internet-of-Things and Cybersecurity
• Biomedical Devices, MEMS and NEMS
For general information about this book series, comments or suggestions, please
contact [email protected].
To submit a proposal or request further information, please contact the Publishing
Editor in your country:
China
Jasmine Dou, Editor ([email protected])
India, Japan, Rest of Asia
Swati Meherishi, Editorial Director ([email protected])
Southeast Asia, Australia, New Zealand
Ramesh Nath Premnath, Editor ([email protected])
USA, Canada
Michael Luby, Senior Editor ([email protected])
All other Countries
Leontina Di Cecco, Senior Editor ([email protected])
** This series is indexed by EI Compendex and Scopus databases. **
Hariharan Muthusamy · János Botzheim ·
Richi Nayak
Editors

Robotics, Control
and Computer Vision
Select Proceedings of ICRCCV 2022
Editors
Hariharan Muthusamy János Botzheim
National Institute of Technology Eötvös Loránd University
Uttarakhand Budapest, Hungary
Srinagar, India

Richi Nayak
Queensland University of Technology
Brisbane, QLD, Australia

ISSN 1876-1100 ISSN 1876-1119 (electronic)

Lecture Notes in Electrical Engineering
ISBN 978-981-99-0235-4 ISBN 978-981-99-0236-1 (eBook)
https://doi.org/10.1007/978-981-99-0236-1

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Singapore Pte Ltd. 2023
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Contents

Computer Vision
Challenges and Opportunity for Salient Object Detection
in COVID-19 Era: A Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Vivek Kumar Singh and Nitin Kumar
Human Activity Recognition Using Deep Learning . . . . . . . . . . . . . . . . . . . 15
Amrit Raj, Samyak Prajapati, Yash Chaudhari,
and Ankit Kumar Rouniyar
Recovering Images Using Image Inpainting Techniques . . . . . . . . . . . . . . . 27
Soureesh Patil, Amit Joshi, and Suraj Sawant
Literature Review for Automatic Detection and Classification
of Intracranial Brain Hemorrhage Using Computed
Tomography Scans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Yuvraj Singh Champawat, Shagun, and Chandra Prakash
A Pilot Study for Profiling Diabetic Foot Ulceration Using Machine
Learning Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Irena Tigga, Chandra Prakash, and Dhiraj
A Deep Learning Approach for Gaussian Noise-Level Quantification . . . 81
Rajni Kant Yadav, Maheep Singh, and Sandeep Chand Kumain
Performance Evaluation of Single Sample Ear Recognition Methods . . . 91
Ayush Raj Srivastava and Nitin Kumar
AI-Based Real-Time Monitoring for Social Distancing Against
COVID-19 Pandemic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Alok Negi, Krishan Kumar, Prachi Chauhan, Parul Saini,
Shamal Kashid, and Ashray Saini

v
vi Contents

Human Activity Recognition in Video Sequences Based

on the Integration of Optical Flow and Appearance of Human
Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Arati Kushwaha and Ashish Khare
Multi-agent Task Assignment Using Swap-Based Particle Swarm
Optimization for Surveillance and Disaster Management . . . . . . . . . . . . . . 127
Mukund Subhash Ghole, Arabinda Ghosh, and Anjan Kumar Ray
Facemask Detection and Maintaining Safe Distance Using AI
and ML to Prevent COVID-19—A Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Ankita Mishra, Piyali Paul, Koyel Mondal, and Sanjay Chakraborty
A Machine Learning Framework for Breast Cancer Detection
and Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Bagesh Kumar, Pradumna Tamkute, Kumar Saurabh,
Amritansh Mishra, Shubham Kumar, Aayush Talesara, and O. P. Vyas
Vision Transformers for Breast Cancer Classification
from Thermal Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Lalit S. Garia and M. Hariharan
An Improved Fourier Transformation Method for Single-Sample
Ear Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Ayush Raj Srivastava and Nitin Kumar
Driver Drowsiness Detection for Road Safety Using Deep Learning . . . . 197
Parul Saini, Krishan Kumar, Shamal Kashid, Alok Negi,
and Ashray Saini
Performance Evaluation of Different Machine Learning Models
in Crop Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Amit Bhola and Prabhat Kumar
Apriori Based Medicine Recommendation System . . . . . . . . . . . . . . . . . . . . 219
Indrashis Mitra, Souvik Karmakar, Kananbala Ray, and T. Kar
NPIS: Number Plate Identification System . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Ashray Saini, Krishan Kumar, Alok Negi, Parul Saini,
and Shamal Kashid
Leveraging Advanced Convolutional Neural Networks
and Transfer Learning for Vision-Based Human Activity
Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
Prachi Chauhan, Hardwari Lal Mandoria, Alok Negi, Krishan Kumar,
Amitava Choudhury, and Sanjay Dahiya
Contents vii

Control Techniques and Their Applications

Real Power Loss Reduction by Chaotic Based Riodinidae
Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Lenin Kanagasabai
5G Enabled IoT Based Automatic Industrial Plant Monitoring
System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Kshitij Shinghal, Amit Saxena, Amit Sharma, and Rajul Misra
Criterion to Determine the Stability of Systems with Finite
Wordlength and Delays Using Bessel-Legendre Inequalities . . . . . . . . . . . 271
Rishi Nigam and Siva Kumar Tadepalli
Adaptive Control for Stabilization of Ball and Beam System Using
H∞ Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Sudhir Raj
Optimal Robust Controller Design for a Reduced Model AVR
System Using CDM and FOPIλ Dμ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Manjusha Silas and Surekha Bhusnur
Neural Network Based DSTATCOM Control for Power Quality
Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Islavatu Srikanth and Pradeep Kumar
An Extensive Critique on FACTS Controllers and Its Utilization
in Micro Grid and Smart Grid Power Systems . . . . . . . . . . . . . . . . . . . . . . . 323
D. Sarathkumar, Albert Alexander Stonier, and M. Srinivasan
Arctangent Framework Based Least Mean Square/Fourth
Algorithm for System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Soumili Saha, Ansuman Patnaik, and Sarita Nanda

Robotics and Autonomous Vehicles

Stabilization of Ball Balancing Robots Using Hierarchical Sliding
Mode Control with State-Dependent Switching Gain . . . . . . . . . . . . . . . . . . 345
Sudhir Raj
Programmable Bot for Multi Terrain Environment . . . . . . . . . . . . . . . . . . . 357
K. R. Sudhindra, H. H. Surendra, H. R. Archana, and T. Sanjana
A Computer Vision Assisted Yoga Trainer for a Naive Performer
by Using Human Joint Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
Ritika Sachdeva, Iresha Maheshwari, Vinod Maan, K. S. Sangwan,
Chandra Prakash, and Dhiraj
Study of Deformation in Cold Rolled Al Sheets . . . . . . . . . . . . . . . . . . . . . . . 387
János György Bátorfi and Jurij J. Sidor
viii Contents

Modelling and Control of Semi-automated Microfluidic Dispensing

System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
M. Prabhu, P. Karthikeyan, D. V. Sabarianand, and N. Dhanawaran
Im-SMART: Developing Immersive Student Participation
in the Classroom Augmented with Mobile Telepresence Robot . . . . . . . . . 407
Rajanikanth Nagaraj Kashi, H. R. Archana, and S. Lalitha
Architecture and Algorithms for a
Pixhawk-Based Autonomous Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Ankur Pratap Singh, Anurag Gupta, Amit Gupta, Archit Chaudhary,
Bhuvan Jhamb, Mohd Sahil, and Samir Saraswati
3D Obstacle Detection and Path Planning for Aerial Platform
Using Modified DWA Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
Ankur Pratap Singh, Amit Gupta, Bhuvan Jhamb,
and Karimulla Mohammad
Vibration Suppression of Hand Tremor Using Active Vibration
Strategy: A Numerical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
Anshul Sharma and Rajnish Mallick
Design of a Self-reconfigurable Robot with Roll, Crawl, and Climb
Features for False Ceiling Inspection Task . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
S. Selvakumaran, A. A. Hayat, K. Elangovan, K. Manivannan,
and M. R. Elara

Smart Technologies for Mobility and Healthcare

Review Paper on Joint Beamforming, Power Control
and Interference Coordination for Non-orthogonal Multiple
Access in Wireless Communication Networks for Efficient Data
Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
Leela Siddiramlu Bitla and Chandrashekhar Sakode
3D Reconstruction Methods from Multi-aspect TomoSAR Method:
A Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
Nazia Akhtar, Tamesh Haldar, Arindam Basak, Arundhati Misra Ray,
and Debashish Chakravarty
Security and Privacy in IoMT-Based Digital Health care: A Survey . . . . 505
Ashish Singh, Riya Sinha, Komal, Adyasha Satpathy, and Kannu Priya
5G Technology-Enabled IoT System for Early Detection
and Prevention of Contagious Diseases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
Amit Saxena, Kshitij Shinghal, Rajul Misra, and Amit Sharma
Contents ix

A Brief Review of Current Smart Electric Mobility Facilities

and Their Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Darbhamalla Satya Sai Surya Varun, Tamesh Halder, Arindam Basak,
and Debashish Chakravarty
Gold-ZnO Coated Surface Plasmon Resonance Refractive
Index Sensor Based on Photonic Crystal Fiber with Tetra Core
in Hexagonal Lattice of Elliptical Air Holes . . . . . . . . . . . . . . . . . . . . . . . . . . 567
Amit Kumar Shakya and Surinder Singh
Fault Detection and Diagnostics in a Cascaded Multilevel Inverter
Using Artificial Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
Stonier Albert Alexander, M. Srinivasan, D. Sarathkumar, and R. Harish
Identification of Multiple Solutions Using Two-Step Optimization
Technique for Two-Level Voltage Source Inverter . . . . . . . . . . . . . . . . . . . . . 589
M. Chaitanya Krishna Prasad, Vinesh Agarwal, and Ashish Maheshwari
A Review on Recent Trends in Charging Stations for
Electric Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
Vinaya Chavan Thombare, Kshitij Nerlekar, and Juhi Mankumbare
IoT-Based Vehicle Charging Eco System for Smart Cities . . . . . . . . . . . . . . 611
N. Dinesh Kumar and F. B. Shiddanagouda
About the Editors

Hariharan Muthusamy received a Ph.D. in Mechatronic Engineering (2010) from

the University of Malaysia Perlis (UniMAP), Malaysia, a Master of Engineering in
Applied Electronics (2006) from the Government College of Technology, India, and
a Bachelor of Engineering in Electrical and Electronics Engineering (2002) from
Government College of Technology (Affiliated to Bharathiar University), India. He
is an Associate Professor in the Department of Electronics Engineering, National
Institute of Technology Uttarakhand, India. He has published over 150 papers in
refereed journals and conference proceedings. His major research interests include
speech signal processing, biomedical signal and image processing, machine learning,
deep learning, and optimization algorithms. He has supervised 9 Ph.D. and 4 Masters
(research) students in the field of his expertise.

János Botzheim earned his M.Sc. and Ph.D. degrees from the Budapest Univer-
sity of Technology and Economics in 2001 and 2008, respectively. He joined the
Department of Automation at Szechenyi Istvan University, Gyor, Hungary in 2007
as a senior lecturer, in 2008 as an assistant professor, and in 2009 as an associate
professor. He was a visiting researcher at the Graduate School of System Design at
the Tokyo Metropolitan University from September 2010 to March 2011 and from
September 2011 to February 2012. He was an associate professor in the Graduate
School of System Design at the Tokyo Metropolitan University from April 2012
to March 2017. He was an associate professor in the Department of Mechatronics,
Optics, and Mechanical Engineering Informatics at the Budapest University of Tech-
nology and Economics from February 2018 to August 2021. He is the Head of the
Department of Artificial Intelligence at Eötvös Loránd University, Faculty of Infor-
matics, Budapest, Hungary, since September 2021. His research interest areas are
computational intelligence, automatic identification of fuzzy rule-based models and
some neural network models, bacterial evolutionary algorithms, memetic algorithms,
applications of computational intelligence in robotics, and cognitive robotics. He has
about 180 papers in journals and conference proceedings.

xi
xii About the Editors

Richi Nayak is the Leader of the Applied Data Science Program at the Centre for
Data Science and a Professor of Computer science at Queensland University of
Technology, Brisbane Australia. She has a driving passion to address pressing soci-
etal problems by innovating the Artificial Intelligence field underpinned by funda-
mental research in machine learning, data mining, and text mining. Her research has
resulted in the development of novel solutions to address industry-specific problems
in Marketing, K 12 Education, Agriculture, Digital Humanities, and Mining. She
has made multiple advances in social media mining, deep neural networks, multi-
view learning, matrix/tensor factorization, clustering, and recommender systems.
She has authored over 180 high-quality refereed publications. Her research leader-
ship is recognized by multiple best paper awards and nominations at international
conferences, QUT Postgraduate Research Supervision awards, and the 2016 Women
in Technology (WiT) Infotech Outstanding Achievement Award in Australia. She
holds a Ph.D. in Computer Science from the Queensland University of Technology
and a Master in Engineering from IIT Roorkee.
Computer Vision
Challenges and Opportunity for Salient
Object Detection in COVID-19 Era:
A Study

Vivek Kumar Singh and Nitin Kumar

1 Introduction

Humans have the ability to identify visually informative scene regions in the image
effortlessly and rapidly based on perceived distinctive features. These filtered regions
contain rich information about objects depicted in an image. Salient Object Detec-
tion (SOD) aims to highlight important objects or regions and suppress background
regions in the image. SOD methods transform an input image into a probability map
called saliency map [1] that expresses how much each image element (pixel/region)
grabs human attention. An example of salient object detection is illustrated in Fig. 1.
Salient Object Detection (SOD) has been widely applied as pre-processing step in
computer vision applications such as object detection [4, 5], video summarization [6],
and image retrieval [7].
Coronavirus disease (COVID-19) is an infectious disease [8–10] which has posed
several challenges to salient object detection, for example, due to use of face mask,
face detection performance is decreased. Diffusion of the disease has been occurring
from person to person quickly in the world. The disease is called COVID-19 and the
virus is denoted as SARS-CoV-2 which is a family of viruses effective for devolving
acute respiratory syndrome. COVID-19 common clinical features are fever, dyspnea,
cough, myalgia, and headache [11]. The most common diagnosis tool used for diag-
nosis of COVID-19 is the reverse-transcription polymerase chain reaction (RT-PCR).
Further, chest radiological imaging including computed tomography (CT) and X-ray

V. Kumar Singh (B)

Sharda University, Greater Noida, India
e-mail: [email protected]
N. Kumar
National Institute of Technology, Uttarakhand, India
e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 3
H. Muthusamy et al. (eds.), Robotics, Control and Computer Vision, Lecture Notes
in Electrical Engineering 1009, https://doi.org/10.1007/978-981-99-0236-1_1
4 V. Kumar Singh and N. Kumar

Fig. 1 An example of salient object detection process, a input image, b saliency map [3], and
c ground truth

Fig. 2 A motivational example of this study, a input image, b saliency map obtained from Graph-
Based Manifold Ranking (GMR) [31] method, and c ground truth

is playing important role in the early diagnosis and treatment of this disease [12].
Researchers are looking for detecting infected patients through medical image pro-
cessing like X-rays and CT scans [13]. COVID-19 is a pandemic virus that infected
many people worldwide and continues spreading from person to person. The disease
also affected the lifestyle of humans such as education, office work, transportation,
economic actives, etc. Therefore, our main motivation is to look at the impact of the
virus on salient object detection performance and the applicability of salient object
detection approach to control spreading of the virus. Figure 2 shows a motivational
example of this study. In this figure, input image contains a human with face mask,
in which saliency map does not highlight the masked region of the face. The purpose
of this research work is to analyze the effectiveness of saliency detection on the
images generated around the current human life activities. In this study, we propose
a dataset which use to validate our suggested challenges in salient object detection
due to COVID-19.
The rest of this paper is structured as follows. Section 2 illustrates the related
works on salient object detection methods and novel Coronavirus-2019 (COVID-
2019). In Sect. 3, a detailed discussion about the challenges and opportunities for
salient object detection in the COVID-19 era is presented. Suggested challenges are
evaluated and analyzed in Sect. 4. Finally, conclusion and future works are presented
in Sect. 5.
Challenges and Opportunity for Salient Object Detection in COVID-19 Era: A Study 5

2 Related Work

A large number of salient object detection methods have been reported in litera-
ture. These methods are broadly categorized into two categories: bottom-up meth-
ods and top-down methods. Bottom-up salient object detection methods utilize the
appearance contrasts between objects and their surrounding regions in the image.
The earliest bio-inspired bottom-up saliency method was proposed by Itti et al. [1].
This method has extracted three low-level visual features such as luminance, color,
and orientation and exploits center-surround mechanisms to compute the saliency
maps. Achanta et al. [14] proposed a simple and efficient saliency detection approach
that computes saliency value of each image pixel by subtracting the Gaussian
blurred version of the image from the mean pixel value of the image. Goferman
et al. [15] presented four principles, namely, local low-level features, global consid-
erations, visual organizational rules, and high-level factors to compute saliency maps.
Perazzi et al. [16] suggested a saliency detection method based on color contrast.
Cheng et al. [17] proposed a global contrast-based saliency computation approach
which utilizes Histogram-based Contrast (HC) and Region-based Contrast (RC) for
saliency estimation. Liu and Yang [18] proposed saliency detection method that
exploited color volume and perceptually uniform color differences and combined
foreground, center, and background saliency to obtain saliency map. Top-down
salient object detection methods calculate the saliency values with the help of high-
level priors. Gao et al. [19] computed saliency values of interest points by their
mutual information and extracted discriminant features. Yang et al. [20] proposed
a novel saliency detection method that jointly learned Conditional Random Field
(CRF) for generation of saliency map. Jiang et al. [21] suggested saliency estimation
method that effectively integrated shape prior into an iterative energy minimization
box. Recently, convolutional neural networks (CNNs) have drawn great attention of
computer vision researchers. Wang et al. [22] presented saliency detection method
that employed two different deep networks to compute the saliency maps. Wang et
al. [23] proposed the PAGE-Net for saliency calculation. Ren et al. [24] suggested
the CANet, which has combined high-level semantic and low-level boundary infor-
mation for salient object detection. Currently, computer vision and machine learning
approaches have been rapidly applied for Coronavirus disease-2019 (COVID-19)
detection. Ozturk et al. [25] proposed an automatic COVID-19 detection model that
exploited deep learning method to detect and classify COVID-19. Waheed et al. [26]
proposed an Auxiliary Classifier Generative Adversarial Network (ACGAN) called
CovidGAN which has produced synthetic chest X-ray (CXR) images. Fan et al. [27]
suggested a novel COVID-19 lung CT infection segmentation network called Inf-Net.
Zhou et al. [28] presented a fully automatic, rapid, accurate, and machine-agnostic
method for identifying the infection regions on CT scans. Wang et al. [29] suggested
a novel noise-robust framework to learn from noisy labels for the segmentation. A
summary of the recent research works for object detection during COVID-19 is given
in Table 1.
6 V. Kumar Singh and N. Kumar

Table 1 Recent research work for object detection during COVID-19

S. no. Authors Method Modality Remarks
1 Ozturk et al. [25] Deep learning Chest X-ray The model is fully
automated, it does
not required
manual features
extraction
2 Waheed et al. [26] Auxiliary Chest X-ray It is a powerful
Classifier method to generate
Generative unseen samples
Adversarial that can be utilized
Network to design effective
(ACGAN) and robust
convolutional
neural networks
(CNNs)
3 Fan et al. [27] Deep Network Lung computed The Inf-Net first
(Inf-Net ) tomography (CT) roughly located an
image infected region and
then exploit the
boundaries by
means of reverse
attention and edge
information for
accurately
identifying the
infected region
4 Zhou et al. [28] A fully automatic, CT scans The segmentation
rapid, accurate, method achieves a
and good trade-off
machine-agnostic between the
complexity of the
deep learning
model and the
accuracy of the
model
5 Wang et al. [29] A noise-robust CT image The method aims
framework for learning from
noisy labels for
COVID-19
pneumonia lesion
segmentation from
CT images where
clean labels are
difficult and
expensive to
acquire
Challenges and Opportunity for Salient Object Detection in COVID-19 Era: A Study 7

3 Challenges and Opportunity for SOD in COVID-19 Era

In this section, we present study of the impact of COVID-19 situation on identifying

the most significant regions at an early stage from the natural images. This study
provides scenarios in which images can be changed due to COVID-19 pandemic but
the target object is still unchanged for any object detection application. Our work
aims at studying this effect to enable the future salient object detection methods
addressing such scenarios.

3.1 Challenges

The first challenging scenario is the complexity of the image where the appearance
such as color and texture of foreground regions and background regions is similar.
This is a difficult scenario for salient object detection methods because several meth-
ods exploit color and texture as distinctive features for calculating saliency value to
each image element. Therefore, if foreground and background image regions have
similar features then the methods may fail to highlight salient regions and suppress
background regions. Secondly, saliency detection process is very challenging in real-
time images in which the target object is partially hidden by some other objects. This
scenario is known as occlusion problem in natural images. The saliency detection
methods may fail to identify object in the image which is partially blocked by other
objects.
Figure 3 shows various visual challenges of salient object detection in natural
images. Similar color and texture of foreground and background regions in the com-
plex natural images are shown in Fig. 3a. An owl is situated in a place where the
surrounding location is homogeneous to the owl, the saliency detection task faces
problem to identify owl bird from real-time image as shown in Fig. 3a. Partial occlu-
sion problem in real-time images is depicted in Fig. 3b. In a cow body, some regions
are blocked by wooden poles which is shown in Fig. 3b, images are taken from
PASCAL-S [30] dataset, and in this scene cow is target object to which salient
regions are identified, but the methods may detect it partially. Figure 2a illustrates
the effect of coronavirus on human real image. In this scene, a man is wearing a white
face mask that is not similar to the human face skin. It is a case of partial occlusion
where the human face is partially hidden by the face mask.
Moreover, the face mask shows high center-surrounding difference than the tar-
geted object (i.e., man). Hence, the salient object detection methods may identify the
face mask as an important object instead of the man. This is a challenge for salient
object detection methods to achieve better performance on the visual data generated
in COVID-19 era. The COVID-19 pandemic has affected appearance of real-time
visual images surrounding human life. For example, nowadays, people are wearing
Personal Protective Equipment (PPE) which includes a face mask, gloves, gowns,
head cover, shoe cover, etc. to safeguard them from COVID-19. All the images taken
8 V. Kumar Singh and N. Kumar

Fig. 3 Visual examples of

some challenging scenarios
in salient object detection.
Appearance similarity
between foreground and
background is illustrated in
(a) [30]. Partial occlusion
scenario in real-time images
is depicted in (b) [30]

Fig. 4 Example of some

people has appeared
together [30]

from public places captured the human face with blockages by face mask. This sit-
uation can be considered as an occlusion problem in the natural images. It poses
a challenge to computer vision applications and most of them fail to identify hid-
den face in the presence of a face mask. This is also challenging for salient object
detection methods to uniformly highlight the human face. In addition, these PPE can
visually appear similar to the surrounding environment in terms of color and texture.
Any object identification computer vision application can be easily misguided to
identify wrong objects in an image.
Further, COVID-19 has also affected the visual appearance of groups of people
due to the following social distancing in public places. On many occasions, people are
capturing group images as shown in Fig. 4, image is adopted from PASCAL-S [30]
dataset. In this image, all people are together to form an object and salient object
detection methods can easily detect it as a salient object. However, today in group
images, people are maintaining minimum defined distance which is popularly known
as social distancing. Such effects may degrade the performance of salient object
detection because the target object is all the people in the image, in contrast, saliency
detection methods may detect some people out of all the people who appeared in the
image. A summary of these challenges is also given in Table 2.
Challenges and Opportunity for Salient Object Detection in COVID-19 Era: A Study 9

Table 2 Challenges and opportunities for salient object detection in COVID-19 era
S. no. Challenges Reason Opportunity
1 Low contrast between People are wearing Need to develop SOD
foreground and Personal Protective methods which can work
background Equipment (PPE) which better in low contrast
may be similar with situations
surrounding environment
2 Occlusion problem with For fighting with SOD methods which can
human face in the COVID-19 humans are address the occlusion
real-time images wearing face mask which problem effectively
illustrates high contrast
between humans face
skin and face mask in
terms of color and texture
3 Group of object may not Today’s people are not SOD methods which can
be detected standing very close due to detect multiple objects at
simultaneously social distancing rule a distance
implemented for
controlling transmission
of coronavirus virus.
Therefore, in group
images each and very
people are considered as
individual objects while
the significant meaning of
the image is to capture all
the people present on the
location.
4 Saliency detection The face mask may Intelligent SOD methods
methods may be become more important are required to detect
misguided by protected object than the human in actual salient object in an
gears to highlight the image. Whereas the image
non-salient regions as image is captured for
salient regions keeping the human as
target object by
photographer
5 Keeping an eye on the It is difficult by an SOD methods are
student activity in online instructor to monitor the required which can keep
teaching students in an online class an eye on the student
due to no direct activities
interaction
10 V. Kumar Singh and N. Kumar

3.2 Opportunities

COVID-19 period has emerged as a great opportunity for computer vision researchers
to contribute in battling COVID-19 disease. This is also an opportunity for salient
object detection methods. For battling with the COVID-19 disease, salient object
detection methods are required to focus on the challenges discussed in Sect. 3.1.
In this section, we discuss research opportunities and directions for handling the
challenges that emerged in COVID-19 era for salient object detection. The low con-
trast image has a similar appearance of foreground and background regions. Such
types of images can be captured during COVID-19 as people are wearing Personal
Protective Equipment (PPE) which may have similar color and texture with the
surrounding environment. This scenario provides an opportunity to discover visual
features which have the discriminative capability to classify foreground and back-
ground regions from the input image. The partial occlusion problem may occur in
COVID-19 environment as people are wearing a face mask. This effect on the visual
scene may influence the performance of salient object detection as partial occlusion
is a challenging scenario for saliency detection. Consequently, it is an opportunity
for researchers to introduce such saliency detection approaches which can deal with
partial occlusion in a better manner.
During COVID-19, people are following social distancing, which affects the visual
appearance of people. However, with the social distancing people are scattered on the
whole image and it is very difficult to identify all the humans who have appeared for
salient object detection. This is an opportunity to find such methodologies which can
deal with multiple object detection in a scene. Furthermore, the education system is
also facing a big problem during this COVID-19 pandemic. The educational institu-
tions are conducting their classes using online platforms. In such a mode, controlling
class behavior is very challenging for the instructor. In this process, the visual data are
coming from various sources, hence it is very difficult to identify which visuals are
important. This is yet another opportunity to identify salient regions from a different
source of visual data. A summary of these opportunities is also given in Table 2.

4 Experimental Result

In this section, we illustrate the evaluation and analysis of different scenarios of

salient object detection which may be affected due to COVID-19 on the proposed
dataset. This study proposes a dataset which contains 100 natural images, out of
which 50 images include face masked humans whereas others consist of unmasked
faces in different scenarios.
This dataset contains a variety of images of three people by capturing these images
through a mobile camera either from the rear angle or from the front angle with proper
illumination. The ground truth is generated manually by one user which provides
consistent result with pixel-wise human annotation. The qualitative evaluation and
Challenges and Opportunity for Salient Object Detection in COVID-19 Era: A Study 11

Fig. 5 Qualitative study on samples of images of proposed dataset. First row represents
original Images, GMR [31] and FF-SVR [32] saliency maps are depicted in second and third
rows, respectively, fourth row shows ground truth (GT)

study is presented under different conditions as shown in Fig. 5. In this figure, Ui

and Mi represent unmasked and masked i-th images, respectively. For this study,
we have applied existing saliency detection methods such as Graph-Based Manifold
RankingGMR [31] and Fusion Framework for Salient Object Detection based on
Support Vector Regression (FF-SVR) [32] for generating saliency maps. It can be
observed from Fig. 5, when a human visually appears with a face mask then visual
attention is distracted by the face mask such as in M1 , only masked region is high-
lighted while in U1 whole face is detected. Similarly, in M2 , masked region is located
whereas in U2 entire human face is highlighted. In addition, in M3 mask region is
not detected, while face is identified in U3 . This evaluation and analysis support our
suggested challenges for salient object detection in COVID-19.

5 Conclusion and Future Work

COVID-19 pandemic has noticeably affected human lives across the world and the
death rate is also alarming. In this study, we have focused on various scenarios of
salient object detection which may be affected due to the presence of the COVID-19
pandemic worldwide. Nowadays, people are wearing various modalities such as Per-
sonal Protective Equipment (PPE), face masks which change the visual appearance
of people in outside places. Such visual changes have put certain challenges in the
12 V. Kumar Singh and N. Kumar

real-time images, namely, low contrast between foreground and background, partial
occlusion and online monitoring, etc. These challenges for salient object detection
have also come with certain opportunities for the researchers and practitioners work-
ing in this research area. We have evaluated these challenges on the proposed dataset
to provide experimental support. In future work, we will explore saliency detection
models that can effectively handle the COVID-19 challenges.

References

1. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene
analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
2. Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up
aggregation and cue integration. In: IEEE conference on computer vision and