See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/384860311

Smart Home Security System and Liveness Detection using Convolutional

Neural Networks

Research · December 2022

DOI: 10.13140/RG.2.2.36801.57448

CITATIONS READS
0 10

7 authors, including:

Satheesh Kumar G R
Sri Sivasubramaniya Nadar College of Engineering
1 PUBLICATION 0 CITATIONS

SEE PROFILE

All content following this page was uploaded by Satheesh Kumar G R on 12 October 2024.

The user has requested enhancement of the downloaded file.

Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997

RESEARCH ARTICLE

Smart Home Security System and Liveness Detection using

Convolutional Neural Networks

Amith Kumar N1, Satheesh Kumar.G.R1, Sreedhar.V1, Surya.S.S1, S.Saraswathi2, V. Lokeshwari2 and K.
Madheswari2*

1Final Year CSE (Student), Department of Computer Science and Engineering, Sri Sivasubramaniya
Nadar College of Engineering, Chennai, Tamil Nadu, India.
2Associate Professor, Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar

College of Engineering, Chennai, Tamil Nadu, India.

Received: 05 Sep 2022 Revised: 14 Oct 2022 Accepted: 15 Nov2022

*Address for Correspondence

K. Madheswari
Associate Professor,
Department of Computer Science and Engineering,
Sri Sivasubramaniya Nadar College of Engineering,
Chennai, Tamil Nadu, India.
Email: [email protected]

This is an Open Access Journal / article distributed under the terms of the Creative Commons Attribution License
(CC BY-NC-ND 3.0) which permits unrestricted use, distribution, and reproduction in any medium, provided the
original work is properly cited. All rights reserved.

ABSTRACT
Yashoda
This project's main aim is to create a smart security system for enhancing home security using the
Internet of Things and Deep Learning that can safeguard and monitor a home from a mobile phone. The
DL-based face recognition system, which is triggered whenever any motion is detected, identifies the
visitor's face using the Viola Jones Haar Cascade algorithm and unlocks specific rooms based on the
person’s category. Live persons are differentiated from still images of the person using the liveliness
detection deep learning model making the system fool-proof. SMTP is used in the system and the mobile
app to customize the numerous profiles and the persons linked with each profile and also to alert the
owner whenever a stranger visits the place. MQTT is used to unlock each and every lock remotely using
the mobile app.

Keywords: Deep Learning, IoT, Home Security System, Haar Cascade algorithm

INTRODUCTION

With the country's fast economic expansion and rising urban population, security has become a top priority. As a
result, theft, burglary, robbery, and other occurrences involving personal security have been on the rise. These
incidences have a significant impact on people's daily lives, particularly those who spend the majority of their time
outside the house. The issue is with the traditional lock and key method of security, which may be easily picked
without leaving any traces. We envision to create an intelligent home security system that overcomes the overheads
51612
Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997

Amith Kumar et al.,

due to the traditional security measures. This project makes advantage of technical breakthroughs such as IoT and
Deep Learning to greatly improve home security. The faces of the owner and his family members are registered and
saved in the database. Along with that, the owner may establish profiles such as family, non-family and customize
the doors that must be unlocked for each profile. When someone enters the house, the camera positioned at the front
entrance records an image of them and recognizes their face. If their face matches a profile in the database, the doors
corresponding to that profile are opened. If no match is identified, the image is sent to the owner, who can choose
whether or not to unlock the door. Section II addresses some of the relevant studies and existing systems in the same
field. Section III introduces the proposed system, and Section IV evaluates the system's performance. Finally, the
project is concluded in section V. To identify research needs, a survey of existing systems was conducted. CCTV
Cameras, Video Door Phones, Traditional Alarm Solutions, and other security systems have become popular in
recent years. CCTV Cameras records videos 24x7 usually in a low quality also wastes a lot of storage space. They do
not immediately notify the residents. When no one is at home or at the office, the Video Door Phone is worthless.
Traditional alarm systems just notify the neighbors in the event of a burglary and do not offer any type of
surveillance. The main issue with present systems is that they can only detect the intruder after the theft has
occurred, which is inefficient. Furthermore, the system does not always distinguish between human and non-human
objects. Wireless Sensor Networks is the most sophisticated existing system (WSNs). It is combined with the usage of
the Internet of Things (IoT) and the Cognitive Internet of Things, which broadens the scope of smart home concepts
and solutions, as well as their applications. The present study provides a revolutionary smart home anti-theft system
that can identify an intruder even if their face is partially or completely disguised by clothes, leather or plastic
materials. The fundamental idea was to create a cost-effective and efficient method allowing an individual to detect
any type of theft in real-time and provide the house owner with immediate warning of the crime. The technology
also claims to provide home security with real-time video data processing. The recognition results verify the
suggested system's success. The accuracy of the system is determined by the many scenarios that occur.

MATERIALS AND METHODS

Using MQTT, the proposed approach is deployed in near real-time to an IoT-based smart home monitoring system.
To analyze unauthorized access, a smart home model is established and constructed based on an integrated
framework of sensors, cameras, and specialized hardware. The architecture of the system as mentioned in Fig.1. The
system functions on two levels: through a hardware interface and a software interface. A sensing node (PIR motion
sensor, Pi camera), solenoid locks, and relay modules are deployed at the hardware interface level, and are linked to
a processing unit (Raspberry Pi 4B) that recognizes the face-data captured by the camera. The main objective of the
software is to detect and report unsupervised human activity.
The software interface level is divided into 3 parts, as follows
 Face recognition
o Face detection
o Identification
o Liveness detection
 Processing unit
 Mobile App

Face Recognition
A pi camera is installed in front of the door and is activated when the PIR motion sensor detects motion as shown in
Fig.2.The pi camera's live video is transferred to the processing unit for face detection, identification, and liveness
detection, as stated below.

51613
Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997
Amith Kumar et al.,

Face detection
The Viola–Jones object detection framework [3], also known as Haar Cascades, is used for face detection. It was
intended to recognize frontal faces the best, rather than those looking sideways, upwards, or downwards. Prior to
detecting a face, the image is transformed to grey scale since it is easier to work with and has less data to process [1-
7]. The algorithm recognizes the face on the grey scale picture first, and then locates it on the colored picture. Viola-
Jones draws a box and looks for a face within it. It is essentially looking for Haar-like characteristics including edge
features, line features, and four-sided features some of which are shown in Fig.3. These features assist the system in
interpreting what the image is. We use this model to detect the face in the live feed and then process it for face
recognition [8].

Calculation of Haar like features with Integral Image

Using integral images, we can achieve constant time evaluation of Haar features.
1. Edge Features or 2 Rectangular Features requires only 6 memory lookups
2. Line Features or 3 Rectangular Features requires only 8 memory lookups.
3. Diagonal Features or 4 Rectangular Features requires only 9 memory lookups.
2 Rectangle = A-2B+C-D+2E-F
3 Rectangle = A-B-2C+2D+2E-2F-G+H
4 Rectangle = A-2B+C-2D+4E-2F+H-2I+J

Identification
We utilized Adam Geitgey's face-recognition package [2] to recognize faces. It is developed with Dlib's cutting-edge
face recognition model using deep learning. On the Labelled Faces in the Wild benchmark, the model has an
accuracy of 99.38 percent. It creates a 128-dimensional encoding vector. Based on this encoding, we can calculate the
similarity of two facial photographs to determine if they belong to the same individual. To recognize a person, we
must encode the detected face and store the 128-dimensional encoded vector in a Python dictionary with the
encoded vector as the key and the person's name as the value. The tolerance value for checking similarity (i.e.
Euclidean distance between the encoding vectors) is set at 0.6 by default. If the Euclidean distance between two face
encoded vectors is less than 0.6, they are from the same person; otherwise, they are from different persons. To make
face comparisons more stringent, a lower tolerance value is required.

Liveness detection
Face recognition technology has advanced quickly in recent years, and it is now more direct, user-friendly, and
convenient than any other previous approaches. Face recognition systems are subject to spoof attacks by fake/non-
real faces [6]. Facial images, such as portrait shots, are a simple approach to fool face recognition algorithms. To
make face recognition systems more reliable, we need to recognize such fake/non-real faces – liveness detection is the
word for such techniques. Some of the approaches are the following:
 Texture analysis
 Frequency analysis
 Variable focusing analysis
 Heuristic-based algorithms
 Optical flow algorithms
 3D face shape
 Binary classification-based analysis

We are employing the binary classification-based analysis, which is described below.

Binary classification using CNN

The technique of anti-spoof problem was introduced by Tan et al [1]. The authors' primary concept is that a real
human face differs from a face in a photograph. A genuine face is a 3D item, but a photograph is only 2D. The
objective is formulated as a binary classification issue, however the distributions of positive and negative are
51614
Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997

Amith Kumar et al.,

substantially overlapping in the input space, hence an appropriate representation space is determined to be
important. The collection of actual and fake facial photos is fed into the CNN, which extracts characteristics from the
photographs [9]. This CNN, also known as Liveness Net, consists of a series of convolutional layers linked
sequentially with a SoftMax layer at the end for classification. We utilize the dataset to train the Liveness Net model,
which is used to determine if a face in a live feed is real or false. Because we are doing this in real-time, we must
confirm by checking a number of times and taking the maximum as the result. A basic Binary classification using
CNN is shown in Fig.4.

RESULTS AND DISCUSSION

The user will be able to control the whole system using the mobile application. A nice and interactive GUI-based
interface and icons lets the user know about its working easily. The user can also keep track of all the entries added
as shown in the figure 1. The main functionalities of the application. Add, Modify or Delete entries: The user details
can be added, modified or deleted in the android app and the data will be automatically sent to the raspberry pi
through SMTP which is also stored in the local database. The user details consist of their name, phone number,
gender, type (Family or Non-Family) and the photo of the user as Shown in figure 6. The users of the system will be
displayed in the first page of the application as shown in Fig.5. Opening door using MQTT: The Message Queuing
Telemetry Transport is a lightweight, publish-subscribe network protocol that transports messages between devices.
The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional
connections can support MQTT. The doors can be opened with a single button click as shown in Fig.6.

Performance Evaluation:
Our proposed model is built using Dlib's state-of-the-art face recognition which uses deep learning. The model has
an accuracy of 99.38% on the labelled Faces in the Wild benchmark.

CONCLUSION

This paper implements a smart home security system to be able to monitor the home with the help of Raspberry pi,
Camera sensor, Motion detection sensor and a Mobile app. The DL-based face recognition system, which is triggered
whenever any motion is detected, identifies the visitor's face using the Viola Jones Haar Cascade algorithm and
unlocks specific rooms based on the person’s category. Live persons are differentiated from still images of the person
using the liveliness detection deep learning model making the system fool-proof. SMTP is used in the system and the
mobile app to customize the numerous profiles and the persons linked with each profile and to alert the owner
whenever a stranger visits the place. MQTT is used to unlock each lock remotely using the mobile app.

REFERENCES

1. X. Tan, Y. Li, J. Liu, and L. Jiang, Face liveness detection from a single image with sparse low rank bilinear
discriminative model, in ECCV, 2010.
2. https://pypi.org/project/face-recognition/
3. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," Proceedings of the
2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001, pp. I-I,
doi: 10.1109/CVPR.2001.990517
4. https://www.researchgate.net/publication/338071682_Smart_Home_Security_System
5. Madheswari K, N Venkateswaran. Particle Swarm Optimization aided Weighted Averaging Fusion Strategy for
CT and MRI Medical Images. International journal of biomedical engineering and technology. August 2, 2019 ,
vol: 31(3): 278-291

51615
Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997
Amith Kumar et al.,

6. Madheswari . K , Venkateswaran, N. Ganeshkumar. Entropy optimizedcontrastenhancement for gray scale

images. International journal ofApplied Engineering Research,Volume 10, Number 55,ISSN 0973-4562, pp. 1590-
1595.
7. Blessy Theresa X, Madheswari. K. Thermal and Visible Video Fusion Using Curvelet TransforminInternational
Journal of Applied Engineering Research ISSN 0973-4562Volume 13, Number 11 (2018) pp. 8831-8836.
8. Sakthi umamaheswari, Parimala Suresh Congovi, Madheswari Kanmani. Medicine Information Mobile
ApplicationUsing Tablet Image Anaysis Using Android Studio. in International Journal of Applied Engineering
Research ISSN 0973-4562 Volume 13, Number 10(2018) pp. 8407-8412
9. Madheswari K, Venkateswaran N. Fusion of Visible and Thermal imagesusing Curvelet Transform and Brain
Storm Optimization. IEEE TENCON2016, Singapore 2016.
10. Sharmadha Senthil Nathan, ShivaniKanmani, Shruthi Kumar, MadheswariKanmani. Survey on Digital Age-
Smarter Cradle System for Enhanced Parenting. International Journalof Applied Engineering Research ISSN
0973-4562 Volume 13, Number 10(2018) pp. 8187-8193.
11. Sharmadha Senthil Nathan, shivaniKanmani, Shruthi kumar, Madheswari Kanmani. OptimizedMulti Scale
ImageFusion Technique Using Discrete Wavelet Transform and Particle SwarmOptimization for Colour Multi
Focus Images. International Journal ofApplied Engineering Research ISSN 0973-4562 Volume 13, Number 10,
(2018) pp. 8179-8186.

Table 1: Overall time taken for the process

Parameters Time duration in seconds
Time taken to detect motion 0.57s
Time taken to detect face 0.87s
Time taken to recognize face 1.46s
Time taken to send mail 16.7s
Time taken to open locks after face recognition 0.24s
Time taken to open locks using mobile app 0.15s

Fig. 1: Overall architecture diagram of the proposed Fig. 2: Snapshot of the Model
system

51616
 Indian Journal of Natural Sciences www.tnsroindia.org.in ©IJONS

Vol.13 / Issue 75 / December/ 2022 International Bimonthly (Print) ISSN: 0976 – 0997

Amith Kumar et al.,

Fig. 3: Rectangle features-Haar Fig. 4: Binary classification using CNN

Fig. 5: Application result 1 Fig. 6: Application result 2

Fig. 7: Application result 3

51617

View publication stats