ObjectTwin - Detect & Recognize Entities

A Project Report submitted in partial fulfilment of the requirements for the award of
the degree of

Bachelor of Technology
in
Computer Science and Engineering
by

Meghshyam 215000966
Shradha Saxena 2115000966
Swapnil Peeyoush Gupta 2115001026
Yash Tyagi 2115000966

Group No.: 137

Under the Guidance of

Ms. Chhavi Bajpai

Department of Computer Engineering & Applications

Institute of Engineering & Technology

GLA University
Mathura- 281406, INDIA
December, 2024
 CONTENTS
Declaration ii

Certificate iii

Acknowledgement iv

Abstract v

List of Figures vi

CHAPTER 1 INTRODUCTION 1-6

1.1 Motivation and Overview 1

1.2 Objective 2
1.3 Issues and Challenges 3
1.4 Summary of Similar Applications 4

CHAPTER 2 LITERATURE REVIEW 7-8

2.1 Vehicle Detection 7

2.2 Animal Classification 7
2.3 Face Detection 8
2.4 Object Identification 8

CHAPTER 3 SOFTWARE DESIGN 9-15

3.1 System Architecture 9

3.2 Data Flow Diagram 11
3.3 UML Diagram 13

CHAPTER 4 USER INTERFACE DESIGN 16

CHAPTER 5 CONCLUSION 17

Bibliography 18

i
 Declaration

We hereby declare that the work presented in this project titled "ObjectTwin – Detect &
Recognise Entities" is our own original work. This project has been carried out under the
guidance and supervision of Ms. Chhavi Bajpai, Assistant Professor, Computer Engineering
& Applications Department, GLA University, Mathura.
We further declare that this project has not been submitted previously for the award of any
degree or diploma at any other university or institution.

Date: ___________

Names of Project Group Members:

1. Shradha Saxena
2. Megh Shyam
3. Swapnil Peeyoush Gupta
4. Yash Tyagi

ii
 Certificate

This is to certify that the project titled "ObjectTwin – Detect & Recognise Entities" has been
completed by Megh Shyam, Shradha Saxena, Swapnil Peeyoush Gupta, and Yash Tyagi,
students of the B.Tech. (CSE) program, Batch 2024-25, under the supervision of Ms. Chhavi
Bajpai, Assistant Professor, Department of Computer Engineering & Applications, GLA
University.
The project was undertaken as part of the academic requirements for the completion of their
degree. The work presented is an original contribution of the students and has been carried out
with full dedication, effort, and teamwork. The students have demonstrated excellent technical
skills and problem-solving abilities in developing a unified system for detecting and
recognizing objects, which has practical applications in various domains.
I wish them success in their future endeavors.

Date: ___________

___________
Project Supervisor
Ms. Chhavi Bajpai
Assistant Professor
Computer Engineering & Applications
GLA University, Mathura

iii
 Acknowledgement

We, the members of the project group "ObjectTwin – Detect & Recognise Entities", would
like to extend our heartfelt thanks to all those who have supported us throughout the course of
this project.
First and foremost, we would like to express our sincere gratitude to our project supervisor,
Ms. Chhavi Bajpai, Assistant Professor, Department of Computer Engineering &
Applications, GLA University, for her invaluable guidance, constant support, and insightful
feedback. Her expertise and mentorship were crucial to the successful completion of this
project.
Our thanks also goes to the faculty and staff of the Department of Computer Engineering &
Applications, GLA University, for providing us with the necessary resources, infrastructure,
and support that made this project feasible.
We would like to express our deep gratitude to our friends and families for their continuous
encouragement, patience, and understanding during the completion of this project.
Finally, we acknowledge the contributions of the researchers and developers whose work in
the fields of object detection, machine learning, and image processing greatly inspired and
influenced our project.
Thank you to everyone who has contributed to the successful execution of this project.

Date: ___________

Names of Project Group Members:

1. Shradha Saxena
2. Megh Shyam
3. Swapnil Peeyoush Gupta
4. Yash Tyagi

iv
 Abstract

The project "ObjectTwin – Detect & Recognise Entities" is an innovative system designed
to provide real-time object detection and recognition capabilities across four distinct categories:
vehicle detection, animal classification, human face detection, and book identification. The
system allows users to upload images or capture them using a camera, after which it processes
the inputs through machine learning models to identify and classify the objects. The goal of
this project is to develop a versatile, user-friendly platform that can serve multiple practical
applications, such as traffic monitoring, security, animal identification, and educational
purposes.
The system integrates various machine learning techniques for object detection and
recognition, providing accurate results by linking real-world objects to their corresponding
identities in a pre-trained database. The dual functionality of detection and recognition is what
makes the system unique, allowing it to address diverse use cases in a single interface. The
project aims to demonstrate the potential of artificial intelligence to solve real-world problems
efficiently, offering a seamless user experience. The platform is designed to be lightweight,
requiring only a laptop and basic software tools to function, making it accessible for use in
various domains without additional hardware requirements.
In conclusion, ObjectTwin represents an efficient, multi-functional solution to object detection
and recognition, with applications in traffic management, security systems, wildlife
monitoring, and educational tools.

v
 LIST OF FIGURES
Page no.
Fig1.1: Block Diagram of YOLOv4 model 5
Fig2.1: Object Identification model 8
Fig3.1: System Architecture 10
Fig3.2: Data Flow Diagram 12
Fig3.3: Class Diagram 13
Fig3.4: Object Diagram 14
Fig3.5: Sequence Diagram 15
Fig4.1: Home Page UI 16
Fig4.2: Classification Page UI 16
Fig4.3: Image Input UI 16

vi
 INTRODUCTION

1.1 Motivation and Overview

In today’s rapidly evolving technological landscape, the need for efficient and accurate object
detection systems is more critical than ever. From smart traffic management systems to enhancing
security and simplifying identification tasks, the applications of object detection are vast and
diverse. However, many existing systems focus on only one specific use case—be it vehicle
detection, animal identification, or facial recognition. These systems often require multiple
separate platforms, making them complex, fragmented, and inefficient to use.

The ObjectTwin – Detect & Recognise Entities project was born out of the need for a unified,
versatile solution that integrates various object detection and classification tasks into one cohesive
platform. The motivation behind this project is to create a seamless, intuitive interface that can
handle multiple detection tasks, including vehicle detection, animal classification, human face
detection, and book identification. By doing so, we aim to simplify the user experience and provide
a comprehensive tool for real-world applications in domains like traffic management, security
systems, and education.

The growing demand for automation and real-time processing in areas like security, traffic
monitoring, and wildlife protection further motivated us to explore the potential of machine
learning in this field. We saw an opportunity to not only create a tool that makes complex tasks
more accessible but also to showcase the power and efficiency of AI in solving practical, everyday
problems.

This project is particularly exciting because it merges machine learning techniques with real-time
applications, allowing users to interact with sophisticated models in a way that is intuitive and
accessible, without requiring deep technical knowledge. The ability to detect and recognize objects
in real-time opens up a wide range of possibilities, making technology more user-centric and
responsive to the needs of various industries.

1
1.2 Objective

The main objective of the ObjectTwin – Detect & Recognise Entities project is to develop a
unified, real-time object detection and recognition platform that can seamlessly handle multiple
tasks. The system will detect and classify objects across four primary categories: vehicles, animals,
human faces, and books. By integrating these tasks into a single interface, the project aims to
simplify the user experience, providing a versatile tool for a wide range of practical applications
in various domains such as traffic monitoring, security, education, and wildlife management.
Key objectives of the project include:
1. Multi-Task Object Detection: To create a system that supports real-time detection and
recognition of various objects, including vehicles, animals, human faces, and books.
2. Real-Time Processing: To ensure that the platform can process images in real time, whether
they are uploaded or captured live using a camera, providing immediate feedback to users.
3. User-Friendly Interface: To design a simple, intuitive user interface that allows users to
interact with the system easily, without requiring technical expertise.
4. Machine Learning Integration: To leverage advanced machine learning algorithms, such
as YOLO (You Only Look Once) and transfer learning, for accurate object detection and
recognition.
5. Versatile Applications: To demonstrate how AI can be used to streamline complex tasks,
making technology accessible for real-world applications in traffic management, security,
education, and other industries.
6. Scalability and Flexibility: To build a scalable platform that can be extended with
additional detection categories or functionalities in the future.
Through this project, we aim to showcase the potential of AI in solving everyday problems while
also providing a solid foundation for future developments in object detection technology.

2
1.3 Issues and Challenges

While object detection and recognition systems have seen significant advancements in recent
years, there are still several issues and challenges faced by existing solutions, especially when it
comes to integrating multiple detection tasks into a single platform. Some of the key challenges
encountered in previous work are:
1. Limited Scope and Focus
Most existing object detection systems focus on solving a single problem, such as vehicle
detection, facial recognition, or animal classification. This narrow focus leads to fragmented
solutions, requiring users to rely on different platforms or systems for different tasks. Integrating
multiple functionalities into one system presents the challenge of designing a unified platform that
can efficiently handle diverse tasks without compromising on performance or accuracy.
2. Real-Time Processing
Real-time object detection is a challenging task, especially when handling large image data or
videos. Ensuring that the system can process and provide feedback immediately—whether for
uploaded images or live camera inputs—while maintaining high accuracy is a significant
challenge. Previous systems often struggle with latency issues, especially when running complex
models on limited hardware.
3. Accuracy and Performance
Achieving high accuracy in object detection and recognition is crucial for ensuring the system’s
effectiveness. However, this often requires deep neural networks and large datasets, which can be
computationally expensive and time-consuming to train. Many existing systems either focus on
speed, sacrificing accuracy, or prioritize accuracy, resulting in slower performance.
4. Data Collection and Annotation
A major challenge in building any object detection system is the collection and annotation of
datasets. In our project, we need datasets for vehicles, animals, human faces, and books. Acquiring
large, diverse, and well-annotated datasets for these categories can be a time-consuming and labor-
intensive process. Moreover, the quality and variety of the data play a significant role in the
model’s ability to generalize to real-world scenarios.
5. User Experience and Interface Design
Creating an intuitive user interface that allows non-technical users to interact with complex
machine learning models remains a key challenge. Previous systems often struggle with providing
an interface that is both simple and powerful, making it difficult for users to get the most out of
the technology without a steep learning curve.
6. Scalability and Maintenance
As the system integrates multiple detection tasks, scalability and long-term maintenance become
increasingly challenging. Ensuring that the platform can handle an increasing amount of data,
adding new models or detection categories without major rewrites, and performing consistent
updates to adapt to new use cases is a continuous challenge faced by previous systems.

3
1.4 Summary of Similar Applications

Several applications and systems have been developed in the field of object detection and
recognition. These applications often focus on specific use cases such as traffic monitoring, animal
detection, facial recognition, and more. However, there are few platforms that integrate multiple
detection tasks into a single cohesive system, as proposed by the ObjectTwin project. Below is a
summary of some similar applications that have inspired and informed the development of this
project:

1. Google Vision AI
Google Vision AI provides a comprehensive image recognition service, offering pre-trained
models for detecting and identifying various objects, including vehicles, faces, animals, and more.
It allows users to upload images for analysis or integrate object detection capabilities into their
applications via APIs. While Google Vision is a powerful tool for object recognition, it focuses
primarily on recognizing a wide range of objects through cloud-based processing. It does not offer
the same level of customizability or integration of diverse use cases into a single platform.
Challenges:
• Requires cloud access, making it less suitable for offline applications.
• Limited customization options for specific use cases.

2. OpenCV (Open Source Computer Vision Library)

OpenCV is an open-source library that provides tools for real-time image processing and computer
vision applications. It includes several pre-built algorithms for object detection, such as Haar
cascades for face detection and other deep learning-based methods. OpenCV is widely used for
developing custom applications and can be integrated with machine learning models to detect
objects such as vehicles, animals, and faces.
Challenges:
• OpenCV requires significant expertise to set up and integrate complex machine learning
models.
• It is a low-level tool that requires developers to build and fine-tune the detection systems
themselves.
• Does not offer a unified, easy-to-use interface for multi-task detection.

4
YOLO (You Only Look Once)
YOLO is a popular deep learning-based object detection algorithm known for its speed and
efficiency. YOLO can detect multiple objects in real-time within a single image, making it ideal
for applications in video surveillance, autonomous driving, and security systems. YOLO models
can detect various objects, including vehicles, animals, and human faces.
Challenges:
• YOLO is designed for a single task at a time, typically detecting multiple objects in a scene,
but not necessarily classifying them in different categories.
• Requires significant computational power, especially in real-time applications.

Fig1.1: Block Diagram of YOLOv4 model

5
4. Microsoft Azure Computer Vision
Similar to Google Vision AI, Microsoft’s Azure Computer Vision service provides a powerful
object detection and image recognition tool. It can identify a range of objects, including vehicles,
animals, and faces. This service can be integrated into various applications for image analysis,
including custom object detection tasks.
Challenges:
• Like Google Vision, it is a cloud-based service and relies on an internet connection.
• It does not provide the flexibility for integrating multiple detection models or custom use
cases in a single interface.

5. IBM Watson Visual Recognition

IBM Watson Visual Recognition offers pre-trained models for detecting and classifying a variety
of objects, such as animals, vehicles, and faces, similar to Google Vision and Microsoft Azure.
The platform provides a visual recognition API that allows developers to integrate object detection
into their applications. Watson’s strength lies in its ability to train custom models for specific tasks
using your own datasets.
Challenges:
• It primarily focuses on cloud-based image analysis.
• Customizing the models for unique use cases requires a higher level of technical expertise.

6
 LITERATURE REVIEW

2.1 Vehicle Detection

Research Work 1: "Real-Time Vehicle Detection and Classification Using YOLO" (2020)
This study demonstrated the use of the YOLO (You Only Look Once) algorithm for real-time
vehicle detection and classification into categories like cars, trucks, and buses. The researchers
highlighted the algorithm's speed and accuracy, making it suitable for traffic management systems.
Key Takeaway: YOLO’s architecture can be adapted for detecting vehicles as either "Mini" or
"Big" in our project.
Research Work 2: "Automated Traffic Monitoring System Using Deep Learning" (2021)
The authors proposed a traffic monitoring system that used convolutional neural networks (CNNs)
to detect and classify vehicles in live feeds. The paper also discussed challenges like object
occlusion and low lighting.
Key Takeaway: Robust preprocessing techniques, such as image enhancement, are crucial for
ensuring consistent detection accuracy in varied conditions.

2.2 Animal Classification

Research Work 3: "Deep Learning for Animal Species Classification in Wildlife" (2019)
This research employed a pre-trained ResNet-50 model to classify animal species from images
captured in the wild. The authors demonstrated the effectiveness of transfer learning in minimizing
training time while achieving high accuracy.
Key Takeaway: Transfer learning can be utilized to classify animals like dogs, cats, and horses
efficiently in our project.
Research Work 4: "Animal Recognition Using MobileNet for Real-Time Applications" (2021)
The study introduced a lightweight deep learning model, MobileNet, for real-time animal
recognition on mobile devices. The researchers emphasized the importance of using lightweight
models for projects with limited hardware resources.
Key Takeaway: Incorporating a lightweight model ensures compatibility with standard laptops
and real-time performance.

7
2.3 Face Detection

Research Work 5: "Face Detection in Unconstrained Environments Using Faster R-CNN" (2020)
This paper presented a robust face detection system using the Faster R-CNN model, which
demonstrated high performance in unconstrained environments, including crowded or poorly lit
areas.
Key Takeaway: Faster R-CNN could inspire the design of a face detection module for identifying
human faces under varying conditions.
Research Work 6: "FaceNet: A Unified Embedding for Face Verification and Recognition"
(2015).
This groundbreaking work introduced the FaceNet model, which creates unified embeddings for
face verification and recognition tasks. The study emphasized its scalability for large datasets.
Key Takeaway: Face embeddings can be integrated into our recognition module to improve
human face identification.

2.4 Object Identification

Research Work 7: "Instance Segmentation for Book Identification Using Mask R-CNN" (2019)
The study focused on book identification in library settings using Mask R-CNN. It demonstrated
how instance segmentation can separate and recognize overlapping objects effectively.
Key Takeaway: Mask R-CNN's segmentation capabilities could enhance book identification
accuracy in our project.
Research Work 8: "Real-Time Object Recognition with SSD and TensorFlow" (2021)
This research implemented the Single Shot Multibox Detector (SSD) for real-time object
recognition, highlighting its balance between speed and accuracy.
Key Takeaway: SSD could be an alternative for real-time performance in our object identification
module.

Fig2.1: Object Identification Model

8
 SOFTWARE DESIGN

3.1 System Architecture

The ObjectTwin – Detect & Recognise Entities system architecture is designed to integrate
multiple object detection and recognition tasks into a unified platform. It consists of several
interconnected components working together to provide a seamless user experience. The
architecture begins with the User Interface (UI), where users can upload images or capture them
through a live camera. This input is sent to the Input Handler, which validates and preprocesses
the data for further processing.
The core processing is handled by two main modules: the Detector and the Recognizer. The
Detector identifies objects in the input image, while the Recognizer links these objects to their
corresponding identities in a pre-defined database. These processes leverage pre-trained machine
learning models optimized for real-time performance. The results are then passed to the Result
Display module, which formats the detection and recognition outputs for visualization in the UI.
This modular architecture ensures scalability, allowing new detection models to be integrated
easily while maintaining an intuitive and efficient workflow.

9
Fig3.1: System Architecture

10
3.2 Data Flow Diagram (DFD)

Level 0 DFD
The Level 0 Data Flow Diagram provides a high-level view of the ObjectTwin system,
representing it as a single process. Users interact with the system by providing inputs either through
image uploads or live camera capture. This input is processed by the system, which performs object
detection and recognition using pre-trained machine learning models. The detection results, such
as the identified object and its category, are then presented back to the user through a user-friendly
interface. The diagram highlights the core input and output interactions of the system.

Level 1 DFD
The Level 1 Data Flow Diagram delves deeper into the internal processes of the system. The Input
Handling module accepts image inputs from the user, validating and managing uploads or live
camera captures. The Preprocessing module standardizes these inputs by resizing and normalizing
them to ensure compatibility with the object detection models. The Detection Module performs
object detection tasks, identifying vehicles, animals, faces, or books within the images. The
Recognition Module further processes these detections by matching the identified objects with
entries in the database for classification and identity recognition. Finally, the Output Display
module presents the processed results in an intuitive format, ensuring that users can easily interpret
the detection and recognition outcomes.

11
Fig3.2: Data Flow Diagram

12
3.3 UML Diagram

Class Diagram
The class diagram represents the core components of the ObjectTwin system and their attributes.
It defines six main classes: User, which stores user details like userID and login credentials;
InputHandler, which manages input types and file paths; Preprocessor, responsible for resizing
and normalizing input images; Detector, which handles object detection tasks using specified
models and stores results; Recognizer, which connects detected objects to their corresponding
database entries; and ResultDisplay, which formats and presents the results to the user. This
diagram serves as a blueprint for the system's structure and relationships.

Fig3.3: Class Diagram

13
Object Diagram
The object diagram provides a snapshot of the system in operation, showcasing real-world
instances of the defined classes. For example, a User instance could have userID: 101 and name:
Shradha, while the InputHandler might process a cameraCapture input saved as liveFeed.jpg.
Similarly, the Detector could use the YOLO model to detect a "Car" and forward results to the
Recognizer. This diagram demonstrates how the system's classes interact in specific scenarios,
offering a tangible view of system behavior.

Fig3.4: Object Diagram

14
Sequence Diagram
The sequence diagram outlines the flow of interactions among system components during a typical
process. When a User uploads an image, the InputHandler processes it and sends it to the
Preprocessor for normalization. The Detector identifies objects in the preprocessed image, and
the Recognizer matches these detections with the database to determine their identities. Finally,
the ResultDisplay sends the results back to the user. This diagram emphasizes the chronological
order of actions and highlights the interdependencies among modules.

Fig3.5: Sequence Diagram

15
 USER INTERFACE DESIGN

Fig4.1: Home Page UI

Fig4.2: Classification Page UI

Fig4.3: Image Input UI

16
 CONCLUSION

The ObjectTwin – Detect & Recognise Entities project demonstrates the successful integration of
machine learning into a unified platform that serves multiple object detection and recognition use
cases. The project effectively combines four major functionalities—vehicle detection, animal
classification, human face detection, and book identification—into a user-friendly interface,
showcasing the potential of AI to address real-world challenges.

6.1 Key Achievements

1. Versatility: The system supports diverse applications across domains like traffic
monitoring, security systems, wildlife identification, and education.
2. Real-Time Processing: By enabling real-time image processing for both uploaded and live-
captured images, the project enhances user interaction and accessibility.
3. Innovation: The integration of multiple tasks into a single platform sets this project apart
from traditional object detection systems, providing a streamlined solution for multiple use
cases.

6.2 Challenges Faced

1. Dataset Preparation: Collecting and annotating diverse datasets for accurate detection and
recognition was time-consuming.
2. Model Optimization: Balancing accuracy and performance on limited hardware resources
posed a challenge, which was addressed using efficient models like YOLO and transfer
learning techniques.
3. UI Integration: Creating an intuitive user interface while ensuring smooth backend
communication required careful planning and implementation.

6.3 Future Scope

The project holds significant potential for future enhancements:
1. Scalability: Expanding the system to include more detection categories (e.g., plant
identification, industrial machinery detection).
2. Edge Deployment: Optimizing the system for deployment on edge devices like Raspberry
Pi for offline usage.
3. Enhanced Recognition: Incorporating deep learning models with higher accuracy and
including voice-based feedback for enhanced user experience.

17
 Bibliography

1.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. - Reference for understanding deep
learning concepts and frameworks used for object detection and recognition.
2. Chollet, F. (2017). Deep Learning with Python. Manning Publications. - Guide to implementing machine learning
models using Python and TensorFlow/Keras.
3.Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). "You Only Look Once: Unified, Real-Time Object
Detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). - Reference
for YOLO (You Only Look Once), a foundational model for real-time object detection.
4.Dalal, N., & Triggs, B. (2005). "Histograms of Oriented Gradients for Human Detection." Proceedings of the IEEE
Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). - Used for understanding feature
extraction in human face detection.
5.Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer. - Resource for fundamental machine
learning techniques used in object recognition.
6.Zhang, Z. (2012). "Microsoft Kinect Sensor and Its Effect." IEEE Multimedia, 19(2), - Background reference for
real-time image processing techniques.
7.OpenCV Documentation. Retrieved from https://docs.opencv.org - Reference for image preprocessing and
processing techniques using OpenCV.
8.TensorFlow Documentation. Retrieved from https://www.tensorflow.org - Reference for implementing and
deploying machine learning models.
9.Flask Documentation. Retrieved from https://flask.palletsprojects.com - Reference for building and deploying the
user interface using Flask.
10.PyTorch Documentation. Retrieved from https://pytorch.org - Reference for alternative model development and
training approaches.
11.LabelImg Documentation. Retrieved from https://github.com/heartexlabs/labelImg - Used for dataset annotation
during the dataset preparation phase.
12. Datasets from Kaggle (https://www.kaggle.com) - Various public datasets used for training and testing the object
detection models.

18