Anomaly Detection in Vibration Pattern Using

Al-Driven Motion Amplification

By

Khurram Ishtiaq
CUI/FA20-EEE-071/ATD
Khizer Fiaz
CUI/FA20-EEE-053/ATD
Khidmat Hussain
CUI/FA20-EEE-052/ATD
BS Thesis
In

Electrical (Electronics) Engineering

COMSATS University Islamabad
Abbottabad Campus-Pakistan

Spring 2024

1
 COMSATS University Islamabad-Abbottabad
Campus

Anomaly Detection in Vibration Pattern

Using Al-Driven Motion Amplification

A Thesis Presented to

COMSATS University Islamabad- Abbottabad Campus

In partial fulfillment
Of the requirement for the degree of

BS Electrical (Electronics) Engineering

By

Khurram Ishtiaq
CUI/FA20-EEE-071/ATD

Khizer Fiaz
CUI/FA20-EEE-053/ATD

Khidmat Hussain
CUI/FA20-EEE-052/ATD

Spring 2024

2
 Anomaly Detection in Vibration Pattern Using Al-
Driven Motion Amplification
An Under Graduate Thesis submitted to Electrical and Computer Engineering

Department as partial fulfillment of the requirement for the award of Degree of

Bachelor of Science in Electrical (Electronics) Engineering.

Name Registration Number

Khurram Ishtiaq CUI/FA20-EEE-071/ATD

Khizer Fiaz CUI/FA20-EEE-053/ATD

Khidmat Hussain CUI/FA20-EEE-052/ATD

Supervisor
Dr. Ihsan Ullah
Associate Professor, Electrical and Computer Engineering
Abbottabad Campus
COMSATS University Islamabad (CUI)
Abbottabad Campus
July 2024

3
 Final Approval

This thesis titled

Anomaly Detection in Vibration Pattern Using

Al-Driven Motion Amplification
By
Khurram Ishtiaq
CUI/FA20-EEE-071/ATD
Khizer Fiaz
CUI/FA20-EEE-053/ATD
Khidmat Hussain
CUI/FA20-EEE-052/ATD
Has been approved

For the COMSATS University Islamabad, Abbottabad Campus

Supervisor: ______________________________________________
Dr. Ihsan Ullah, Associate Professor
Department of Electrical and Computer Engineering Engineering/CUI,
Abbottabad Campus

HOD: ______________________________________________
Dr.Irfanullah, Professor
Department of Electrical and Computer Engineering, CUI, Abbottabad Campus

4
 Declaration

I Khurram Ishtiaq (FA20-EEE-071), Khizer Fiaz (FA20-EEE-053), and Khidmat

Hussain
(FA20-EEE-052) hereby declare that I have produced the work presented in this thesis,
during the scheduled period of study. I also declare that I have not taken any material
from any source except referred to wherever due that amount of plagiarism is within
acceptable range. If a violation of HEC rules on research has occurred in this thesis, I
shall be liable to punishable action under the plagiarism rules of the HEC.

Date: _________________ Signature of the student:

Khurram Ishtiaq
CUI/FA20-EEE-071/ATD

Khizer Fiaz
CUI/FA20-EEE-053/ATD

Khidmat Hussain
CUI/FA20-EEE-052/ATD

5
6
 Certificate

It is certified that Khurram Ishtiaq (FA20-EEE-071), Khizer Fiaz (FA20-EEE-053), and

Khidmat Hussain (FA20-EEE-052) has carried out all the work related to this report
under my supervision at the Department of Electrical and Computer engineering,
COMSATS University Islamabad, Abbottabad Campus and the work fulfills the
requirement for award of BS degree.

Date: _________________

Supervisor:

Dr. Ihsan Ullah

Department of Electrical and Computer Engineering

Head of Department:

Dr.Irfanullah
Department of Electrical and Computer Engineering

7
 DEDICATION

In profound gratitude, we dedicate this monumental work to the pillars of our success our
beloved parents and our esteemed supervisor. A special thanks to our parents for their
love and sacrifices we offer this work as a testament to the profound impact of your belief
in our abilities. A special acknowledgment is reserved for our respected supervisor,
whose words of encouragement and relentless push for tenacity reverberate in our minds
as a source of inspiration. I would also dedicate this work to our teachers who have
always encouraged us and to our friends who have helped us in this endeavor.

8
 ACKNOWLEGDGEMENTS

We express our heartfelt gratitude to ALLAH Almighty for blessing us throughout the
completion of this project. Our sincere thanks extend to our Supervisor, Dr. Ihsan Ullah,
an Associate Professor in the Electrical Engineering Department, for his invaluable
guidance and support. His wisdom and assistance have been instrumental in enabling us
to submit this thesis report successfully. Without his mentorship, this project would not
have reached fruition. Our appreciation also goes to COMSATS Institute of Information
Technology, Abbottabad, for fostering a conducive study environment and providing the
necessary facilities for our research. We acknowledge the unwavering love, prayers, and
support from our parents and siblings, whose encouragement has been a driving force.
Special thanks are due to our respected teachers and colleagues for their assistance on
various occasions. We extend our gratitude to everyone who has directly or indirectly
contributed to our journey, and we owe our respect to those who have inspired us. Thank
you all for being a part of this significant milestone in our academic journey.

Khurram Ishtiaq CUI/FA20-EEE-071/ATD

Khizer Fiaz CUI/FA20-EEE-053/ATD

Khidmat Hussain CUI/FA20-EEE-052/ATD

9
 ABSTRACT

The project is about on enhancing anomaly detection techniques through the use of
motion amplification technology. Motion amplification (Eulerian Video Magnification
EVM) amplifies subtle motions, making it easier to observe and analyze vibration
patterns that are often imperceptible to the naked eye or traditional sensors. By capturing
high-speed video data of machinery or structures, we utilize advanced algorithms to
amplify minute vibrations and convert these visual data into quantitative measurements.
The Lucas-Kanade method is a widely used (Optical Flow Algorithm) in Computer
vision, specifically designed for estimating motion between consecutive frames in a video
sequence to find the intensity of motion in x and y directions. We then apply machine
learning (SVM) techniques to these amplified patterns to detect anomalies that indicate
potential defects or failures. The integration of motion amplification with machine
learning not only improves the accuracy of anomaly detection but also provides a non-
intrusive and cost-effective solution for continuous monitoring. By combining AI-driven
algorithms with motion amplification, this research aims to enhance the sensitivity of
vibration pattern analysis, enabling the early detection of anomalies indicative of
machinery malfunctions. The project involves the development of a comprehensive
conceptual framework, including data collection in the form if videos and the
implementation of AI models. By identifying these problems early, we can help prevent
serious breakdowns, save on repair costs, and ensure machines operate safely and
efficiently.

10
 TABLE OF CONTENTS

1. Introduction
1.1 Introduction
1.2 Background and Motivation
1.3 Objective 1.4 System Introduction1.5 Description of Achievable........19
1.6 Scope and Significance of the study......................................................19
1.6.1 Technological Integration.......................................................19
1.6.2 Industrial Focus......................................................................19
1.6.3 Sensitivity Improvement.........................................................19
1.6.4 Data Optimization...................................................................20
1.6.5 Real-World Applicability.......................................................20
1.6.6 Experimental Validation.........................................................20

2. Literature Review
2.1 Literature Review
2.2 Existing Approaches and Methods
2.2.1 Lagrangian Motion Amplification2.2.2 Phase-Based Motion
Amplification..........................................................................................................24
2.2.3 Eulerian Motion Amplification................................................24
2.2.4 Lucas-Kanade method.............................................................25
2.2.4.1 Differential techniques..............................................25
2.2.4.2 Region-based matching.............................................25
2.2.4.3 Energy based method.................................................26
2.2.4.4 Phased based Method................................................26
2.2.4.5 Lucas-Kanade method...............................................26
2.2.5 Machine Learning Algorithms...............................................27
2.3 Project flow
3. Methodology
3.1 Hardware Implementation
3.2 Video Recording3.3 Motion Amplification...........................................32
3.4 Optical Flow..........................................................................................34

11
 3.4.1 Optical flow Constraint Equation3.4.2 Lukas Kanade
Algorithm................................................................................................................36
3.5 Datasets..................................................................................................38
3.6 Feature Extraction..................................................................................38
3.6.1 Time-Domain Features............................................................38
3.6.1.1 Sum ...........................................................................38
3.6.1.2 Average .....................................................................39
3.6.1.3 Maximum ..................................................................39
3.6.1.4 Standard Deviation ...................................................39
3.7 Machine Learning .................................................................................39
3.8 Support Vector Machines .....................................................................40

4. Results and discussion ....................................................................................43

4.1 Simulation Results ................................................................................44
4.1.1 Input Data ...............................................................................44
4.1.2 Results from Motion Amplification ........................................44
4.1.2.1 Normal input video and its results ............................44
4.1.2.2 Abnormal Input Video and its Results .....................45
4.1.2.3 Video testing table ....................................................47
4.1.3 Results from optical Flow .......................................................48
4.1.4 Result of Features extraction ..................................................49
4.1.5 Result of SVM ......................................................................... 50

5. Conclusion ........................................................................................................52
References …………………………………………...………………………
54
LIST OF FIGURES

Fig 2.1 Eulerian Motion Amplification ........................................................................25

12
 Fig 2.2 Lucas-Kanade method 27
Fig 2.3 Flow diagram of Project 28
Fig 3.1 Project Methodology 30
Fig 3.2 Hardware Design 31
Fig 3.3 Flow diagram of Eulerian Motion Amplification 34
Fig 3.4 Optical flow 35
Fig 3.5: Flow diagram of Lukas Kanade 37
Fig 3.6 hyperplane in Support Vector in SVM 41
Fig 4.1 Normal without amplify video frame ............................................................................................
Fig 4.2 Normal Amplify video frame .........................................................................................................
Fig 4.3 Abnormal without amplify video frame.........................................................................................
Fig 4.4 Abnormal amplify video frame ......................................................................................................
Fig 4.5 video Resolution 1080x1920p Computation for different fps........................................................
Fig 4.6 Normal Video Optical Flow result..................................................................................................
Fig 4.7 Abnormal Video Optical Flow result..............................................................................................
Fig 4.8 Show the Maximum behavior of Normal and Abnormal Value.....................................................
Fig 4.9 Summation behavior of Normal and Abnormal Value...................................................................
Fig 4.10 Average behavior of Normal and Abnormal Value......................................................................
Fig 4.11 Standard Deviation of Normal and Abnormal Value...................................................................
Fig 4.12 Plot of two classes Confusion matrix using four feature..............................................................

LIST OF TABLES

Table 4.1 Input Parameter for Motion Amplification .......................................................44

13
Table 4.2 Video Resolution 1080x1920p Computation for different frame rate .............47
Table 4.3 Performance evaluation of SVM classifier with four feature ...........................51

ABBREVIATIONS

(AI) Artificial Intelligence

14
(LMA) Lagrangian Motion Amplification

(EMA) Eulerian Motion Amplification

(OFCE) Optical flow constraint equation

(SSD) Sum of Squared Differences
(SVM) Support Vector Machines
(ML) Machine learning

15
Chapter 1:
Introduction

16
1.1 Introduction:
The rise of Industry and the integration of smart technologies into industrial systems have
emphasized the importance of predictive maintenance and anomaly detection. Our project
give integrating advanced artificial intelligence (AI) techniques with motion amplification
technology to enhance the detection of irregularities in machine vibrations. This innovative
approach aims to transform the way predictive maintenance is conducted, offering a more
precise and proactive solution to identifying potential faults in machinery.
Vibration analysis is a well-established method for monitoring the health of mechanical
systems. Traditionally, this process involves manual inspection and basic statistical analysis,
which can be both labor-intensive and prone to inaccuracies. It significant improve by
employing AI motion amplification to visually enhance minute vibrations. This technique
amplifies subtle motions that are often undetectable by the human eye, making it easier to
identify anomalies that could indicate mechanical issues.
The heart of our system is a sophisticated AI framework that leverages machine learning and
deep learning models trained on extensive historical vibration data. These models are capable
of distinguishing between normal operational patterns and anomalies that deviate from these
patterns. By continuously learning and adapting to new data, the AI models improve their
accuracy over time, ensuring reliable detection of potential faults. The integration of AI
motion amplification with these models provides a comprehensive tool for real-time anomaly
detection.
By combining the visual amplification of vibrations with state-of-the-art AI techniques, our
project offers a novel solution for enhancing the accuracy and reliability of anomaly
detection in industrial machinery. This innovative approach not only improves operational
efficiency but also contributes to the broader objectives of smart manufacturing and
maintenance in the era of Industry.

1.2 Background and Motivation:

 Background
In the landscape of industrial machinery, the monitoring of vibration patterns has long been
employed as a key diagnostic tool for assessing equipment health. However, conventional
methods often encounter limitations in their ability to discern subtle anomalies, leading to

17
 potential operational risks and increased maintenance costs. As industries increasingly rely
on predictive maintenance strategies, there arises a demand for advanced technologies to
enhance the efficacy of anomaly detection in vibration patterns. The background of this
research project is rooted in the recognition of these challenges, motivating the exploration of
a novel approach: the integration of artificial intelligence (AI) with motion amplification
technology. This amalgamation aims to address the shortcomings of traditional methods by
leveraging AI-driven algorithms to intelligently analyze vibration data and employing motion
amplification to highlight otherwise imperceptible patterns. By situating the project in this
context, it seeks to contribute to the evolution of predictive maintenance practices, offering a
sophisticated solution for the early identification of anomalies and the optimization of
machinery reliability in industrial environments

 Motivation
The motivation behind undertaking a project on Anomaly Detection in Vibration Pattern
using AI-Driven Motion Amplification stems from the critical need within industries to
revolutionize predictive maintenance strategies and enhance machinery and structural health
monitoring. Traditional methods of vibration analysis have proven effective but often
struggle to detect subtle 15 anomalies early enough to prevent significant breakdowns or
failures. The motivation for this project arises from several key factors:
 Enhancing Early Fault Detection
 Advancing Predictive Maintenance Techniques
 Overcoming Challenges in Traditional Analysis
 Industry Relevance and Real-World Impact
 Promoting Safety and Cost-Efficiency

1.3 Objectives:
Our main objective is to develop an innovative system that leverages AI and motion
amplification technology to enhance the detection and diagnosis of anomalies in the
vibration patterns of industrial machinery. By visually amplifying subtle vibrations
(Eulerian Video Magnification) and employing advanced machine learning models, the

18
 system aims to provide accurate, real-time monitoring and predictive maintenance
capabilities, ultimately improving the reliability, efficiency, and operational longevity of
industrial equipment.
In Anomaly detection of vibration some problem occur which are required to be solved.
 Use high-quality sensors to capture precise vibration data.
 Apply techniques to filter out irrelevant noise from the vibration data.
 To amplify small vibrations so they can be easily seen and analyzed.
 Train AI models to recognize normal and abnormal vibration patterns.
 Set up systems to process vibration data instantly as it is collected.
 Connect the system with maintenance scheduling tools to automate repair alerts.
 Design the system to be easily adaptable to different machines and
industrial settings

1.4 System Introduction:

Presents an advanced system designed to enhance the reliability and efficiency of
industrial machinery maintenance. Utilizing AI-driven motion amplification, the system
visually enhances subtle vibrations, making potential anomalies more apparent.

The core of our system is a sophisticated AI framework that employs machine learning
and deep learning models trained on extensive historical vibration data. These models
accurately differentiate between normal operational patterns and anomalies that indicate
potential faults. By continuously processing and analyzing real-time data, the system
ensures timely detection of any deviations.

A user-friendly interface allows operators to easily visualize amplified vibrations and

identified anomalies, providing actionable insights for predictive maintenance. The
system also integrates seamlessly with existing maintenance management tools,
automating repair alerts and scheduling, thus reducing downtime and maintenance costs.

19
1.5 Description of Achievable:
Our aims to achieve significant advancements in anomaly detection within vibration
patterns using AI motion amplification technology. By leveraging this innovative
approach, we seek to enhance the accuracy and timeliness of anomaly detection,
ultimately enabling proactive maintenance measures that reduce downtime and minimize
repair costs. Through the development of real-time monitoring capabilities and an
intuitive user interface, our system will empower operators to quickly identify and
respond to potential faults, thereby contributing to improved operational efficiency and
machinery longevity. Additionally, the seamless integration with existing maintenance
systems will streamline workflows, automating repair alerts and scheduling to further
optimize maintenance practices.

1.6 Scope and Significance of the study:

The significance of this study lies in its potential to transform machinery health
monitoring and predictive maintenance strategies across various industries:

1.6.1 Technological Integration

Explore the integration of artificial intelligence (AI) algorithms with motion

amplification technology to create an effective system for detecting anomalies in
vibration patterns.

1.6.2 Industrial Focus

Concentrate on the application of the developed system to industrial machinery,

emphasizing the importance of early anomaly detection for preventing operational
failures.

1.6.3 Sensitivity Improvement

Enhance the sensitivity of vibration analysis to detect subtle anomalies that may be
overlooked by conventional methods.

20
1.6.4 Data Optimization

Optimize methods for collecting vibration data from industrial machinery, ensuring the
relevance and quality of data for accurate anomaly detection.

1.6.5 Real-World Applicability

Design the system with considerations for real-world implementation, addressing factors
such as scalability and adaptability to diverse industrial environments.

1.6.6 Experimental Validation

Conduct controlled experiments using calibrated vibration sensors to validate the
effectiveness of the developed system under various industrial conditions.

21
 Chapter 2:
Literature Review

22
2.1 Literature Review:
Monitoring vibration patterns is crucial in various applications, including structural health
monitoring and industrial machinery, as it provides valuable insights into the condition and
performance of equipment, structures, and systems. Vibration analysis is a widely used
technique that involves the measurement and analysis of vibrations to understand the
dynamic behavior of objects.

Motion amplification is a technique used to enhance subtle motions or movements in videos

to make them more visible and analyzable. The idea is to amplify the magnitude of small
displacements that may not be easily perceptible to the naked eye, allowing for a more
detailed examination of dynamic behaviors in various contexts. This technique is particularly
valuable in fields such as structural engineering, mechanical diagnostics, and medical
imaging. [1]

The process of motion amplification involves capturing video footage of a scene, tracking the
motion of interest, and then applying algorithms to amplify and visualize the motion. By
exaggerating the motion, engineers, researchers, and analysts can gain insights into the
behavior of structures, machinery, or biological systems that might otherwise go unnoticed.

Optical flow is a computer vision technique that aims to characterize and quantify the
apparent motion of objects within a sequence of images or video frames. Built on the
principle of brightness constancy, it assumes that the intensity of pixels remains constant
across frames. Optical flow generates a 2D vector field representing motion, where each
vector corresponds to the displacement of pixels between consecutive frames. The
fundamental equation incorporates spatial and temporal intensity gradients. Algorithms such
as the Lucas-Kanade method, Horn-Schunck method, and Farneback method are employed to
estimate optical flow, with applications ranging from video compression and object tracking
to motion analysis in robotics and autonomous vehicles. Despite challenges like the aperture
problem and ambiguity, optical flow remains a cornerstone in computer vision for
understanding dynamic scenes and extracting crucial motion information. [3]

23
 The development of an AI model for anomaly detection in motion-amplified vibration data
involves a multi-step process. Initially, motion-amplified vibration data is collected and
preprocessed to handle noise and outliers. Relevant features are then extracted, including
statistical measures like mean, standard deviation, and Mean, Sum and Average value over
time. For optimal anomaly detection, a choice between traditional machine learning
algorithms like Isolation Forest, One-Class SVM. The model is trained on a labeled dataset,
comprising both normal and anomalous instances, and its performance is evaluated using
metrics like precision, recall, and F1 score. After validation, the model is tested on unseen
data and deployed for continuous monitoring, providing a robust solution for early fault
detection and predictive maintenance in various applications. [6]

2.2 Existing Approaches and Methods:

The existing approaches for anomaly detection in machines span a spectrum of

methodologies, from traditional signal processing to advanced machine learning and digital
twin integration. The ongoing exploration of these methods reflects a dynamic and evolving
landscape aimed at enhancing the efficiency and accuracy of anomaly detection systems in
industrial applications.

The motion Amplification are categorized as Lagrangian Motion Amplification and Eulerian
Motion Amplification. However, motion estimation is a computationally expensive operation
that can be bypassed given a suitable assumption. Our goal is to reveal temporal variations in
videos that are difficult or impossible to see with the naked eye and display them in an
indicative manner.

2.2.1 Lagrangian Motion Amplification:

Lagrangian Motion Amplification (LMA) is an advanced video processing technique

used to amplify subtle movements in a sequence of frames, making otherwise
imperceptible motions visible to the naked eye. This type of motion analysis tracks the
movement (via techniques such as optical flow of particles over time. Horn technically

24
 describes perception of motions as changes of brightness patterns due to relative
movement between visual sensors retinas in human eyes or CMOS arrays in camera.
Optical flow techniques capture these changes in terms of vector fields, accurate
estimations of motions at (sub-) pixel-wise levels given certain assumptions or
constraints about smoothness in target motions and integrity of target objects. [2]

2.2.2 Phase-Based Motion Amplification:

Phase-Based Motion Amplification leverages the phase information from video frames to
enhance motion visibility. By decomposing the video into various spatial and temporal
frequency bands, the technique modifies the phase of these bands to amplify the motion.
After adjusting the phase, the bands are recombined to produce a video where small
movements are visibly amplified. This method is effective for detecting minute vibrations
and subtle motions in structural health monitoring and biomedical applications. [5]

2.2.3 Eulerian Motion Amplification:

Eulerian Motion Amplification (EMA) is a sophisticated video processing technique
designed to enhance and visualize subtle motions and changes in video sequences that are
typically invisible to the naked eye.

The process of Eulerian Motion Amplification begins with the spatial decomposition of
video frames using techniques like the Laplacian pyramid, which breaks down the frames
into different spatial frequency bands. This decomposition allows for isolating specific
frequency components where subtle changes occur. After the spatial decomposition, the
temporal processing stage involves applying temporal filters to the decomposed frames to
identify and extract the frequency bands corresponding to the subtle motions of interest.
Once the subtle motions are identified, they are amplified by a predefined magnification
factor. This amplification is applied to the pixel intensity changes across the video
frames. After amplifying the desired motions, the modified frames are reconstructed by

25
 reversing the spatial decomposition process, combining the amplified components to
form the final video. [1]

Figure 2.1: Eulerian Motion Amplification

2.2.4 Lucas-Kanade method:

There are 4 major optical flow techniques which are

2.2.4.1 Differential techniques:

Differential techniques are among the most commonly used methods for estimating
optical flow. These techniques rely on the assumption that the intensity of a moving pixel
remains constant over time, leading to the optical flow constraint equation (OFCE). The
basic idea is to use the spatial and temporal derivatives of the image intensity to estimate
the motion. One of the most well-known differential techniques is the Lucas-Kanade
method, which approximates the optical flow by assuming that the flow is constant within
a small neighborhood around each pixel. By solving a set of linear equations using least
squares optimization, this method provides an estimate of the motion vectors for each
pixel. [9]

2.2.4.2 Region-based matching:

Region-based matching techniques aim to find correspondences between image regions

(blocks) in consecutive frames. Unlike differential techniques that operate at the pixel
level, region-based matching works at a higher level of abstraction by comparing blocks

26
 of pixels. The most common approach is block matching, where a block in the current
frame is matched with a block in the next frame by minimizing a similarity measure, such
as the sum of absolute differences (SAD) or the sum of squared differences (SSD).

2.2.4.3 Energy based method:

Energy-based methods formulate the optical flow estimation problem as an optimization

problem, where the goal is to minimize a global energy function. This energy function
typically comprises two terms: a data term that enforces the brightness constancy
assumption and a regularization term that imposes smoothness on the optical flow field.

2.2.4.4 Phased based Method:

Phase-based approaches utilize the phase information of image signals to estimate optical
flow. These methods are based on the observation that the phase of a signal, as opposed
to its amplitude, is more robust to changes in illumination and other artifacts. By
analyzing the phase shifts between corresponding points in consecutive frames, phase-
based methods can accurately estimate the motion.

2.2.4.5 Lucas-Kanade method:

The Lucas-Kanade method is a widely used optical flow algorithm in Computer vision,
specifically designed for estimating motion between consecutive frames in a video
sequence. By analyzing the spatial gradients of intensity in the x and y directions, the
Lucas-Kanade method formulates a system of linear equations to estimate the flow
parameters. This technique is particularly suitable for scenarios where motion is
relatively smooth and exhibits local rigidity, making it a valuable tool in various
applications such as object tracking, video stabilization, and motion analysis in robotics.
The Lucas-Kanade method relies on the assumption that the brightness of a point in an
image remains constant over time as it moves. This is known as the brightness constancy
assumption. To track the motion, the algorithm examines how a small patch around a
point of interest shifts from one frame to the next. It uses mathematical equations to
determine the displacement (movement) of the patch by solving a set of linear equations
derived from this assumption. It helps to reduce shakiness by tracking and compensating
27
 for unwanted motion. In object tracking, it allows for following the motion of objects,
making it valuable for surveillance and video analysis. [3]

Figure 2.2: Optical Flow using Lucas-Kanade method

2.2.5 Machine Learning Algorithms:

Machine learning approaches have gained prominence in recent years for their ability to
handle complex patterns within vibrational data. Support Vector Machines (SVM), neural
networks, and decision trees are applied for classification tasks, distinguishing between
normal and anomalous vibrational signatures. Supervised learning methods, trained on
labeled datasets, prove effective in capturing diverse fault scenarios.

The Machine learning involves training a model using a dataset, where the model learns
to associate inputs with desired outputs. This training process can be supervised, where
the algorithm is provided with labeled data (e.g., images with tags), or unsupervised,
where the algorithm finds patterns and relationships in data without explicit labels (e.g.,
grouping similar items together).

28
2.3 Project flow:

Figure 2.3: Flow diagram of Project

This flowchart outlines a real-time anomaly detection system in vibration data using
motion amplification. It starts with recording video using a camera, followed by
amplifying the motion in the video. Optical flow techniques are then applied to analyze
the motion patterns, and this data is collected into a dataset. Features are extracted from
this dataset and used to train and test a machine learning model. The model then makes a
decision based on the input data: if a fault is detected, it indicates so; otherwise, it
confirms there is no fault.

29
Chapter 3:
Methodology

30
In our methodology, camera is used to record videos of machinery or structures experiencing
vibrations. The recorded videos are then processed with motion amplification techniques to make
even tiny vibrations more noticeable. Optical flow algorithms are applied to analyze these
amplified vibrations and track their movements. From this data, key features are extracted that
describe the vibration patterns. These features are used to train a machine learning model to
recognize normal vibration patterns and identify anomalies. The trained model can then detect
and signal any unusual or faulty vibrations, helping to predict and prevent potential issues in
machinery or structures. Our final approach is given below

Figure 3.1: Project Methodology

3.1 Hardware Implementation:

The hardware setup for detecting anomalies is designed for practical testing and includes
essential components. 3-phase induction motor, commonly used in industries for its
reliability. This motor generates the necessary vibrations for the analysis. Rigid couplings
connect the motor to other parts, ensuring there is no looseness in the system and that the
motor's rotational motion is efficiently transferred. A well-designed shaft links the motor to

31
 the bearing housing, which holds the bearings. These bearings are crucial for keeping the
shaft stable, reducing friction, and ensuring smooth rotation. The setup also includes a
mechanical load that mimics real-world conditions and affects the vibration patterns. This
complete testbed, with the motor, couplings, shaft, bearing housing, and mechanical load,
accurately represents industrial environments and allows for controlled experiments. The
data collected from this setup helps develop algorithms for detecting anomalies, aiding in the
advancement of predictive maintenance for rotating machinery.

Figure 3.2: Hardware Design

3.2 Video Recording:

Video recording is the process of capturing moving images, typically using a camera or other
recording device. It involves converting light patterns into electronic signals, which are then
stored as digital files. We utilized high resolution cameras optimized for our application,
specifically [camera model and specifications], capable of recording [resolution,
1080p*1920p] video at [frame rate, e.g., 15 frames per second]. Environmental
considerations were paramount, with controlled lighting and minimal disturbances during
recording. Cameras were strategically positioned for optimal coverage of key areas,
enhancing the efficacy of motion amplification algorithms. To ensure synchronization with
vibration sensor data, temporal alignment mechanisms were implemented. This setup aimed
to capture high-quality footage crucial for subsequent motion amplification and anomaly
detection.

32
3.3 Motion Amplification:
Our goal is to reveal temporal variations in videos that are difficult or impossible to see with
the naked eye and display them in an indicative manner. Our method, which we call Eulerian
Video Amplification, Takes a standard video sequence as input, and applies spatial
decomposition, followed by temporal filtering to the frames. The resulting signal is then
amplified to reveal hidden information. For example, human skin color varies slightly with
blood circulation. This variation, while invisible to the naked eye, can be exploited to extract
pulse rate. The success of these tools motivates the development of new techniques to reveal
invisible signals in videos. We combine the spatial and temporal processing of videos can
amplify subtle variations that reveal important aspects of the world around us. [1]

Our basic approach is to consider the time series of color values at any spatial location (pixel)
and amplify variation in a given temporal frequency band of interest. Our approach combines
spatial and temporal processing to emphasize subtle temporal changes in a video. We first
decompose the video sequence into different spatial frequency bands. These bands might be
magnified differently because they might exhibit different signal-to-noise ratios or they
might contain spatial frequencies for which the linear approximation used in our motion
magnification. When the goal of spatial processing is simply to increase temporal signal-to-
noise ratio by pooling multiple pixels, we spatially low-pass filter the frames of the video and
down sample them for computational efficiency. We then perform temporal processing on
each spatial band and then we apply a band pass filter to extract the frequency bands of
interest.
We take input video to process by Eulerian video magnification, there are four steps the user
needs to take:

 select a temporal bandpass filter

 select an amplification factor (α)
 select a spatial frequency cutoff (specified by spatial wavelength, λc) beyond which an
attenuated version of α is used
 select the form of the attenuation for α

33
For motion magnification, a filter with a broad passband is Preferred; for a color
amplification of blood flow, a narrow passband produces a more noise-free result. Low-order
IIR filters can be useful for both color amplification and motion magnification. We consider
the time series corresponding to the value of a pixel in a frequency band and apply a
bandpass filter to extract the frequency bands of interest.

We typically use the notation I (x, t) to represent the intensity (of an image frame) at the
position x and time t. Initially, we have I (x, 0) = f (x); then after going through translational
motion, we see that the observed intensities can be expressed as:

I ( x , t )=f ¿ (3.1)

Where δ (t) denotes some displacement at time t from its original position.

Motion magnification aims to produce the amplified signal:

^I ( x , t )=f ¿ (3.2)

For some amplification factor α.

The image at time t can be approximated using the standard first-order Taylor series
expansion about x:

(
I ( x , t )=f x + δ ( t ) ≈ f ( x ) +δ ( t )
∂ f (x)
∂x ) (3.3)

Let B(x, t) denote the output from a temporal bandpass filter that filters out f (x), i.e.

∂ f (x)
B ( x , t )=δ ( t )
∂x
(3.4) Then the result of amplifying the bandpass
output and subsequently adding it back to the image gives:

^I ( x , t )=I ¿ (3.5)

≈f¿ (3.6)

34
 ≈f¿ (3.7)

Comparison between two method Eulerian and Lagrangian motion amplification is that
Lagrangian approaches explicitly track motions, while our Eulerian approach does not they
can be used for complementary motion domains. Lagrangian approaches, work better to
enhance motions of fine point features and support larger amplification factors, small
amplifications. Our technique is apply on any type of motion.

Figure 3.3: Flow diagram of Eulerian Motion Amplification

3.4 Optical Flow:

Optical flow plays an important role in estimating the motion of the objects from a sequence
of images. Basically, optical flow is a velocity field of the image generated from the
transformation of one image into the next image in a sequence

35
 Built on the principle of brightness constancy, it assumes that the intensity of pixels remains
constant across frames. Optical flow generates a 2D vector field representing motion, where
each vector corresponds to the displacement of pixels between consecutive frames. The
fundamental equation incorporates spatial and temporal intensity gradients. We will only
implement differential approaches mainly on Lucas Kanade. The differential techniques
computes faster than other type of method due to its simple set of linear equation. [3]

3.4.1 Optical flow Constraint Equation:

The optical flow cannot be computed at a point in the image independently of
neighboring points without introducing additional constraints, because the velocity field
at each image point has two components while the change in image brightness at a point
in the image plane due to motion yields only one constraint. Assumption made by most of
the optical flow method is that the intensity I of moving points is constant over a period
of time. This is known as brightness constancy assumption.

I¿ (3.8)

Figure 3.4: Optical flow

The optical flow constraint equation (OFCE) is obtained by using Taylor expansion on
(1) and dropping its nonlinear terms. Therefore the OCFE can be expressed in the form as

36
 I x u+ I y v+ I t =0 (3.9)

dy dy
Where u and V represent the optical flow vectors ( , ), and ( I x , I y , I t ) represent the
dt dt
derivatives of image intensities at coordinate (x, y, t).

There is little hope of recovering the velocities if every point of brightness pattern can
move independently. This constraint assumes that adjacent points or pixels in an image or
video should exhibit minimal variation in intensity or color, reflecting the natural
continuity observed in real-world scenes. One way to express the additional constraint is
to minimize the square of the magnitude of the gradient of the optical flow velocity.

¿And¿ (3.10)

Another measure of the smoothness of the optical flow field is the sum of the squares of the
Laplacians of the X- and y components of the flow. The Laplacians of u and v are defined

2
∇ u=¿ (3.11)

3.4.2 Lukas Kanade Algorithm:

Lukas Kanade assumed as an additional constraint that the optical flow is varying
smoothly long the neighboring object points that possessed the same velocity. The least
squares estimator is used to minimize the squared error. [3]

∑ (u , v )=∑ g( ⃗x ¿)[ I x u+ I x v+ I t ]2 ¿ (3.12)

LK ⃗x

Here g(x) is the Gaussian weighting function that determines the support of the centered
estimator. Thus, two partial derivative equations can be expressed as

∂ E(u , v )
=∑ g ( ⃗x ) [u I 2x +v I x I y + I x I t ]=0 (3.13)
∂u ⃗x

∂ E(u , v )
=∑ g ( ⃗x ) [u I 2y + u I x I y + I y I t ]=0 (3.14)
∂v ⃗x

Lukas Kanade solves these equations using Least Mean Square estimation
−1
u⃗ =( A A ) A b
T T
(3.15)

37
 [ ∑ g I 2x ∑ g I x I y
∑ g I x I y ∑ g I 2y [] uv ]=−[∑∑ gg II II ]
x

y
t

t
(3.16)

T T
A aA b

The images of optical flow are computed using partial derivatives between pixels in the
x, y and t directions

( )
I x =m x∗( I 1+ I 2 ) , mx =
1 −1 1
4 −1 1
(3.17)

I =m ∗( I + I ) ,m = (
1)
1 −1 −1
y y 1 2 x (3.18)
4 1

I =m ∗( I −I ) , m = (
4 1 1)
1 1 1
t t 2 1 t (3.19)

By using optical flow we can extract motion where it Occur. This method is especially
effective for detecting and following object movement in videos, making it a fundamental
tool in applications like video stabilization, object tracking, and optical flow estimation.
Its simplicity and efficiency make it a popular choice for real-time motion analysis.

38
 Figure 3.5: Flow diagram of Lukas Kanade

3.5 Datasets:
A dataset is a collection of related data, typically organized in a structured form that is used
for analysis and decision-making. By recording videos of machinery and equipment and
apply optical flow (Lukas Kanade Algorithm) we collect our datasets while they operate
under normal and faulty conditions. These videos capture the subtle vibrations and
movements that occur during the operation of the machinery. By using high-resolution
cameras and recording at various angles and speed. This involves breaking down the video
frames and analyzing the movement within each frame. We process them to extract relevant
features that highlight the vibrations.

3.6 Feature Extraction:

Feature extraction is the process of identifying important characteristics from the raw
vibration data that can help distinguish between normal and abnormal patterns. Imagine you
have a camera recording the movement of a machine. The video captures every shake and
jitter as the machine operates, instead of manually watching hours of footage.

39
 First, you process the video frames to make the data manageable. This involves breaking
down the video into individual frames. Then, for each frame, you can detect and track
specific points or areas of the machine that show movement. By analyzing these movements
over time, you can measure the intensity, frequency, and patterns of the vibrations.
Techniques such as Optical Flow can help track the direction and speed of moving parts.

Key Feature Extraction Techniques:

3.6.1 Time-Domain Features:

Time-domain features capture various aspects of the vibrational signal's behavior over
time. Common time-domain features include

3.6.1.1 Sum:
The sum feature involves adding up all the values in a dataset. The sum would be the
total of all these measurements. This feature can provide insights into the overall energy
or magnitude of the vibrations experienced by the machinery.

3.6.1.2 Average:
The average is calculated by dividing the sum of all values by the number of values. It
represents the central value or the typical measurement in the dataset.

3.6.1.3 Maximum:

The maximum value is the highest value in the dataset. This feature is particularly useful
in identifying peak vibrations. By analyzing the maximum vibration levels, one can
detect potential outliers.

3.6.1.4 Standard Deviation:

The standard deviation measures the amount of variation or dispersion in a dataset. A low
standard deviation indicates that the values are close to the mean, while a high standard
deviation shows a wide range of values.

Maximum max ¿ (3.20)

40
 Standard deviation

xσ
√ 1
N
¿¿ (3.21)

N
1
Mean Xmean =
N
∑ ¿ 1(x k ) (3.22)
K

3.7 Machine Learning:

Machine learning (ML) is a branch of artificial intelligence (AI) that focuses on creating
systems that can learn from data and improve their performance over time without being
explicitly programmed. At its core, machine learning involves using algorithms to analyze
data, identify patterns, and make decisions based on these patterns. This capability allows
machines to perform tasks that were traditionally thought to require human intelligence, such
as recognizing speech, detecting anomalies, and making predictions.

There are three types of Machine learning:

1. Supervised Learning:

In supervised learning, the algorithm is trained on a labeled dataset, where each input is
paired with its corresponding output. The model learns to map inputs to outputs and can
then make predictions on new, unseen data.

2. Unsupervised Learning:

Unsupervised learning involves training the algorithm on an unlabeled dataset. The

model explores the data's inherent structure, identifying patterns or relationships without
predefined output labels.

3. Reinforcement Learning:

41
 Reinforcement learning involves an agent interacting with an environment and learning
to make decisions by receiving feedback in the form of rewards or penalties. The agent
aims to maximize cumulative rewards over time.

We used both approaches of Supervised Learning.

3.8 Support Vector Machines (SVM):

Support Vector Machine (SVM) is a powerful supervised machine-learning algorithm used

for classification and regression tasks. The primary objective of SVM is to find a hyperplane
in a high-dimensional space that best separates the data points of different classes. SVM is
particularly effective in scenarios where the decision boundary is not linear and has complex
shapes. [7]

Particularly, One-Class SVM is used for anomaly detection. It learns the boundary of normal
vibration patterns and identifies points that fall outside this boundary as anomalies.

Figure 3.6: hyperplane in Support Vector in SVM

Key Component of SVM

1. Hyperplane:

42
 In SVM, a hyperplane is a decision boundary that separates different classes in the feature
space. For instance, in a two-dimensional space, this boundary is a line, while in three
dimensions, it’s a plane. In higher dimensions, it’s referred to as a hyperplane.

2. Support Vectors:

Support vectors are the data points that are closest to the hyperplane. These points are critical
because they are used to define and maximize the margin.

3. Margin:

The margin is the distance between the hyperplane and the nearest data points from each
class. The goal of SVM is to find the hyperplane that maximizes this margin. A larger margin
is generally associated with better generalization to unseen data.

4. Kernels:

The kernel trick allows SVM to compute the dot product of the transformed data in the high-
dimensional space without actually computing the transformation. This makes the
computation efficient.

43
44
 Chapter 4:
Results and Discussion

4.1 Simulation Results:

The code was run on different software to make more efficient and robust. Following are
the simulation results.

4.1.1 Input Data:

First we record the video with the help of camera. The recorded video is amplify with
the help Eulerian Video Amplification algorithm to see the vibration which cannot see
with the naked eye and display them in an indicative manner. The input parameter for
Motion Amplification.

Parameter

Amplification Factor (α )

spatial frequency cutoff, λ

45
 cutoff frequencies ω1

cutoff frequencies ω2

frame rate of the camera, fs

Table 4.1: Input Parameter for Motion Amplification

4.1.2 Results from Motion Amplification:

The average time taken by the algorithm to process motion amplification for normal
and abnormal is 425.5 sec. The MATLAB simulation result is given below.

4.1.2.1 Normal input video and its results:

We start with normal video recordings of 31 sec video (15 frames per second) of
machinery in operation, captured using high-resolution cameras (1920*1080). The
recording setup often includes stable camera positioning and proper lighting to minimize
any external noise or distortions in the video.

1. Input Normal Result:

Figure 4.1: Normal without amplify video frame

2. Normal Amplify Result:

46
 Figure 4.2: Normal Amplify video frame

4.1.2.2 Abnormal Input Video and its Results:

We start with abnormal video recordings of 31 sec video (15 frames per second) of
machinery in operation, captured using high-resolution cameras (1920*1080). The
recording setup often includes stable camera positioning and proper lighting to
minimize any external noise or distortions in the video.
1. Input Abnormal Result:

Figure 4.3: Abnormal without amplify video frame

47
2. Abnormal Amplify Result:

Figure 4.4: Abnormal amplify video frame

4.1.2.3 Video testing table:

Table 4.2: Video Resolution 1080x1920p Computation for different frame rate

48
 We take different test case on same resolution with different frame rate per second in
order to find out the time for motion amplification execution.

1800
1600 1661.6
1400
1288.5
1200

Times(sec)
1000 955.71
800
600
400 425.4
200
0
Fame Rate/sec
10 20 30 40 50 60 70

Figure 4.5: Graph of video Resolution 1080x1920p Computation for different

frame rate

4.1.3 Results from optical Flow:

Optical flow analysis provide information about the motion of objects between
consecutive video frames. By tracking how pixels move from one frame to the next,
we can detect the direction and speed of movement. We use the optical flow
differential technique approaches mainly on Lucas Kanade. We detect motion from
two dimension (x, y) and store the result in numerical form data. The blue Color in
the below picture show that motion.

1. Optical Flow Normal Video Result:

49
 Figure 4.6: Normal Video Optical Flow result

2. Optical Flow Abnormal Video Result:

Figure 4.7: Abnormal Video Optical Flow result

4.1.4 Result of Features extraction:

The Features extraction include the Sum, Average, Maximum Value and Standard
Deviation which were crucial for identifying any irregularities. The AI algorithms
effectively processed these features to differentiate between normal and abnormal
vibration patterns, making it easier to detect potential issues early on.

50
Figure 4.8: Show the Maximum behavior of Normal and Abnormal Value

Figure 4.9: Show the Summation behavior of Normal and Abnormal Value

51
 Figure 4.10: Show the Average behavior of Normal and Abnormal Value

Figure 4.11: Show the Standard Deviation of Normal and Abnormal Value

4.1.5 Result of SVM:

The results of using Support Vector Machine (SVM), evaluated through a confusion
matrix, showed excellent performance in detecting anomalies in vibration patterns.
The confusion matrix revealed high accuracy with a large number of true positives
(correctly identified anomalies) is 96 and true negatives (correctly identified normal
patterns) is 95. There were very few false positives (normal patterns incorrectly
marked as anomalies) is 0, and false negatives (anomalies missed by the system) is 0,

52
indicating that the SVM was highly reliable in distinguishing between normal and
abnormal vibrations.

Figure 4.12: Plot of two classes Confusion matrix using four feature

Speed Precision Recall F1score

20Hz 1.00 1.00 1.00

Table 4.3: Performance evaluation of SVM classifier with four feature

53
Chapter 5:
Conclusion

54
This thesis presented a comprehensive study on Anomaly Detection in Vibration Using
Motion Amplification, that how advanced technology can improve the maintenance and
monitoring of mechanical systems. By using AI to analyze video-based vibration data, we
were able to detect even the smallest irregularities in the vibration patterns of machinery.
These small changes often signal potential problems that, if identified early, can prevent
major breakdowns and ensure the machinery runs smoothly and efficiently.

The core of our approach involved video-based feature extraction to observe and analyze
vibration patterns. The AI algorithms processed these video features to distinguish
between normal and abnormal vibrations. This method was effective in highlighting very
subtle vibrations that are typically difficult to notice with traditional sensors or human
observation. As a result, our system achieved higher accuracy in detecting anomalies,
which can greatly aid in scheduling timely maintenance and reducing unplanned
downtime. The AI-driven motion amplification technique we used turned out to be a cost-
effective solution. Traditional methods of vibration analysis often require expensive and
sophisticated equipment along with significant manual intervention

Our project illustrates the significant advantages of integrating AI with innovative

techniques like motion amplification for anomaly detection in vibration patterns. The
positive outcomes of our research suggest that such technologies can greatly enhance
maintenance and monitoring processes in various industries. This leads to improved
safety, increased efficiency, and extended equipment lifespan.

55
 References:
1. Wu, H. Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., & Freeman, W. (2012).
Eulerian video magnification for revealing subtle changes in the world. ACM
transactions on graphics (TOG), 31(4), 1-8.
2. Śmieja, M., Mamala, J., Prażnowski, K., Ciepliński, T., & Szumilas, Ł. (2021). Motion
magnification of vibration image in estimation of technical object condition-
review. Sensors, 21(19), 6572.
3. Siong, L. Y., Mokri, S. S., Hussain, A., Ibrahim, N., & Mustafa, M. M. (2009, August).
Motion detection using Lucas Kanade algorithm and application enhancement. In 2009
International Conference on Electrical Engineering and Informatics (Vol. 2, pp. 537-
542). IEEE.
4. Zhang, X., Gu, C., & Lin, J. (2006, June). Support vector machines for anomaly
detection. In 2006 6th World Congress on Intelligent Control and Automation (Vol. 1,
pp. 2594-2598). IEEE.
5. Wadhwa, N., Rubinstein, M., Durand, F., & Freeman, W. T. (2013). Phase-based video
motion processing. ACM Transactions on Graphics (ToG), 32(4), 1-10.
6. Pinedo-Sanchez, L. A., Mercado-Ravell, D. A., & Carballo-Monsivais, C. A. (2020).
Vibration analysis in bearings for failure prevention using CNN. Journal of the Brazilian
Society of Mechanical Sciences and Engineering, 42(12), 628.
7. Amir, R. B., Gul, S. T., & Khan, A. Q. (2016, October). A comparative analysis of
classical and one class SVM classifiers for machine fault detection using vibration
signals. In 2016 International Conference on Emerging Technologies (ICET) (pp. 1-6).
IEEE.
8. Kale, K., Pawar, S., & Dhulekar, P. (2015, September). Moving object tracking using
optical flow and motion vector estimation. In 2015 4th international conference on
reliability, infocom technologies and optimization (ICRITO)(trends and future
directions) (pp. 1-6). IEEE.
9. Verri, A., Girosi, F., & Torre, V. (1990). Differential techniques for optical flow. JOSA
A, 7(5), 912-922.

56