Chapters by Thierry BOUWMANS
Handbook on "Technological Prospects and Social Applications of Society 5.0 ", 2023
Society 5.0 is based on environmental sensor information, equipment operating status, people rela... more Society 5.0 is based on environmental sensor information, equipment operating status, people related information, etc. A part of this information comes from video surveillance applications using static or moving cameras, which often require moving objects detection in their first step. Background subtraction is then used to distinguish the background and the foreground. In literature, the methods employ either mathematical concepts, machine learning or signal-processing models to deal with the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research can be employed in video surveillance applications in Society 5.0. In this chapter, we review different methods developed over time to reach efficient moving objects detection. Thus, we survey methods based on mathematical concepts, machine learning or signal-processing models. Finally, we provide perspectives.
Handbook on " Computer Vision and Internet of Things: Technologies and Applications, 2022
Detection of moving objects is often the first stage in many computer vision applications through... more Detection of moving objects is often the first stage in many computer vision applications through the use of static or moving cameras. Subsequently, the separation process between the background and the foreground is carried out using the background subtraction approach. In the state of the art, it has been cited that the application of machine learning as well as signal processing offer promising results in the face of the challenges to be overcome in test video sequences. However, the fundamental objective is that algorithms based on the background subtraction approach can be used in real applications under the context of IoT. In this chapter, we analyze different background subtraction strategies developed over time with efficient results in real-time applications. Thus, we survey GPU implementations, embbeded implementions in smart cameras and systems, and development of specific architectures using DSP, FPGA and VLSI technologies. Second, we review fog computing and edge computing strategies employed for global video systems in IoT context. Finally, we provide perspectives.
Handbook on "Artificial Intelligence Technologies, Applications, and Challenges", 2020
Background subtraction is one of the fundamental tasks for many robotics and computer vision appl... more Background subtraction is one of the fundamental tasks for many robotics and computer vision applications. Recently, graph signal processing techniques have attained significant attention leading to new advances and insights in the field of background subtraction for video analysis in the past years. In this chapter, we present the concept of blue-noise sampling on graphs for background subtraction, leading to a new active semi-supervised learning technique called ActiveBGS. This algorithm is composed of instance segmentation, background initialization, graph construction, blue-noise sampling on unseen videos, and a semi-supervised learning algorithm. The proposed algorithm has outperformed random sampling-based methods for some challenges in publicly available change detection 2014 dataset for background subtraction.
CRC Press, Taylor and Francis Group, 2020
Mathematical tools, machine learning and signal processing tools have achieved enormous success i... more Mathematical tools, machine learning and signal processing tools have achieved enormous success in computer vision. In this chapter, we present a state-of-art of the progress that have occured in moving objects detection, classification and recognition in video sequences taken by fixed cameras. More specifically, we focus on the last breaktrought made by Deep Learning (DL) that can be used for smart cities. The corresponding computer vision pipeline allows to detect humans and vehicles for smart homes and cities. First, we survey from the origin until 2020 developements in the field of moving objects detection using background subtraction techniques. Second, approaches which have been proposed to classify the extracted moving objects are also reviewed and classified into various categories, including supervised, semi-supervised, and unsupervised learning. More specifically, they cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Auto-Encoder (AE), Deep Belief Network (DBN) and Generative Adversarial Networks (GANs). Third, this chapter reviewed approaches that have been explored and evaluated in the field of face recognition to identify people previously extracted using background subtraction and classified using DNNs. Thus, this survey aims to provide a comprehensive review of the segmentation, classification and recognition of moving objects (i.e humans, vehicles, etc.), covering conventional and recent advances.
World Scientific Publishing, 2020
Machine learning has been widely applied for detection of moving objects from static cameras. Rec... more Machine learning has been widely applied for detection of moving objects from static cameras. Recently, many methods using deep learning for background subtraction have been reported, with very promising performance. This chapter provides a survey of different deep-learning based background subtraction methods. First, a comparison of the architecture of each method is provided, followed by a discussion against the specific application requirements such as spatio-temporal and real-time constraints. After analyzing the strategies of each method and showing their limitations, a comparative evaluation on the large scale CDnet2014 dataset is provided. Finally, we conclude with some potential future research directions.
CRC Press, Taylor and Francis Group, 2019
For the observation of animals and insects in their natural environment, a noninvasive, simple in... more For the observation of animals and insects in their natural environment, a noninvasive, simple intelligent system is necessary. Therefore, a vision-based system is a viable option to obtain information regarding animal and insect behavior in the environment. A visionbased system can be used to study social behaviors within the same group; for example, studies of bees show very close interaction with partners [1]. The system can also analyze the climate impact and changes in the ecosystem when animals, such as birds, interact with their environment [2–5]. A specific application called Fish4Knowledge (F4K) can obtain information on animal species in different climatic conditions, specifically fish behaviors to adapt to the environment in the presence of storms, sea currents, or typhoons [6–9]. According to Iwatani et al., vision-based systems can also aid in the design of robots that mimic the locomotion of certain animal species of interest [10]. Researchers in environmental development, biology, or ethology are interested in the behavior of animals that are important to maintain environmental balance, such as bees that are pollinators of flowers on the planet. Most observations of insects and animals are made in a totally natural environment; therefore, it is also of interest to detect foreign objects that do not belong to the environment and the damages that they may cause. In this chapter we will try to analyze the main works and publications in visual surveillance in natural environments that make use of the background subtraction approach with corresponding videos and experiments whose purpose was the detection of foreground objects and background modeling for specific applications. Thus, this chapter refers only to background subtraction methods that are previously and currently used in these applications. For more details on the existing methods under the background subtraction approach, the reader can refer to many surveys in the field [11–18].
CRC Press, Taylor and Francis Group, 2019
The increase in video camera usage has led to background subtraction being used more and more in ... more The increase in video camera usage has led to background subtraction being used more and more in different visual surveillance applications of human activities, which involve different uncontrolled environments and types of targets to be detected. In video surveillance systems, the main objectives are the identification and tracking of objects. Traffic scenes are the most common environments for the detection of incidents such as vehicles stopped on roads [1–4] or to monitor vehicular traffic on highways, classifying them as empty, fluid, heavy, or with traffic jams. This makes it necessary to detect, track, and count vehicles [5–7]. The background subtraction approach can also be applied in congestion in urban traffic areas, for the detection of illegal parking [8–12], and for the detection of free parking places [13–15]. Safety at train stations and airports is another area of interest because baggage theft is a frequent problem.
Within maritime scenes, surveillance can include counting the number of ships, as well as detecting and tracking vessels in fluvial channels. Other settings are stores, that is, the detection and monitoring of customers [16–19]. Background subtraction also plays an important role in rapid decision-making in sports activities. For example, Hawk-Eye software has become a key part of soccer and tennis contests. Background subtraction can be used for precise analysis of athletic performance, because it has no physical effect on the athlete as in Tamas et al. [20] for rowing motions, and for surveillance as in Bastos [21] for activities of surfers. In this chapter, we thus attempt to survey the main visual surveillance applications of human activities that use background subtraction in their process by classifying them in terms of aims, environments, and objects of interest. We reviewed only the publications that specifically address the problem of background subtraction in the concerned applications with experiments on corresponding videos. Thus, this chapter refers only to background subtraction methods that are previously and currently used in these applications. To have an overview of all the background subtraction methods in research, the reader can refer to numerous surveys [22–29] in the field.
In this chapter, we present OR-PCA with its application to background/foreground segmentation. Fi... more In this chapter, we present OR-PCA with its application to background/foreground segmentation. First, we give an overview of stochastic RPCA (also known as OR-PCA), then background/foreground segmentation is presented using stochastic RPCA.
The recent advances in robust matrix and tensor factorization are fundamental and can be applied ... more The recent advances in robust matrix and tensor factorization are fundamental and can be applied to background modeling and foreground detection for video surveillance. It was for this reason that the LRSLibrary was developed. The goal is to provide an easy-to-use library to apply low-rank and sparse decomposition tools for background modeling and subtraction in videos. The library is open-source and free for academic/research purpose (non-commercial).
RPCA via decomposition in low-rank and sparse matrices proposed by Candes et al. in 2009 is curre... more RPCA via decomposition in low-rank and sparse matrices proposed by Candes et al. in 2009 is currently the most investigated RPCA method. In this chapter, we reviewed this method and all these modifications in terms of decomposition, solvers, incremental algorithms and real time implementations. These different RPCA methods via decomposition in low rank and sparse matrices are fundamental in several applications. Indeed, as this decomposition is nonparametric and does not make many assumptions, it is widely applicable to a large scale of problems ranging from latent variable model selection, image processing, video processing and 3D computer vision. Here, we choose to focus on the application of background/foreground separation which is a representative application of RPCA, and which witnessed very numerous papers (more than 200) since 2009. Applying RPCA via decomposition in low rank and sparse matrices in video-surveillance, the background sequence is modeled by the low-rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. So, the different advances in the different problem formulations of the decomposition into low-rank and sparse matrices are fundamental and can be applied to background modeling and foreground detection in video surveillance.
Considering all of this, this chapter develops a comprehensive review on the different RPCA methods based on decomposition into low-rank and sparse matrices. The rest of this chapter is organized as follows. Firstly, we provide a preliminary overview and a unified view of RPCA via decomposition into low-rank and sparse matrices. Then, we review each original method in its section). For each method, we investigate how it is solved, and if incremental and real-time versions are available for real time applications such as background/foreground separation. Finally, we conclude with promising research directions.
The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to perform ba... more The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to perform background subtraction. First released in March 2012, currently the library offers 32 background subtraction algorithms. The source code is available under GNU GPL v3 license and the library is free for non-commercial use, open source and platform independent. Note that the license of the algorithms included in BGSLibrary not necessarily have the same license of the library. Some authors do not allow that their algorithms will be used for a commercial purpose, first is needed to contact them to ask permission. However, by default, we decided to adopt the GPL-v3 license.
The BGSLibrary also provides one Java based GUI (Graphical User Interface) allowing the users to configure the input video-source, region of interest, and the parameters of each BS algorithm. A MFC-based GUI is also provided for a quick access in Windows computers. But, a QT-based GUI is coming for a platform independent usage. To build/run the BGSLibrary, it is necessary to have the OpenCV library installed previously. Everyone is invited to collaborate with the BGSLibrary. In this chapter some efforts has been made for how to make and add your contributions in the library."
This chapter gives an overview of recent background modeling and foreground detection, and presen... more This chapter gives an overview of recent background modeling and foreground detection, and presents resources, datasets and codes publicly available.
This chapter gives an overview of traditional background modeling and foreground detection, and p... more This chapter gives an overview of traditional background modeling and foreground detection, and presents resources, datasets and codes publicly available.
The analysis and understanding of video sequences is currently quite an active research field. Man... more The analysis and understanding of video sequences is currently quite an active research field. Many applications such as video surveillance, optical motion capture or those of multimedia need to first be able to detect the objects moving in a scene filmed by a static camera. This requires the basic operation that consists of separating the moving objects called "foreground" from the static information called "background". Many background subtraction methods have been developed (Bouwmans et al. (2010); Bouwmans et al. (2008)). A recent survey (Bouwmans (2009)) shows that subspace learning models are well suited for background subtraction. Principal Component Analysis (PCA) has been used to model the background by significantly reducing the data’s dimension. To perform PCA, different Robust Principal Components Analysis (RPCA) models have been recently developped in the literature. The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. However, authors compare their algorithm only with the PCA (Oliver et al. (1999)) or another RPCA model. Furthermore, the evaluation is not made with the datasets and the measures currently used in the field of background subtraction. Considering all of this, we propose to evaluate RPCA models in the field of video-surveillance.
""Developing a background subtraction method, researchers must design each step and choose the fe... more ""Developing a background subtraction method, researchers must design each step and choose the features in relation to the critical situations they want to handle. All these critical situations generates imprecision and uncertainties in all the process of background subtraction. Therefore, some authors have recently introduced fuzzy concepts in the different steps of background subtraction as follows:
• Fuzzy Background Modeling: The main challenge consists in modeling multimodal background. The algorithm usually used is the Gaussian Mixture Models. The parameters are determined using a training sequence which contains insufficient or noisy data. So, the parameters are not well determined. In this context, Type-2 Fuzzy Gaussian Mixture Models are used to model uncertainties when dynamic backgrounds occurs.
• Fuzzy Foreground Detection: In this case, a saturing linear function is used to avoid crips decision in the classification of the pixels as background or foreground. The background model can be unimodal such as the running average or multi-modal such as the background modeling with confidence measure. Another approach consists in aggregating different features such as color and texture features by using the Sugeno integral or the Choquet integral. Fuzzy foreground detection is more robust to illumination changes and shadows than crisp foreground detection.
• Fuzzy Background Maintenance: The idea is to update the background following the membership of the pixel at the class background or foreground. This membership comes from the fuzzy foreground detection. This fuzzy adaptive background maintenance allows to deal robustly with illumination changes and shadows.
• Fuzzy Post-Processing: Fuzzy inference can be used between the previous and the current foreground masks to perform the detection of the moving objects as developed recently by Sivabalakrishnan and Manjula.""
Background modeling is often used in the context of moving objects detection from static cameras.... more Background modeling is often used in the context of moving objects detection from static cameras. Numerous methods have been developed over the recent years and the most used are the statistical ones. The purpose of this chapter is to provide a recent survey of these different statistical methods. For this, we have classified them in term of generation following the years of publication and the statistical tools used. We then focus on the first generation methods: Single Gaussian, Mixture of Gaussians, Kernel Density Estimation and Subspace Learning using PCA. These original methods are reminded and then we have classified their different improvements in term of strategies. After analyzing the strategies and identifying their limitations, we conclude with several promising directions for future research.
Papers by Thierry BOUWMANS
Environmental Modelling and Software, 2025
assive Sonic Monitoring (PSM) refers to the analysis of patterns and structures shaped by sound, ... more assive Sonic Monitoring (PSM) refers to the analysis of patterns and structures shaped by sound, offering a complementary approach to traditional landscape analysis methods, such as satellite imagery. In particular, satellite-based methods alone may overlook specific dynamics of the organism at multiple taxonomic levels and local abiotic interactions. This paper introduces a novel unsupervised methodology for mapping similarities between soundscapes. Using Gaussian Mixture Models (GMM), this approach generates soundscape maps that reveal ecological processes throughout the day. We applied our methodology to data from 94 sites within a heterogeneous Colombian Orinoquia ecosystem. We found correlations between the cluster maps, satellite images, and biotic presences (bird and amphibian sonotypes). Our results align with established remote-sensing data and uncover previously unrecognized sonic patterns, offering new ecological insights that complement traditional landscape assessments. Our approach bridges the gap between image satellite-based assessments and ecological sonic processes, paving the way for comprehensive long-term biodiversity monitoring.
Computer Vision and Image Understanding, 2025
Moving object segmentation (MOS) using passive underwater image processing is an important techno... more Moving object segmentation (MOS) using passive underwater image processing is an important technology for monitoring marine habitats. It aids marine biologists studying biological oceanography and the associated fields of chemical, physical, and geological oceanography to understand marine organisms. Dynamic backgrounds due to marine organisms like algae and seaweed, and improper illumination of the environment pose challenges in detecting moving objects in the scene. Previous graph-learning methods have shown promising results in MOS, but are mostly limited to terrestrial surface videos such as traffic video surveillance. Traditional object modeling fails in underwater scenes, due to fish shape and color degradation in motion and the lack of extensive underwater datasets for deep-learning models. Therefore, we propose a semi-supervised graph-learning approach (GraphMOS-U) to segment moving objects in underwater environments. Additionally, existing datasets were consolidated to form the proposed Teleost Fish Classification Dataset, specifically designed for fish classification tasks in complex environments to avoid unseen scenes, ensuring the replication of the transfer learning process on a ResNet-50 backbone. GraphMOS-U uses a six-step approach with transfer learning using Mask R-CNN and a ResNet-50 backbone for instance segmentation, followed by feature extraction using optical flow, visual saliency, and texture. After concatenating these features, a k-NN Graph is constructed, and graph node classification is applied to label objects as foreground or background. The foreground nodes are used to reconstruct the segmentation map of the moving object from the scene. Quantitative and qualitative experiments demonstrate that GraphMOS-U outperforms state-of-the-art algorithms, accurately detecting moving objects while preserving fine details. The proposed method enables the use of graph-based MOS algorithms in underwater scenes.
International Journal of Remote Sensing, 2024
Hyperspectral imagery has a high-dimensional curse due to numerous spectral bands. Band selection... more Hyperspectral imagery has a high-dimensional curse due to numerous spectral bands. Band selection (BS) is crucial for efficiently reducing dimensionality, retaining only essential bands containing valuable information. However, deep learning-based techniques have gained more attention through trained networks for band selection. Recently, graph-based learning has been extensively used in hyperspectral imagery, revealing intrinsic data relationships. This article presents a novel hybrid approach for hyperspectral band selection, addressing the curse of dimensionality in hyperspectral imagery (HSI). Integrating Long Short Term Memory (LSTM) and Graph Transformer (GT), the method employs Bi-dimensional Empirical Mode Decomposition (BEMD) for spatial data enhancement. Using transfer learning, we explore a ResNet-50 deep network to identify optimal intrinsic mode functions (IMFs). The final band subset will be obtained by concatenating the features extracted from the graph transformer and LSTM networks from selected IMFs and residual IMF, respectively. The proposed HybridGT-BS technique surpasses state-of-the-art methods in classification accuracy across three well-known HSI datasets – IP-Indian Pines, SA-Salinas, and PU-PaviaU. With the support of experimental results, the proposed technique significantly outperforms the classification accuracy with the best bands of the HSIs
IEEE Transactions on Signal and Information Processing Over Networks, 2024
Graph Neural Networks (GNNs) have shown great
promise in modeling relationships between nodes in... more Graph Neural Networks (GNNs) have shown great
promise in modeling relationships between nodes in a graph,
but capturing higher-order relationships remains a challenge for
large-scale networks. Previous studies have primarily attempted
to utilize the information from higher-order neighbors in the
graph, involving the incorporation of powers of the shift operator,
such as the graph Laplacian or adjacency matrix. This approach
comes with a trade-off in terms of increased computational and
memory demands. Relying on graph spectral theory, we make a
fundamental observation: the regular and the Hadamard power
of the Laplacian matrix behave similarly in the spectrum. This
observation has significant implications for capturing higher
order information in GNNs for various tasks such as node
classification and semi-supervised learning. Consequently, we
propose a novel graph convolutional operator based on the sparse
Sobolev norm of graph signals. Our approach, known as Sparse
Sobolev GNN (S2-GNN), employs Hadamard products between
matrices to maintain the sparsity level in graph representations.
S2-GNN utilizes a cascade of filters with increasing Hadamard
powers to generate a diverse set of functions. We theoretically
analyze the stability of S2-GNN to show the robustness of the
model against possible graph perturbations. We also conduct
a comprehensive evaluation of S2-GNN across various graph
mining, semi-supervised node classification, and computer vision
tasks. In particular use cases, our algorithm demonstrates com
petitive performance compared to state-of-the-art GNNs in terms
of performance and running time.
Uploads
Chapters by Thierry BOUWMANS
Within maritime scenes, surveillance can include counting the number of ships, as well as detecting and tracking vessels in fluvial channels. Other settings are stores, that is, the detection and monitoring of customers [16–19]. Background subtraction also plays an important role in rapid decision-making in sports activities. For example, Hawk-Eye software has become a key part of soccer and tennis contests. Background subtraction can be used for precise analysis of athletic performance, because it has no physical effect on the athlete as in Tamas et al. [20] for rowing motions, and for surveillance as in Bastos [21] for activities of surfers. In this chapter, we thus attempt to survey the main visual surveillance applications of human activities that use background subtraction in their process by classifying them in terms of aims, environments, and objects of interest. We reviewed only the publications that specifically address the problem of background subtraction in the concerned applications with experiments on corresponding videos. Thus, this chapter refers only to background subtraction methods that are previously and currently used in these applications. To have an overview of all the background subtraction methods in research, the reader can refer to numerous surveys [22–29] in the field.
Considering all of this, this chapter develops a comprehensive review on the different RPCA methods based on decomposition into low-rank and sparse matrices. The rest of this chapter is organized as follows. Firstly, we provide a preliminary overview and a unified view of RPCA via decomposition into low-rank and sparse matrices. Then, we review each original method in its section). For each method, we investigate how it is solved, and if incremental and real-time versions are available for real time applications such as background/foreground separation. Finally, we conclude with promising research directions.
The BGSLibrary also provides one Java based GUI (Graphical User Interface) allowing the users to configure the input video-source, region of interest, and the parameters of each BS algorithm. A MFC-based GUI is also provided for a quick access in Windows computers. But, a QT-based GUI is coming for a platform independent usage. To build/run the BGSLibrary, it is necessary to have the OpenCV library installed previously. Everyone is invited to collaborate with the BGSLibrary. In this chapter some efforts has been made for how to make and add your contributions in the library."
• Fuzzy Background Modeling: The main challenge consists in modeling multimodal background. The algorithm usually used is the Gaussian Mixture Models. The parameters are determined using a training sequence which contains insufficient or noisy data. So, the parameters are not well determined. In this context, Type-2 Fuzzy Gaussian Mixture Models are used to model uncertainties when dynamic backgrounds occurs.
• Fuzzy Foreground Detection: In this case, a saturing linear function is used to avoid crips decision in the classification of the pixels as background or foreground. The background model can be unimodal such as the running average or multi-modal such as the background modeling with confidence measure. Another approach consists in aggregating different features such as color and texture features by using the Sugeno integral or the Choquet integral. Fuzzy foreground detection is more robust to illumination changes and shadows than crisp foreground detection.
• Fuzzy Background Maintenance: The idea is to update the background following the membership of the pixel at the class background or foreground. This membership comes from the fuzzy foreground detection. This fuzzy adaptive background maintenance allows to deal robustly with illumination changes and shadows.
• Fuzzy Post-Processing: Fuzzy inference can be used between the previous and the current foreground masks to perform the detection of the moving objects as developed recently by Sivabalakrishnan and Manjula.""
Papers by Thierry BOUWMANS
promise in modeling relationships between nodes in a graph,
but capturing higher-order relationships remains a challenge for
large-scale networks. Previous studies have primarily attempted
to utilize the information from higher-order neighbors in the
graph, involving the incorporation of powers of the shift operator,
such as the graph Laplacian or adjacency matrix. This approach
comes with a trade-off in terms of increased computational and
memory demands. Relying on graph spectral theory, we make a
fundamental observation: the regular and the Hadamard power
of the Laplacian matrix behave similarly in the spectrum. This
observation has significant implications for capturing higher
order information in GNNs for various tasks such as node
classification and semi-supervised learning. Consequently, we
propose a novel graph convolutional operator based on the sparse
Sobolev norm of graph signals. Our approach, known as Sparse
Sobolev GNN (S2-GNN), employs Hadamard products between
matrices to maintain the sparsity level in graph representations.
S2-GNN utilizes a cascade of filters with increasing Hadamard
powers to generate a diverse set of functions. We theoretically
analyze the stability of S2-GNN to show the robustness of the
model against possible graph perturbations. We also conduct
a comprehensive evaluation of S2-GNN across various graph
mining, semi-supervised node classification, and computer vision
tasks. In particular use cases, our algorithm demonstrates com
petitive performance compared to state-of-the-art GNNs in terms
of performance and running time.
Within maritime scenes, surveillance can include counting the number of ships, as well as detecting and tracking vessels in fluvial channels. Other settings are stores, that is, the detection and monitoring of customers [16–19]. Background subtraction also plays an important role in rapid decision-making in sports activities. For example, Hawk-Eye software has become a key part of soccer and tennis contests. Background subtraction can be used for precise analysis of athletic performance, because it has no physical effect on the athlete as in Tamas et al. [20] for rowing motions, and for surveillance as in Bastos [21] for activities of surfers. In this chapter, we thus attempt to survey the main visual surveillance applications of human activities that use background subtraction in their process by classifying them in terms of aims, environments, and objects of interest. We reviewed only the publications that specifically address the problem of background subtraction in the concerned applications with experiments on corresponding videos. Thus, this chapter refers only to background subtraction methods that are previously and currently used in these applications. To have an overview of all the background subtraction methods in research, the reader can refer to numerous surveys [22–29] in the field.
Considering all of this, this chapter develops a comprehensive review on the different RPCA methods based on decomposition into low-rank and sparse matrices. The rest of this chapter is organized as follows. Firstly, we provide a preliminary overview and a unified view of RPCA via decomposition into low-rank and sparse matrices. Then, we review each original method in its section). For each method, we investigate how it is solved, and if incremental and real-time versions are available for real time applications such as background/foreground separation. Finally, we conclude with promising research directions.
The BGSLibrary also provides one Java based GUI (Graphical User Interface) allowing the users to configure the input video-source, region of interest, and the parameters of each BS algorithm. A MFC-based GUI is also provided for a quick access in Windows computers. But, a QT-based GUI is coming for a platform independent usage. To build/run the BGSLibrary, it is necessary to have the OpenCV library installed previously. Everyone is invited to collaborate with the BGSLibrary. In this chapter some efforts has been made for how to make and add your contributions in the library."
• Fuzzy Background Modeling: The main challenge consists in modeling multimodal background. The algorithm usually used is the Gaussian Mixture Models. The parameters are determined using a training sequence which contains insufficient or noisy data. So, the parameters are not well determined. In this context, Type-2 Fuzzy Gaussian Mixture Models are used to model uncertainties when dynamic backgrounds occurs.
• Fuzzy Foreground Detection: In this case, a saturing linear function is used to avoid crips decision in the classification of the pixels as background or foreground. The background model can be unimodal such as the running average or multi-modal such as the background modeling with confidence measure. Another approach consists in aggregating different features such as color and texture features by using the Sugeno integral or the Choquet integral. Fuzzy foreground detection is more robust to illumination changes and shadows than crisp foreground detection.
• Fuzzy Background Maintenance: The idea is to update the background following the membership of the pixel at the class background or foreground. This membership comes from the fuzzy foreground detection. This fuzzy adaptive background maintenance allows to deal robustly with illumination changes and shadows.
• Fuzzy Post-Processing: Fuzzy inference can be used between the previous and the current foreground masks to perform the detection of the moving objects as developed recently by Sivabalakrishnan and Manjula.""
promise in modeling relationships between nodes in a graph,
but capturing higher-order relationships remains a challenge for
large-scale networks. Previous studies have primarily attempted
to utilize the information from higher-order neighbors in the
graph, involving the incorporation of powers of the shift operator,
such as the graph Laplacian or adjacency matrix. This approach
comes with a trade-off in terms of increased computational and
memory demands. Relying on graph spectral theory, we make a
fundamental observation: the regular and the Hadamard power
of the Laplacian matrix behave similarly in the spectrum. This
observation has significant implications for capturing higher
order information in GNNs for various tasks such as node
classification and semi-supervised learning. Consequently, we
propose a novel graph convolutional operator based on the sparse
Sobolev norm of graph signals. Our approach, known as Sparse
Sobolev GNN (S2-GNN), employs Hadamard products between
matrices to maintain the sparsity level in graph representations.
S2-GNN utilizes a cascade of filters with increasing Hadamard
powers to generate a diverse set of functions. We theoretically
analyze the stability of S2-GNN to show the robustness of the
model against possible graph perturbations. We also conduct
a comprehensive evaluation of S2-GNN across various graph
mining, semi-supervised node classification, and computer vision
tasks. In particular use cases, our algorithm demonstrates com
petitive performance compared to state-of-the-art GNNs in terms
of performance and running time.
costly required labeled samples have limited their practical uses. Furthermore, the performance of these models decreases dramatically due to the scarcity of the labeled data. Thus, it is crucial to uplift the unlabeled inputs and develop semi-supervised learning approaches to enhance the capacities of machine learning models. In this context, this letter proposes an algorithm based on the combination of instance segmentation and Graph Signal Processing (GSP). It includes instance segmentation; Gray- Level Co-occurrence Matrix (GLCM), Local Binary Patterns (LBP) and statistical SAR features to represent the nodes of
the graph; K-nearest neighbors to construct the graph; and Sobolev minimization algorithm to tackle the problem of semisupervised
semantic segmentation. The proposed algorithm is trained and tested using the publicly available SSDD and HRSID ship detection datasets. Experiments show that SemiSegSAR outperforms recent state-of-the-art supervised methods while requiring few labeled data.
signals are inherently time-varying and the smoothness of the temporal differences of such graph signals may be used as a prior assumption. In the current work, we assume that the temporal differences of graph signals are smooth, and we introduce a novel algorithm based on the extension of a Sobolev smoothness function for the reconstruction of time-varying graph signals from discrete samples. We explore some theoretical aspects of the convergence rate of our Time-varying Graph signal Reconstruction via Sobolev Smoothness (GraphTRSS) algorithm
by studying the condition number of the Hessian associated with our optimization problem. Our algorithm has the advantage of converging faster than other methods that are based on Laplacian operators without requiring expensive eigenvalue decomposition or matrix inversions.
information fusion strategy comprising a common low-rank subspace for the fusion of different types of features and tracker responses. Firstly, we interpret the response maps as smoothly varying functions which
can be efficiently represented using individual low-rank matrices, thus removing high frequency noise and sparse artifacts. Secondly, we estimate a common low-rank subspace which is constrained to remain close to each individual low-rank subspace resulting in an efficient fusion strategy. The proposed algorithm achieves good performance by integrating the information contained in heterogeneous feature types.We demonstrate the efficiency of our algorithm using several combinations of features as well as correlation filter and end-to-end deep trackers. The proposed common subspace fusion algorithm is generic and can be used to efficiently fuse the response maps of varying types of feature representations as well as trackers. Extensive experiments on several tracking benchmarks including OTB15, TC128, VOT-ST 2018, VOT-LT 2018, UAV123, GOT-10K and LaSOT have demonstrated significant performance improvements compared to many state-of-the-art tracking methods
the segmented objects. We evaluate our method with multiple datasets and the results confirm the effectiveness of the proposed approach which achieves superior performance over the state of the art methods having the capabilities of segmenting and classifying moving objects from videos surveillance.
covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions.
Thus, challenges are investigated in terms of camera, foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.
large number of studies on convolutional neural networks applied to background subtraction have been published, and a continual gain of performance has been achieved. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. To do so, we first surveyed the background initialization and background subtraction methods based on deep neural networks concepts, and also deep learned features. We then discuss the adequacy of deep neural networks for the task of background subtraction. Finally, experimental results are presented for the CDnet 2014 dataset.
variation, dynamic camera motion, cluttered background and occlusion. This limitation can be attributed to the lack of systematic investigation concerning the role and importance of features within background modeling and foreground detection. With the availability of a rather large set of invariant features, the challenge is in determining the best combination of features that would improve accuracy and robustness in detection. The purpose of this study is to initiate a rigorous and comprehensive survey of features used within background modeling and foreground detection. Further, this paper presents a systematic experimental and statistical analysis of techniques that provide valuable insight on the trends in background modeling and use it to draw
meaningful recommendations for practitioners. In this paper, a preliminary review of the key characteristics of features based on the types and sizes is provided in addition to investigating their intrinsic spectral, spatial and temporal properties. Furthermore,
improvements using statistical and fuzzy tools are examined and techniques based on multiple features are benchmarked against reliability and selection criterion. Finally, a description of the different resources available such as datasets and codes is provided.
The process mainly used is the background subtraction which one key step is the foreground detection. The goal is to classify pixels of the current image as foreground or background.
Some critical situations as shadows, illumination variations can occur in the scene and generate a false classification of image pixels. To deal with the uncertainty in the classification issue, we propose to use the Choquet integral as aggregation operator. Experiments on different data sets in video surveillance have shown a robustness of the proposed method against some critical situations when fusing color and texture features. Different color spaces have been tested to improve the insensitivity of the detection to the illumination changes. Then, the algorithm has been compared with another fuzzy approach based on the Sugeno integral and has proved its robustness
level operations in video analysis. The aim is to separate
static information called "background" from the moving
objects called "foreground". The background needs to be
modeled and updated over time to allow robust foreground
detection. Reconstructive subspace learning such as Prin-
cipal Component Analysis (PCA) has been widely used in
background modeling by significantly reducing the data’s
dimension. However, reconstructive representations strive to
be as informative as possible in terms of well approximating
the original data. On the other hand, discriminative meth-
ods such as Linear Discriminant Analysis (LDA) provides
a supervised reconstruction of the data which will often
give better classification results when compared to the
reconstructive methods. In this paper, we offer to use and
validate the combination of a reconstructive method with
a discriminative one to model robustly the background. The
objective is firstly to enable a robust model of the background
and secondly a robust classification of pixels as background
or foreground. Results on different datasets demonstrate the
performance of the
low-dimensional subspace called low-rank matrix and sparse error con-stitutes the foreground objects. But RPCA presents the limitations of computational complexity and memory storage due to batch optimization methods, as a result it is di.cult to apply for real-time system. To handle these challenges, this work presents a robust foreground detection algorithm via Online Robust PCA (OR-PCA). OR-PCA with good initialization scheme using image decomposition approach improves the accuracy of foreground detection and the computation time as well. Moreover, solving MRF with graph-cuts exploits structural information using spatial neighborhood system and similarities to further improve the foreground segmentation in highly dynamic backgrounds.
too. The experiments conducted on both synthetic and real videos (from the Background Models Challenge) of outdoor urban scenes under various conditions show that the proposed XCS-LBP outperforms its direct competitors for the background subtraction task.
In this context, this handbook solicited contributions to address this wide range of robust low-rank and sparse matrix decompositions for applications in image and video processing. Thus, it groups the works of the leading teams in this field over the recent years. By incorporating both existing and new ideas, this handbook gives a complete overview of the concepts, theories, algorithms, and applications related to robust low-rank and sparse matrix decompositions. First, an introduction to robust principal component analysis via low-rank and sparse matrices decomposition for beginners is provided by surveying the different decompositions, loss functions, optimization problems and solvers. Furthermore, leading methods and algorithms for robust low-rank and sparse matrix decompositions are presented. Moreover, an accompanying website is provided. This website contains the list of chapters, their abstracts and links to some software demonstrations. It allows the reader to have quick access to the main resources and codes in the field. Finally, with this handbook, we aim to bring a one-stop solution, i.e., access to a number of different decompositions, algorithms, implementations and benchmarking techniques in a single volume