Execution of a manipulation after learning from demonstration many times requires intricate plann... more Execution of a manipulation after learning from demonstration many times requires intricate planning and control systems or some form of manual guidance for a robot. Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example.
A novel robust method for surface tracking in range-image sequences is presented which combines a... more A novel robust method for surface tracking in range-image sequences is presented which combines a clustering method based on surface models with a particle-filter-based 2-D affine-motion estimator. Segmented regions obtained at previous time steps are used to create seed areas by comparing measured depth values with those obtained from surface-model fitting. The seed areas are further refined using a motion-probability region estimated by the particle-filter-based tracker through prediction of future states. This helps resolving ambiguities that arise when surfaces belonging to different objects are in physical contact with each other, for example during hand-object manipulations. Region growing allows recovering the complete segment area. The obtained segmented regions are then used to improve the predictions of the tracker for the next frame. The algorithm runs in quasi real-time and uses on-line learning, eliminating the need to have a priori knowledge about the surface being tracked. We apply the method to in-house depth videos acquired with both time-of-flight and structured-light sensors, demonstrating object tracking in real-world scenarios, and we compare the results with those of an ICP-based tracker.
Noise-Induced Signal Enhancement in Heterogeneous Neural Networks
Lecture Notes in Computer Science, 2001
Neural networks can represent complex functions, but are often constructed of very simple units. ... more Neural networks can represent complex functions, but are often constructed of very simple units. We investigate the limitations imposed by such a simple unit, the McCulloch-Pitts neuron. We explore the role of stochastic resonance in units of finite precision and show how to construct neural networks that overcome the limitations of single units.
IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2015
Plant growth is a dynamic process, and the precise course of events during early plant developmen... more Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found.
Consistent depth video segmentation using adaptive surface models
IEEE transactions on cybernetics, 2015
We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using ... more We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using adaptive surface models. Starting from an initial configuration, the algorithm converges to a stable segmentation through a new iterative split-and-merge procedure, which includes an adaptive mechanism for the creation and removal of segments. This allows the segmentation to adjust to changing input data along the movie, leading to stable, temporally coherent, and traceable segments. We tested the method on a large variety of data acquired with different range imaging devices, including a structured-light sensor and a time-of-flight camera, and successfully segmented the videos into surface segments. We further demonstrated the feasibility of the approach using quantitative evaluations based on ground-truth data.
Procedings of the British Machine Vision Conference 2008, 2008
A novel Fourier-based technique for the estimation of optic-flow fields from image sequences is p... more A novel Fourier-based technique for the estimation of optic-flow fields from image sequences is proposed. In this method, the instantaneous velocities of local image points are inferred directly from the global 3D Fourier components of the image sequence. This is done by selecting those velocities for which the superposition of the corresponding Fourier gratings leads to constructive interference at the image point. Hence, uncertainties caused through a windowed measurement process typical for local methods do not arise. The algorithm is tested on both synthetic and real image sequences and the results are compared to those of local techniques for optic-flow computation.
Procedings of the British Machine Vision Conference 2009, 2009
In stereo vision, 3D information is reconstructed from two images of the same scene taken from di... more In stereo vision, 3D information is reconstructed from two images of the same scene taken from different viewpoints. Different approaches to stereo disparity estimation have been extensively compared in several studies . Most stereo algorithms perform well in textured image areas, but often fail when there is only weak texture, due to the correspondence problem. Here local matching fails, and, as a consequence, global methods do not deliver correct disparities either, simply because the energy functions used in global methods remain under-constrained. However, stereo from weakly textured images is important for many applications, which take place in urban or industrial settings, where little texture exists, requiring novel solutions to the stereo problem.
Efficient segmentation of color images is important for many applications in computer vision. Non... more Efficient segmentation of color images is important for many applications in computer vision. Non-parametric solutions are required in situations where little or no prior knowledge about the data is available. In this paper, we present a novel parallel image segmentation algorithm which segments images in real-time in a non-parametric way. The algorithm finds the equilibrium states of a Potts model in the superparamagnetic phase of the system. Our method maps perfectly onto the Graphics Processing Unit (GPU) architecture and has been implemented using the framework NVIDIA Compute Unified Device Architecture (CUDA). For images of 256 × 320 pixels we obtained a frame rate of 30 Hz that demonstrates the applicability of the algorithm to video-processing tasks in real-time 1 .
A novel real-time framework for model-free stereo-video segmentation and stereo-segment tracking ... more A novel real-time framework for model-free stereo-video segmentation and stereo-segment tracking is presented, combining real-time optical flow and stereo with image segmentation running separately on two GPUs. The stereosegment tracking algorithm achieves a frame rate of 23 Hz for regular videos with a frame size of 256 × 320 pixels and nearly real time for stereo videos. The computed stereo segments are used to construct 3D segment graphs, from which main graphs, representing a relevant change in the scene, are extracted, which allow us to represent a movie of e.g. 396 original frames by only 12 graphs, each containing only a small number of nodes, providing a condensed description of the scene while preserving data-intrinsic semantics. Using this method, human activities, e.g., handling of objects, can be encoded in an efficient way. The method has potential applications for manipulation action recognition and learning, and provides a vision-front end for applications in cognitive robotics.
A wave interference model of motion contrast sensitivity
The detection of relative motion cues has been reported for motion sensitive neurons in several v... more The detection of relative motion cues has been reported for motion sensitive neurons in several vertebrate systems including avian optic tectum (Frost and Nakayama, 1983) and primate area MT (Allman et al., 1985). To date, the mechanism behind motion contrast sensitivity remains unknown. Here we propose a model for computing motion contrast, that could be implemented by existing horizontal cell
We propose a novel algorithm for stereo matching using a dynamical systems approach. The stereo c... more We propose a novel algorithm for stereo matching using a dynamical systems approach. The stereo correspondence problem is first formulated as an energy minimization problem. From the energy function, we derive a system of differential equations describing the corresponding dynamical system of interacting elements, which we solve using numerical integration. Optimization is introduced by means of a damping term and a noise term, an idea similar to simulated annealing. The algorithm is tested on the Middlebury stereo benchmark.
Execution of a manipulation after learning from demonstration many times requires intricate plann... more Execution of a manipulation after learning from demonstration many times requires intricate planning and control systems or some form of manual guidance for a robot. Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example.
A novel robust method for surface tracking in range-image sequences is presented which combines a... more A novel robust method for surface tracking in range-image sequences is presented which combines a clustering method based on surface models with a particle-filter-based 2-D affine-motion estimator. Segmented regions obtained at previous time steps are used to create seed areas by comparing measured depth values with those obtained from surface-model fitting. The seed areas are further refined using a motion-probability region estimated by the particle-filter-based tracker through prediction of future states. This helps resolving ambiguities that arise when surfaces belonging to different objects are in physical contact with each other, for example during hand-object manipulations. Region growing allows recovering the complete segment area. The obtained segmented regions are then used to improve the predictions of the tracker for the next frame. The algorithm runs in quasi real-time and uses on-line learning, eliminating the need to have a priori knowledge about the surface being tracked. We apply the method to in-house depth videos acquired with both time-of-flight and structured-light sensors, demonstrating object tracking in real-world scenarios, and we compare the results with those of an ICP-based tracker.
Noise-Induced Signal Enhancement in Heterogeneous Neural Networks
Lecture Notes in Computer Science, 2001
Neural networks can represent complex functions, but are often constructed of very simple units. ... more Neural networks can represent complex functions, but are often constructed of very simple units. We investigate the limitations imposed by such a simple unit, the McCulloch-Pitts neuron. We explore the role of stochastic resonance in units of finite precision and show how to construct neural networks that overcome the limitations of single units.
IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2015
Plant growth is a dynamic process, and the precise course of events during early plant developmen... more Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found.
Consistent depth video segmentation using adaptive surface models
IEEE transactions on cybernetics, 2015
We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using ... more We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using adaptive surface models. Starting from an initial configuration, the algorithm converges to a stable segmentation through a new iterative split-and-merge procedure, which includes an adaptive mechanism for the creation and removal of segments. This allows the segmentation to adjust to changing input data along the movie, leading to stable, temporally coherent, and traceable segments. We tested the method on a large variety of data acquired with different range imaging devices, including a structured-light sensor and a time-of-flight camera, and successfully segmented the videos into surface segments. We further demonstrated the feasibility of the approach using quantitative evaluations based on ground-truth data.
Procedings of the British Machine Vision Conference 2008, 2008
A novel Fourier-based technique for the estimation of optic-flow fields from image sequences is p... more A novel Fourier-based technique for the estimation of optic-flow fields from image sequences is proposed. In this method, the instantaneous velocities of local image points are inferred directly from the global 3D Fourier components of the image sequence. This is done by selecting those velocities for which the superposition of the corresponding Fourier gratings leads to constructive interference at the image point. Hence, uncertainties caused through a windowed measurement process typical for local methods do not arise. The algorithm is tested on both synthetic and real image sequences and the results are compared to those of local techniques for optic-flow computation.
Procedings of the British Machine Vision Conference 2009, 2009
In stereo vision, 3D information is reconstructed from two images of the same scene taken from di... more In stereo vision, 3D information is reconstructed from two images of the same scene taken from different viewpoints. Different approaches to stereo disparity estimation have been extensively compared in several studies . Most stereo algorithms perform well in textured image areas, but often fail when there is only weak texture, due to the correspondence problem. Here local matching fails, and, as a consequence, global methods do not deliver correct disparities either, simply because the energy functions used in global methods remain under-constrained. However, stereo from weakly textured images is important for many applications, which take place in urban or industrial settings, where little texture exists, requiring novel solutions to the stereo problem.
Efficient segmentation of color images is important for many applications in computer vision. Non... more Efficient segmentation of color images is important for many applications in computer vision. Non-parametric solutions are required in situations where little or no prior knowledge about the data is available. In this paper, we present a novel parallel image segmentation algorithm which segments images in real-time in a non-parametric way. The algorithm finds the equilibrium states of a Potts model in the superparamagnetic phase of the system. Our method maps perfectly onto the Graphics Processing Unit (GPU) architecture and has been implemented using the framework NVIDIA Compute Unified Device Architecture (CUDA). For images of 256 × 320 pixels we obtained a frame rate of 30 Hz that demonstrates the applicability of the algorithm to video-processing tasks in real-time 1 .
A novel real-time framework for model-free stereo-video segmentation and stereo-segment tracking ... more A novel real-time framework for model-free stereo-video segmentation and stereo-segment tracking is presented, combining real-time optical flow and stereo with image segmentation running separately on two GPUs. The stereosegment tracking algorithm achieves a frame rate of 23 Hz for regular videos with a frame size of 256 × 320 pixels and nearly real time for stereo videos. The computed stereo segments are used to construct 3D segment graphs, from which main graphs, representing a relevant change in the scene, are extracted, which allow us to represent a movie of e.g. 396 original frames by only 12 graphs, each containing only a small number of nodes, providing a condensed description of the scene while preserving data-intrinsic semantics. Using this method, human activities, e.g., handling of objects, can be encoded in an efficient way. The method has potential applications for manipulation action recognition and learning, and provides a vision-front end for applications in cognitive robotics.
A wave interference model of motion contrast sensitivity
The detection of relative motion cues has been reported for motion sensitive neurons in several v... more The detection of relative motion cues has been reported for motion sensitive neurons in several vertebrate systems including avian optic tectum (Frost and Nakayama, 1983) and primate area MT (Allman et al., 1985). To date, the mechanism behind motion contrast sensitivity remains unknown. Here we propose a model for computing motion contrast, that could be implemented by existing horizontal cell
We propose a novel algorithm for stereo matching using a dynamical systems approach. The stereo c... more We propose a novel algorithm for stereo matching using a dynamical systems approach. The stereo correspondence problem is first formulated as an energy minimization problem. From the energy function, we derive a system of differential equations describing the corresponding dynamical system of interacting elements, which we solve using numerical integration. Optimization is introduced by means of a damping term and a noise term, an idea similar to simulated annealing. The algorithm is tested on the Middlebury stereo benchmark.
Uploads
Papers by Babette Dellen