Object Segmentation with PCL and ROS for Robotic Vision

Remote Sensing

Building information models (BIM) in the civil industry are very popular nowadays. The basic information of these models is the 3D geometric model of a building structure. The most applied methodology to model the existing buildings is by generating 3D geometric information from point clouds provided by laser scanners. The fundamental principle of this methodology is the recognition of structures shaped in basic geometric primitives, e.g., planes, spheres, and cylinders. The basic premise of the efficiency of this methodology is the automation of detection, since manual segmentation of a point cloud can be challenging, time-consuming, and, therefore, inefficient. This paper presents a novel algorithm for the automated segmentation of geometric shapes in point clouds without needing pre-segmentation. With the designed algorithm, structures formed in three types of basic geometrical primitive can be identified and segmented: planar (e.g., walls, floors, ceilings), spherical (e.g., las...

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015

Numerous applications related to urban scene analysis demand automatic recognition of buildings and distinct sub-elements. For example, if LiDAR data is available, only 3D information could be leveraged for the segmentation. However, this poses several risks, for instance, the in-plane objects cannot be distinguished from their surroundings. On the other hand, if only image based segmentation is performed, the geometric features (e.g., normal orientation, planarity) are not readily available. This renders the task of detecting the distinct sub-elements of the building with similar radiometric characteristic infeasible. In this paper the individual sub-elements of buildings are recognized through sub-segmentation of the building using geometric and radiometric characteristics jointly. 3D points generated from Unmanned Aerial Vehicle (UAV) images are used for inferring the geometric characteristics of roofs and facades of the building. However, the image-based 3D points are noisy, error prone and often contain gaps. Hence the segmentation in 3D space is not appropriate. Therefore, we propose to perform segmentation in image space using geometric features from the 3D point cloud along with the radiometric features. The initial detection of buildings in 3D point cloud is followed by the segmentation in image space using the region growing approach by utilizing various radiometric and 3D point cloud features. The developed method was tested using two data sets obtained with UAV images with a ground resolution of around 1-2 cm. The developed method accurately segmented most of the building elements when compared to the plane-based segmentation using 3D point cloud alone.

2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC)

Inspired by the ideas behind superpixels, which segment an image into homogenous regions to accelerate subsequent processing steps (e.g. tracking), we present a sensorfusion-based segmentation approach that generates dense depth regions referred to as supersurfaces. This method aggregates both a point cloud from a LiDAR and an image from a camera to provide an over-segmentation of the three-dimensional scene into piece-wise planar surfaces by utilizing a multi-label Markov Random Field (MRF). A comparison between this method that generates supersurfaces, image-based superpixels, and RGBDbased segments using a subset of KITTI dataset is provided in the experimental results. We observed that supersurfaces are less redundant and more accurate in terms of average boundary recall for a fixed number of segments.

Robotics and Automation Letters, 2019

Mobile robots need to create high-definition 3D maps of the environment for applications such as remote surveillance and infrastructure mapping. Accurate semantic processing of the acquired 3D point cloud is critical for allowing the robot to obtain a high-level understanding of the surrounding objects and perform context-aware decision making. Existing techniques for point cloud semantic segmentation are mostly applied on a single-frame or offline basis, with no way to integrate the segmentation results over time. This paper proposes an online method for mobile robots to incrementally build a semantically-rich 3D point cloud of the environment. The proposed deep neural network, MCPNet, is trained to predict class labels and object instance labels for each point in the scanned point cloud in an incremental fashion. A multi-view context pooling (MCP) operator is used to combine point features obtained from multiple viewpoints to improve the classification accuracy. The proposed architecture was trained and evaluated on ray-traced scans derived from the Stanford 3D Indoor Spaces dataset. Results show that the proposed approach led to 15% improvement in point-wise accuracy and 7% improvement in NMI compared to the next best online method, with only a 6% drop in accuracy compared to the PointNet-based offline approach.

2021 IEEE International Conference on Robotics and Automation (ICRA)

The segmentation of a point cloud into planar primitives is a popular approach to first-line scene interpretation and is particularly useful in mobile robotics for the extraction of drivable or walkable surfaces and for tabletop segmentation for manipulation purposes. Unfortunately, the planar segmentation task becomes particularly challenging when the point clouds are obtained from an inherently noisy, robot-mounted sensor that is often in motion, therefor requiring real time processing capabilities. We present a real time-capable plane segmentation technique based on a region growing algorithm that exploits the organized structure of point clouds obtained from RGB-D sensors. In order to counteract the sensor noise, we invest into careful selection of seeds that start the region growing and avoid the computation of surface normals whenever possible. We implemented our algorithm in C++ and thoroughly tested it in both simulated and real-world environments where we are able to compare our approach against existing state-of-the-art methods implemented in the Point Cloud Library. The experiments presented here suggest that our approach is accurate and fast, even in the presence of considerable sensor noise.

2015

The acquisition of 3D point clouds representing the surface structure of real-world scenes has become common practice in many areas including architecture, cultural heritage and urban planning. Improvements in sample acquisition rates and precision are contributing to an increase in size and quality of point cloud data. The management of these large volumes of data is quickly becoming a challenge, leading to the design of algorithms intended to analyse and decrease the complexity of this data. Point cloud segmentation algorithms partition point clouds for better management, and scene understanding algorithms identify the components of a scene in the presence of considerable clutter and noise. In many cases, segmentation algorithms operate within the remit of a specific context, wherein their effectiveness is measured. Similarly, scene understanding algorithms depend on specific scene properties and fail to identify objects in a number of situations. This work addresses this lack of ...

Human-centric Computing and Information Sciences, 2019

Ground segmentation is an important step for any autonomous and remote-controlled systems. After separating ground and nonground parts, many works such as object tracking and 3D reconstruction can be performed. In this paper, we propose an efficient method for segmenting the ground data of point clouds acquired from multi-channel Lidar sensors. The goal of this study is to completely separate ground points and nonground points in real time. The proposed method segments ground data efficiently and accurately in various environments such as flat terrain, undulating/rugged terrain, and mountainous terrain. First, the point cloud in each obtained frame is divided into small groups. We then focus on the vertical and horizontal directions separately, before processing both directions concurrently. Experiments were conducted, and the results showed the effectiveness of the proposed ground segment method. For flat and sloping terrains, the accuracy is over than 90%. Besides, the quality of ...

ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences

As lidar point clouds become larger streamed processing becomes more attractive. This paper presents a framework for the streamed segmentation of point clouds with the intention of segmenting unstructured point clouds in real-time. The framework is composed of two main components. The first component segments points within a window shifting over the point cloud. The second component stitches the segments within the windows together. In this fashion a point cloud can be streamed through these two components in sequence, thus producing a segmentation. The algorithm has been tested on airborne lidar point cloud and some results of the performance of the framework are presented.

Multimedia Tools and Applications

In the field of SLAM (Simultaneous Localization And Mapping) for robot navigation, mapping the environment is an important task. In this regard the Lidar sensor can produce near accurate 3D map of the environment in the format of point cloud, in real time. Though the data is adequate for extracting information related to SLAM, processing millions of points in the point cloud is computationally quite expensive. The methodology presented proposes a fast algorithm that can be used to extract semantically labelled surface segments from the cloud, in real time, for direct navigational use or higher level contextual scene reconstruction. First, a single scan from a spinning Lidar is used to generate a mesh of subsampled cloud points online. The generated mesh is further used for surface normal computation of those points on the basis of which surface segments are estimated. A novel descriptor to represent the surface segments is proposed and utilized to determine the surface class of the segments (semantic label) with the help of classifier. These semantic surface segments can be further utilized for geometric reconstruction of objects in the scene, or can be used for optimized trajectory planning by a robot. The proposed methodology is compared with number of point cloud segmentation methods and state of the art semantic segmentation methods to emphasize its efficacy in terms of speed and accuracy.

Black Sea Journal of Engineering and Science, 2020

In recent years, point cloud data generated with RGB-D cameras, 3D lasers, and 3D LiDARs have been employed frequently in robotic applications. In indoor environments, RGB-D cameras, which have short-range and can only describe the vicinity of the robots, generally are opted due to their low cost. On the other hand, 3D lasers and LiDARs can capture long-range measurements and generally are used in outdoor applications. In this study, we deal with the segmentation of indoor planar surfaces such as wall, floor, and ceiling via point cloud data. The segmentation methods, which are situated in Point Cloud Library (PCL) were executed with 3D laser point cloud data. The experiments were conducted to evaluate the performance of these methods with the publicly available Fukuoka indoor laser dataset, which has point clouds with different noise levels. The test results were compared in terms of segmentation accuracy and the time elapsed for segmentation. Besides, the general characteristics of each method were discussed. In this way, we revealed the positive and negative aspects of these methods for researchers that plan to apply them to 3D laser point cloud data.

Heritage

The creation of 2D–3D architectural vector drawings constitutes a manual, labor-intensive process. The scientific community has not provided an automated approach for the production of 2D–3D architectural drawings of cultural-heritage objects yet, regardless of the undoubtable need of many scientific fields. This paper presents an automated method which addresses the problem of detecting 3D edges in point clouds by leveraging a set of RGB images and their 2D edge maps. More concretely, once the 2D edge maps have been produced exploiting manual, semi-automated or automated methods, the RGB images are enriched with an extra channel containing the edge semantic information corresponding to each RGB image. The four-channel images are fed into a Structure from Motion–Multi View Stereo (SfM-MVS) software and a semantically enriched dense point cloud is produced. Then, using the semantically enriched dense point cloud, the points belonging to a 3D edge are isolated from all the others base...

IPCV'13 The 2013 International Conference on Image Processing, Computer Vision, and Pattern Recognition, 2013

We present algorithms for fast segmentation and classification of sparse 3D point clouds from rotating LIDAR sensors used for real-time applications such as autonomous mobile systems. Such systems must continuously process large amounts of data with update rates as high as 10 frames per second which makes complexity and performance of the algorithms very critical. Our approach to the segmentation of large and sparse point clouds is efficient and accurate which frees system resources for implementing other more demanding tasks such as classification. Segmentation is the emphasis of this paper as a necessary important first step for subsequent classification and further processing. We propose methods for segmenting sparse point clouds from rotating LIDAR sensors such as the Velodyne HDL-64E using rectangular and radial grids. The main part of the segmentation is performed on small grid images instead of large point clouds which makes the process computationally fast, simple, and very efficient. The proposed algorithms do not require flat horizontal structure of the ground and they demonstrate stable performance in different urban conditions and scenes for detection of different street objects including pedestrians, cars, and bicyclists.

2008

Disclaimer This document describes work undertaken as part of a programme of study at the International Institute for Geo-information Science and Earth Observation. All views and opinions expressed therein remain the sole responsibility of the author, and do not necessarily represent those of the institute. Abstract Segmentation is a method which groups points based on certain similarity. This is required for information extraction from unstructured laser point cloud data. Many studies have been done on segmentation of point cloud data. The algorithms which are designed to extract planar surfaces, most commonly found surface in man made objects, group points exploiting the mathematical representation of the planar surface. This is because point clouds do not have any explicit information about the object except its 3D positional information. Recently, laser scanning systems also provide colour information as Red, Green and Blue (RGB) channels to each point in addition to the 3D coor...

2010 IEEE International Workshop on Robotic and Sensors Environments, 2010

Autonomous robotic exploration in a 3D environment requires the acquisition of 3D data to create a consistent internal model of the environment from which objects can be recognized for the robot to interact with. As the acquisition of 3D data with stereo vision or a laser range finder can be a relatively long process, selective sensing is desired to optimize the amount of data collected to accurately represent the environment in a minimal amount of time. In order to perform selective sensing, a coarse acquisition of the environment first needs to take place. Regions of interest, such as edges and other boundaries, can then be identified so that an acquisition with higher spatial resolution can occur over bounded regions. For that purpose a segmentation method of the coarse data is proposed so that regions can be efficiently distinguished from each other. The method takes a raw 3D surface profile point cloud of varying point densities, organizes it into a mesh, and then segments the surfaces present in this point cloud, producing a segmented mesh, as well as an octree of labeled voxels corresponding to the segmentation. This mesh and octree may then be used for sensory selection to drive a robot exploration task. The method is demonstrated on actual datasets collected in a laboratory environment.

IEEE Transactions on Geoscience and Remote Sensing, 2016

This paper investigates the problems of outliers and/or noise in surface segmentation and proposes a statistically robust segmentation algorithm for laser scanning 3-D point cloud data. Principal component analysis (PCA)-based local saliency features, e.g., normal and curvature, have been frequently used in many ways for point cloud segmentation. However, PCA is sensitive to outliers; saliency features from PCA are nonrobust and inaccurate in the presence of outliers; consequently, segmentation results can be erroneous and unreliable. As a remedy, robust techniques, e.g., RANdom SAmple Consensus (RANSAC), and/or robust versions of PCA (RPCA) have been proposed. However, RANSAC is influenced by the well-known swamping effect, and RPCA methods are computationally intensive for point cloud processing. We propose a region growing based robust segmentation algorithm that uses a recently introduced maximum consistency with minimum distance based robust diagnostic PCA (RDPCA) approach to get robust saliency features. Experiments using synthetic and laser scanning data sets show that the RDPCA-based method has an intrinsic ability to deal with outlier-and/or noisecontaminated data. Results for a synthetic data set show that RDPCA is 105 times faster than RPCA and gives more accurate and robust results when compared with other segmentation methods. Compared with RANSAC and RPCA based methods, RDPCA takes almost the same time as RANSAC, but RANSAC results are markedly worse than RPCA and RDPCA results. Coupled with a segment merging algorithm, the proposed method is efficient for huge volumes of point cloud data consisting of complex objects surfaces from mobile, terrestrial, and aerial laser scanning systems.