Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
A framework for photo-realistic view-dependent image synthesis of a shiny object from a sparse se... more A framework for photo-realistic view-dependent image synthesis of a shiny object from a sparse set of images and a geometric model is proposed. Each image is aligned with the 3D model and decomposed into two images with regards to the reflectance components based on the intensity variation of object surface points. The view-independent surface reflection (diffuse reflection) is stored as one texture map. The view-dependent reflection (specular reflection) images are used to recover the initial approximation of the illumination distribution, and then a two step numerical minimization algorithm utilizing a simplified Torrance-Sparrow reflection model is used to estimate the reflectance parameters and refine the illumination distribution. This provides a very compact representation of the data necessary to render synthetic images from arbitrary viewpoints. We have conducted experiments with real objects to synthesize photorealistic view-dependent images within the proposed framework.
We introduce a new surface representation for recognizing curved objects. Our approach begins by ... more We introduce a new surface representation for recognizing curved objects. Our approach begins by representing an object by a discrete mesh of points built from range data or from a geometric model of the object The mesh is computed from the data by deforming a standard shaped mesh, for example, an ellipsoid, until it fits the surface of the object. We define local regularity constraints that the mesh must satisfy. We then define a canonical mapping between the mesh describing the object and a standard spherical mesh. A surface curvature index that is pose-invariant is stored at every node of the mesh. We use this object representation for recognition by comparing the spherical model of a reference object with the model extracted from a new observed scene. We show how the similarity between reference model and observed data can be evaluated and we show how the pose of the reference object in the observed scene can be easily computed using this representation. We present results on re...
2009 IEEE 12th International Conference on Computer Vision, 2009
This paper presents a simple yet practical 3-D modeling method for recovering surface shape and r... more This paper presents a simple yet practical 3-D modeling method for recovering surface shape and reflectance from a set of images. We attach a point light source to a hand-held camera to add a photometric constraint to the multi-view stereo problem. Using the photometric constraint, we simultaneously solve for shape, surface normal, and reflectance. Unlike prior approaches, we formulate the problem using realistic assumptions of a near light source, non-Lambertian surfaces, perspective camera model, and the presence of ambient lighting. The effectiveness of the proposed method is verified using simulated and real-world scenes.
In this paper, we propose a vision-based technique for recognition of a mechanical linkage. The a... more In this paper, we propose a vision-based technique for recognition of a mechanical linkage. The aim is to realize a robot system which can autonomously operate an object with a mechanical linkage. First, using stereo vision, the system observes the operated mechanical linkage. It recognizes the position and posture of its parts over a sequence of time frames using a geometric model-based approach. Next, the system estimates the position and direction of the rotation axis from the relative trajectory of these parts. Robustness of the vision system with regard to occlusion is needed because there is a great deal of overlap between the parts of the linkage, depending on its operating state. Moreover, the rotation axis has to be estimated from a sequence of positions and postures in the presence of noise. We present the following two techniques to solve these problems: 1)Occlusion robust object tracking based on prediction. 2)Parameter estimation of the mechanical rotation linkage from the noisy relative trajectories of its parts. Good experimental results were achieved by adapting these methods to some objects pairs with a linkage mechanism.
Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999
This paper describes a new method for estimating the illumination distribution of a real scene fr... more This paper describes a new method for estimating the illumination distribution of a real scene from a radiance distribution inside shadows cast by an object in the scene. First, the illumination distribution of the scene is approximated by discrete sampling of an extended light source. Then the illumination distribution of the scene is estimated from a radiance distribution inside shadows cast by an object of known shape onto another object in the scene. Instead of assuming any particular reflectance properties of the surface inside the shadows, both the illumination distribution of the scene and the reflectance properties of the surface are estimated simultaneously, based on iterative optimization framework. In addition, this paper introduces an adaptive sampling of the illumination distribution of a scene. Rather than using a uniform discretization of the overall illumination distribution, we adaptively increase sampling directions of the illumination distribution based on the estimation at the previous iteration. Using the adaptive sampling framework, we are able to estimate overall illumination more efficiently by using fewer sampling directions. The proposed method is effective for estimating an illumination distribution even under a complex illumination environment.
2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012
The projections of world parallel lines in an image intersect at a single point called the vanish... more The projections of world parallel lines in an image intersect at a single point called the vanishing point (VP). VPs are a key ingredient for various vision tasks including rotation estimation and 3D reconstruction. Urban environments generally exhibit some dominant orthogonal VPs. Given a set of lines extracted from a calibrated image, this paper aims to (1) determine the line clustering, i.e. find which line belongs to which VP, and (2) estimate the associated orthogonal VPs. None of the existing methods is fully satisfactory because of the inherent difficulties of the problem, such as the local minima and the chicken-and-egg aspect. In this paper, we present a new algorithm that solves the problem in a mathematically guaranteed globally optimal manner and can inherently enforce the VP orthogonality. Specifically, we formulate the task as a consensus set maximization problem over the rotation search space, and further solve it efficiently by a branch-and-bound procedure based on the Interval Analysis theory. Our algorithm has been validated successfully on sets of challenging real images as well as synthetic data sets.
IPSJ Transactions on Computer Vision and Applications, 2009
This paper presents a fast registration method based on solving an energy minimization problem de... more This paper presents a fast registration method based on solving an energy minimization problem derived by implicit polynomials (IPs). Once a target object is encoded by an IP, it will be driven fast towards a corresponding source object along the IP's gradient flow without using point-wise correspondences. This registration process is accelerated by a new IP transformation method. Instead of the time-consuming transformation to a large discrete data set, the new method can transform the polynomial coefficients to maintain the same Euclidean transformation. Its computational efficiency enables us to improve a new application for real-time Ultrasound (US) pose estimation. The reported experimental results demonstrate the capabilities of our method in overcoming the limitations of a noisy, unconstrained, and freehand US image, resulting in fast and robust registration.
2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008
Real-time pose estimation of a free-hand Ultrasound (US) image without any position sensor is des... more Real-time pose estimation of a free-hand Ultrasound (US) image without any position sensor is desirable for diagnostics and image guidance, but it suffers from poor image quality as a result of processing. The paper confronts this problem by proposing a new 6-DOF pose estimation method based on a fast registration process making use of 3D implicit polynomials (IP). The proposed registration method has some main advantages over the traditional methods. First, our formulation is based on minimization of energy functional derived from IP gradient flow, and thus it is more efficient than traditional registration because it eliminates the cost for calculating point-wise correspondences. Second, the efficient calculation benefits from the property of IP having few coefficients, which means that both the gradient field and its transformation can be calculated in an extremely light manner. Third, applying a realtime US image pose estimation, we demonstrate the capabilities of overcoming the limitations of unconstrained freehand US data, resulting in robust and fast pose estimation.
2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008
We propose a method for estimating camera response functions using a probabilistic intensity simi... more We propose a method for estimating camera response functions using a probabilistic intensity similarity measure. The similarity measure represents the likelihood of two intensity observations corresponding to the same scene radiance in the presence of noise. We show that the response function and the intensity similarity measure are strongly related. Our method requires several input images of a static scene taken from the same viewing position with fixed camera parameters. Noise causes pixel values at the same pixel coordinate to vary in these images, even though they measure the same scene radiance. We use these fluctuations to estimate the response function by maximizing the intensity similarity function for all pixels. Unlike prior noise-based estimation methods, our method requires only a small number of images, so it works with digital cameras as well as video cameras. Moreover, our method does not rely on any special image processing or statistical prior models. Real-world experiments using different cameras demonstrate the effectiveness of the technique.
Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205), 2001
In this paper, we propose an image-based approach to synthesize a novel view image for mixed real... more In this paper, we propose an image-based approach to synthesize a novel view image for mixed reality (MR) systems. Theoretically, the image-based method is good for synthesizing realistic images, but it is difficult to achieve interactive handling of the object. As a solution, we propose a new method based on the "surface light field rendering" technique. With this method, we can synthesize the objects with arbitrary deformation and illumination changes. To demonstrate the efficiency of this method, we describe successful experiments that we performed using objects with non-rigid effects (e.g. velvet and Tatami carpet) which are difficult to render correctly by the use of general modelbased rendering techniques.
2013 IEEE International Conference on Robotics and Automation, 2013
The goal of this work is to teach a robot to regrasp an object using knowledge obtained from huma... more The goal of this work is to teach a robot to regrasp an object using knowledge obtained from human demonstration. This paper presents a task model that represents a human regrasping movement. The task model is based on the topological information and comprised of four task primitives. Human regrasping movement is recognised and represented as a sequence of these task primitives by the proposed recognition algorithm. The proposed method then maps each task primitive to the target robot hand using knowledge obtained from human demonstration. The experimental result verified the proposed task model by executing the regrasping movement on the real robot hand.
This paper presents a new perspective for 3D scene understanding by reasoning object stability an... more This paper presents a new perspective for 3D scene understanding by reasoning object stability and safety using intuitive mechanics. Our approach utilizes a simple observation that, by human design, objects in static scenes should be stable in the gravity field and be safe with respect to various physical disturbances such as human activities. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Given a 3D point cloud captured for a static scene by depth cameras, our method consists of three steps: i) recovering solid 3D volumetric primitives from voxels; ii) reasoning stability by grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior; and iii) reasoning safety by evaluating the physical risks for objects under physical disturbances, such as human activity, wind or earthquakes. We adopt a novel intuitive physics model and represent the energy landscape of each primitive and object in the scene by a disconnectivity graph (DG). We construct a contact graph with nodes being 3D volumetric primitives and edges representing the supporting relations. Then we adopt a Swendson-Wang Cuts Algorithm to group/partition the contact graph into groups. Each group is a stable object. In order to detect unsafe objects in a static scene, our method infers hidden and situated causes (disturbances) of the scene, and then introduces intuitive physical mechanics to pre
Proceedings Ninth IEEE International Conference on Computer Vision, 2003
In this paper, we propose a novel method to recover the surface shape of transparent objects. The... more In this paper, we propose a novel method to recover the surface shape of transparent objects. The degree of polarization of the light reflected from the object surface depends on the reflection angle which, in turn, depends on the object's surface normal; thus, by measuring the degree of polarization, we are able to calculate the surface normal of the object. However, degree of polarization and surface normal does not correspond one-to-one, making us to analyze two polarization images taken from two different view in order to solve the ambiguity. A parabolic curve will be a strong clue to correspond a point in one image to a point in the other image, where both points represent the same point on object surface. By comparing the degree of polarization at such corresponding points, the true surface normal can be determined.
International Conference on Robotics and Automation, 1995
The measurement of shape is a basic object inspection task. We use a noncontact method to determi... more The measurement of shape is a basic object inspection task. We use a noncontact method to determine shape called photometric stereo. The method uses three light sources which sequentially illuminate the object under inspection and a video camera for taking intensity images of the object. A significant problem with using photometric stereo is determining where to place the 3 light
In the real world, the color appearances of objects are generally not consistent. It depends prin... more In the real world, the color appearances of objects are generally not consistent. It depends principally on two factors: illumination spectral power distribution (illumination color) and intrinsic surface properties. Consequently, to obtain objects' consistent color descriptors, we have to deal with those two factors. The former is commonly referred to as color constancy: a capability to estimate and discount the illumination color, while the latter is identical to the problem of recovering body color from highlights. This recovery is crucial because highlights emitted from opaque inhomogeneous objects can cause the surface colors to be inconsistent with regard to the change of viewing and illuminant directions. We base our color constancy methods on analyzing highlights or specularities emitted from opaque inhomogeneous objects. We have successfully derived a linear correlation between image chromaticity and illumination chromaticity. This linear correlation is clearly described in inverse-intensity chromaticity space, a novel two-dimensional space we introduce. Through this space, we become able to effectively estimate illumination chromaticity (illumination color) from both uniformly colored surfaces and highly textured surfaces in a single integrated framework, thereby making our method significantly advanced over the existing methods. Meanwhile, for separating reflection components, we propose an approach that is based on an iterative framework and a specularfree image. The specular-free image is an image that is free from specularities yet has different body color from the input image. In general, the approach relies principally on image intensity and color. All methods of color constancy and reflection-components separation proposed in this paper are analyzed based on physical phenomena of the real world, making the estimation more accurate, and have strong basics of analysis. In addition, all methods require only a single input image. This is not only practical, but also challenging in term of complexity.
Hyperspectral imaging is a promising tool for applications in geosensing, cultural heritage and b... more Hyperspectral imaging is a promising tool for applications in geosensing, cultural heritage and beyond. However, compared to current RGB cameras, existing hyperspectral cameras are severely limited in spatial resolution. In this paper, we introduce a simple new technique for reconstructing a very high-resolution hyperspectral image from two readily obtained measurements: A lower-resolution hyperspectral image and a high-resolution RGB image. Our approach is divided into two stages: We first apply an unmixing algorithm to the hyperspectral input, to estimate a basis representing reflectance spectra. We then use this representation in conjunction with the RGB input to produce the desired result. Our approach to unmixing is motivated by the spatial sparsity of the hyperspectral input, and casts the unmixing problem as the search for a factorization of the input into a basis and a set of maximally sparse coefficients. Experiments show that this simple approach performs reasonably well on both simulations and real data examples.
To separate a color signal into its components: illumination spectral power distribution and surf... more To separate a color signal into its components: illumination spectral power distribution and surface spectral reflectance, is an important issue on computer vision. And to separate, we use spectral power distribution, we proposed a minimization technique that, unlike the existing methods, uses multiple color signals. In our implementation, we introduce three different approaches: first, color signals obtained from two different surface reflectance lit by an identical illumination spectral power distribution; second, color signal from an identical surface reflectance lit by different illumination spectral power distributions; and third, color signals from identical surface reflectance but with different types of reflection components (diffuse and specular reflectance) lit by identical illumination spectral power distribution. Using multiple color signals can improve the robustness of the estimation, since we can obtain more constraints in the input data.
This paper discusses issues and techniques to automatically compile object and sensor models into... more This paper discusses issues and techniques to automatically compile object and sensor models into a visual recognition strategy for recognizing and locating an object in three-dimensional space from visual data. Historically, and even today, most successful model-based vision programs are handwritten; relevant knowledge of objects for recognition is extracted from examples of the object, tailored for the particular environment, and coded into the program by the implementors. If this is done properly, the resulting program is effective and efficient, but it requires long development time and many vision experts. Automatic generation of recognition programs by compilation attempts to automate this process. In particular, it extracts from the object and sensor models those features that are useful for recognition, and the control sequence which must be applied to deal with possible variations of the object appearances. The key components in automatic generation are: object modeling, sensor modeling, prediction of appearances, strategy generation, and program generation. An object model describes geometric and photometric properties of an object to be recognized. A sensor model specifies the sensor characteristics in predicting oblect appearances and variations of feature values. The appearances can be systematically grouped into aspects, where aspects are topologically equivalent classes with respect to the object features "visible" to the sensor. Once aspects are obtained, a recognition strategy is generated in the form of an interpretation tree from the aspects and their predicted feature values. An interpretation tree consists of two parts; a part which classifies an unknown region into one of the aspects, and a part which determines its precise attitude (position and orientation) within the classified aspects. Finally, the strategy is converted into an executable program by using object-oriented programming. One major emphasis of this paper is the sensors, as well as objects, must be explicitly modeled in order to achieve the goal of automatic generation of reliable and efficient recognition programs. Actual creation of interpretation trees for two objects and their execution for recognition from a bin ofparts are demonstrated.
Techniques for modeling an object through observation are very important in object recognition an... more Techniques for modeling an object through observation are very important in object recognition and virtual reality. A wide variety of techniques have been developed for modeling objects with opaque surfaces, whereas less attention has been paid to objects with transparent surfaces. A transparent surface has only surface reflection; it has little body reflection. We present a new method for obtaining surface orientations of transparent surfaces through analysis of the degree of polarization in surface reflection and emission in visible and far-infrared wavelengths, respectively. This parameter, the polarization degree of reflected light at the visible wavelengths, is used for determining the surface orientation at a surface point. The polarization degree at visible wavelengths provides two possible solutions, and the proposed method uses the polarization degree at far-infrared wavelengths to resolve this ambiguity.
Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
A framework for photo-realistic view-dependent image synthesis of a shiny object from a sparse se... more A framework for photo-realistic view-dependent image synthesis of a shiny object from a sparse set of images and a geometric model is proposed. Each image is aligned with the 3D model and decomposed into two images with regards to the reflectance components based on the intensity variation of object surface points. The view-independent surface reflection (diffuse reflection) is stored as one texture map. The view-dependent reflection (specular reflection) images are used to recover the initial approximation of the illumination distribution, and then a two step numerical minimization algorithm utilizing a simplified Torrance-Sparrow reflection model is used to estimate the reflectance parameters and refine the illumination distribution. This provides a very compact representation of the data necessary to render synthetic images from arbitrary viewpoints. We have conducted experiments with real objects to synthesize photorealistic view-dependent images within the proposed framework.
We introduce a new surface representation for recognizing curved objects. Our approach begins by ... more We introduce a new surface representation for recognizing curved objects. Our approach begins by representing an object by a discrete mesh of points built from range data or from a geometric model of the object The mesh is computed from the data by deforming a standard shaped mesh, for example, an ellipsoid, until it fits the surface of the object. We define local regularity constraints that the mesh must satisfy. We then define a canonical mapping between the mesh describing the object and a standard spherical mesh. A surface curvature index that is pose-invariant is stored at every node of the mesh. We use this object representation for recognition by comparing the spherical model of a reference object with the model extracted from a new observed scene. We show how the similarity between reference model and observed data can be evaluated and we show how the pose of the reference object in the observed scene can be easily computed using this representation. We present results on re...
2009 IEEE 12th International Conference on Computer Vision, 2009
This paper presents a simple yet practical 3-D modeling method for recovering surface shape and r... more This paper presents a simple yet practical 3-D modeling method for recovering surface shape and reflectance from a set of images. We attach a point light source to a hand-held camera to add a photometric constraint to the multi-view stereo problem. Using the photometric constraint, we simultaneously solve for shape, surface normal, and reflectance. Unlike prior approaches, we formulate the problem using realistic assumptions of a near light source, non-Lambertian surfaces, perspective camera model, and the presence of ambient lighting. The effectiveness of the proposed method is verified using simulated and real-world scenes.
In this paper, we propose a vision-based technique for recognition of a mechanical linkage. The a... more In this paper, we propose a vision-based technique for recognition of a mechanical linkage. The aim is to realize a robot system which can autonomously operate an object with a mechanical linkage. First, using stereo vision, the system observes the operated mechanical linkage. It recognizes the position and posture of its parts over a sequence of time frames using a geometric model-based approach. Next, the system estimates the position and direction of the rotation axis from the relative trajectory of these parts. Robustness of the vision system with regard to occlusion is needed because there is a great deal of overlap between the parts of the linkage, depending on its operating state. Moreover, the rotation axis has to be estimated from a sequence of positions and postures in the presence of noise. We present the following two techniques to solve these problems: 1)Occlusion robust object tracking based on prediction. 2)Parameter estimation of the mechanical rotation linkage from the noisy relative trajectories of its parts. Good experimental results were achieved by adapting these methods to some objects pairs with a linkage mechanism.
Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999
This paper describes a new method for estimating the illumination distribution of a real scene fr... more This paper describes a new method for estimating the illumination distribution of a real scene from a radiance distribution inside shadows cast by an object in the scene. First, the illumination distribution of the scene is approximated by discrete sampling of an extended light source. Then the illumination distribution of the scene is estimated from a radiance distribution inside shadows cast by an object of known shape onto another object in the scene. Instead of assuming any particular reflectance properties of the surface inside the shadows, both the illumination distribution of the scene and the reflectance properties of the surface are estimated simultaneously, based on iterative optimization framework. In addition, this paper introduces an adaptive sampling of the illumination distribution of a scene. Rather than using a uniform discretization of the overall illumination distribution, we adaptively increase sampling directions of the illumination distribution based on the estimation at the previous iteration. Using the adaptive sampling framework, we are able to estimate overall illumination more efficiently by using fewer sampling directions. The proposed method is effective for estimating an illumination distribution even under a complex illumination environment.
2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012
The projections of world parallel lines in an image intersect at a single point called the vanish... more The projections of world parallel lines in an image intersect at a single point called the vanishing point (VP). VPs are a key ingredient for various vision tasks including rotation estimation and 3D reconstruction. Urban environments generally exhibit some dominant orthogonal VPs. Given a set of lines extracted from a calibrated image, this paper aims to (1) determine the line clustering, i.e. find which line belongs to which VP, and (2) estimate the associated orthogonal VPs. None of the existing methods is fully satisfactory because of the inherent difficulties of the problem, such as the local minima and the chicken-and-egg aspect. In this paper, we present a new algorithm that solves the problem in a mathematically guaranteed globally optimal manner and can inherently enforce the VP orthogonality. Specifically, we formulate the task as a consensus set maximization problem over the rotation search space, and further solve it efficiently by a branch-and-bound procedure based on the Interval Analysis theory. Our algorithm has been validated successfully on sets of challenging real images as well as synthetic data sets.
IPSJ Transactions on Computer Vision and Applications, 2009
This paper presents a fast registration method based on solving an energy minimization problem de... more This paper presents a fast registration method based on solving an energy minimization problem derived by implicit polynomials (IPs). Once a target object is encoded by an IP, it will be driven fast towards a corresponding source object along the IP's gradient flow without using point-wise correspondences. This registration process is accelerated by a new IP transformation method. Instead of the time-consuming transformation to a large discrete data set, the new method can transform the polynomial coefficients to maintain the same Euclidean transformation. Its computational efficiency enables us to improve a new application for real-time Ultrasound (US) pose estimation. The reported experimental results demonstrate the capabilities of our method in overcoming the limitations of a noisy, unconstrained, and freehand US image, resulting in fast and robust registration.
2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008
Real-time pose estimation of a free-hand Ultrasound (US) image without any position sensor is des... more Real-time pose estimation of a free-hand Ultrasound (US) image without any position sensor is desirable for diagnostics and image guidance, but it suffers from poor image quality as a result of processing. The paper confronts this problem by proposing a new 6-DOF pose estimation method based on a fast registration process making use of 3D implicit polynomials (IP). The proposed registration method has some main advantages over the traditional methods. First, our formulation is based on minimization of energy functional derived from IP gradient flow, and thus it is more efficient than traditional registration because it eliminates the cost for calculating point-wise correspondences. Second, the efficient calculation benefits from the property of IP having few coefficients, which means that both the gradient field and its transformation can be calculated in an extremely light manner. Third, applying a realtime US image pose estimation, we demonstrate the capabilities of overcoming the limitations of unconstrained freehand US data, resulting in robust and fast pose estimation.
2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008
We propose a method for estimating camera response functions using a probabilistic intensity simi... more We propose a method for estimating camera response functions using a probabilistic intensity similarity measure. The similarity measure represents the likelihood of two intensity observations corresponding to the same scene radiance in the presence of noise. We show that the response function and the intensity similarity measure are strongly related. Our method requires several input images of a static scene taken from the same viewing position with fixed camera parameters. Noise causes pixel values at the same pixel coordinate to vary in these images, even though they measure the same scene radiance. We use these fluctuations to estimate the response function by maximizing the intensity similarity function for all pixels. Unlike prior noise-based estimation methods, our method requires only a small number of images, so it works with digital cameras as well as video cameras. Moreover, our method does not rely on any special image processing or statistical prior models. Real-world experiments using different cameras demonstrate the effectiveness of the technique.
Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205), 2001
In this paper, we propose an image-based approach to synthesize a novel view image for mixed real... more In this paper, we propose an image-based approach to synthesize a novel view image for mixed reality (MR) systems. Theoretically, the image-based method is good for synthesizing realistic images, but it is difficult to achieve interactive handling of the object. As a solution, we propose a new method based on the "surface light field rendering" technique. With this method, we can synthesize the objects with arbitrary deformation and illumination changes. To demonstrate the efficiency of this method, we describe successful experiments that we performed using objects with non-rigid effects (e.g. velvet and Tatami carpet) which are difficult to render correctly by the use of general modelbased rendering techniques.
2013 IEEE International Conference on Robotics and Automation, 2013
The goal of this work is to teach a robot to regrasp an object using knowledge obtained from huma... more The goal of this work is to teach a robot to regrasp an object using knowledge obtained from human demonstration. This paper presents a task model that represents a human regrasping movement. The task model is based on the topological information and comprised of four task primitives. Human regrasping movement is recognised and represented as a sequence of these task primitives by the proposed recognition algorithm. The proposed method then maps each task primitive to the target robot hand using knowledge obtained from human demonstration. The experimental result verified the proposed task model by executing the regrasping movement on the real robot hand.
This paper presents a new perspective for 3D scene understanding by reasoning object stability an... more This paper presents a new perspective for 3D scene understanding by reasoning object stability and safety using intuitive mechanics. Our approach utilizes a simple observation that, by human design, objects in static scenes should be stable in the gravity field and be safe with respect to various physical disturbances such as human activities. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Given a 3D point cloud captured for a static scene by depth cameras, our method consists of three steps: i) recovering solid 3D volumetric primitives from voxels; ii) reasoning stability by grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior; and iii) reasoning safety by evaluating the physical risks for objects under physical disturbances, such as human activity, wind or earthquakes. We adopt a novel intuitive physics model and represent the energy landscape of each primitive and object in the scene by a disconnectivity graph (DG). We construct a contact graph with nodes being 3D volumetric primitives and edges representing the supporting relations. Then we adopt a Swendson-Wang Cuts Algorithm to group/partition the contact graph into groups. Each group is a stable object. In order to detect unsafe objects in a static scene, our method infers hidden and situated causes (disturbances) of the scene, and then introduces intuitive physical mechanics to pre
Proceedings Ninth IEEE International Conference on Computer Vision, 2003
In this paper, we propose a novel method to recover the surface shape of transparent objects. The... more In this paper, we propose a novel method to recover the surface shape of transparent objects. The degree of polarization of the light reflected from the object surface depends on the reflection angle which, in turn, depends on the object's surface normal; thus, by measuring the degree of polarization, we are able to calculate the surface normal of the object. However, degree of polarization and surface normal does not correspond one-to-one, making us to analyze two polarization images taken from two different view in order to solve the ambiguity. A parabolic curve will be a strong clue to correspond a point in one image to a point in the other image, where both points represent the same point on object surface. By comparing the degree of polarization at such corresponding points, the true surface normal can be determined.
International Conference on Robotics and Automation, 1995
The measurement of shape is a basic object inspection task. We use a noncontact method to determi... more The measurement of shape is a basic object inspection task. We use a noncontact method to determine shape called photometric stereo. The method uses three light sources which sequentially illuminate the object under inspection and a video camera for taking intensity images of the object. A significant problem with using photometric stereo is determining where to place the 3 light
In the real world, the color appearances of objects are generally not consistent. It depends prin... more In the real world, the color appearances of objects are generally not consistent. It depends principally on two factors: illumination spectral power distribution (illumination color) and intrinsic surface properties. Consequently, to obtain objects' consistent color descriptors, we have to deal with those two factors. The former is commonly referred to as color constancy: a capability to estimate and discount the illumination color, while the latter is identical to the problem of recovering body color from highlights. This recovery is crucial because highlights emitted from opaque inhomogeneous objects can cause the surface colors to be inconsistent with regard to the change of viewing and illuminant directions. We base our color constancy methods on analyzing highlights or specularities emitted from opaque inhomogeneous objects. We have successfully derived a linear correlation between image chromaticity and illumination chromaticity. This linear correlation is clearly described in inverse-intensity chromaticity space, a novel two-dimensional space we introduce. Through this space, we become able to effectively estimate illumination chromaticity (illumination color) from both uniformly colored surfaces and highly textured surfaces in a single integrated framework, thereby making our method significantly advanced over the existing methods. Meanwhile, for separating reflection components, we propose an approach that is based on an iterative framework and a specularfree image. The specular-free image is an image that is free from specularities yet has different body color from the input image. In general, the approach relies principally on image intensity and color. All methods of color constancy and reflection-components separation proposed in this paper are analyzed based on physical phenomena of the real world, making the estimation more accurate, and have strong basics of analysis. In addition, all methods require only a single input image. This is not only practical, but also challenging in term of complexity.
Hyperspectral imaging is a promising tool for applications in geosensing, cultural heritage and b... more Hyperspectral imaging is a promising tool for applications in geosensing, cultural heritage and beyond. However, compared to current RGB cameras, existing hyperspectral cameras are severely limited in spatial resolution. In this paper, we introduce a simple new technique for reconstructing a very high-resolution hyperspectral image from two readily obtained measurements: A lower-resolution hyperspectral image and a high-resolution RGB image. Our approach is divided into two stages: We first apply an unmixing algorithm to the hyperspectral input, to estimate a basis representing reflectance spectra. We then use this representation in conjunction with the RGB input to produce the desired result. Our approach to unmixing is motivated by the spatial sparsity of the hyperspectral input, and casts the unmixing problem as the search for a factorization of the input into a basis and a set of maximally sparse coefficients. Experiments show that this simple approach performs reasonably well on both simulations and real data examples.
To separate a color signal into its components: illumination spectral power distribution and surf... more To separate a color signal into its components: illumination spectral power distribution and surface spectral reflectance, is an important issue on computer vision. And to separate, we use spectral power distribution, we proposed a minimization technique that, unlike the existing methods, uses multiple color signals. In our implementation, we introduce three different approaches: first, color signals obtained from two different surface reflectance lit by an identical illumination spectral power distribution; second, color signal from an identical surface reflectance lit by different illumination spectral power distributions; and third, color signals from identical surface reflectance but with different types of reflection components (diffuse and specular reflectance) lit by identical illumination spectral power distribution. Using multiple color signals can improve the robustness of the estimation, since we can obtain more constraints in the input data.
This paper discusses issues and techniques to automatically compile object and sensor models into... more This paper discusses issues and techniques to automatically compile object and sensor models into a visual recognition strategy for recognizing and locating an object in three-dimensional space from visual data. Historically, and even today, most successful model-based vision programs are handwritten; relevant knowledge of objects for recognition is extracted from examples of the object, tailored for the particular environment, and coded into the program by the implementors. If this is done properly, the resulting program is effective and efficient, but it requires long development time and many vision experts. Automatic generation of recognition programs by compilation attempts to automate this process. In particular, it extracts from the object and sensor models those features that are useful for recognition, and the control sequence which must be applied to deal with possible variations of the object appearances. The key components in automatic generation are: object modeling, sensor modeling, prediction of appearances, strategy generation, and program generation. An object model describes geometric and photometric properties of an object to be recognized. A sensor model specifies the sensor characteristics in predicting oblect appearances and variations of feature values. The appearances can be systematically grouped into aspects, where aspects are topologically equivalent classes with respect to the object features "visible" to the sensor. Once aspects are obtained, a recognition strategy is generated in the form of an interpretation tree from the aspects and their predicted feature values. An interpretation tree consists of two parts; a part which classifies an unknown region into one of the aspects, and a part which determines its precise attitude (position and orientation) within the classified aspects. Finally, the strategy is converted into an executable program by using object-oriented programming. One major emphasis of this paper is the sensors, as well as objects, must be explicitly modeled in order to achieve the goal of automatic generation of reliable and efficient recognition programs. Actual creation of interpretation trees for two objects and their execution for recognition from a bin ofparts are demonstrated.
Techniques for modeling an object through observation are very important in object recognition an... more Techniques for modeling an object through observation are very important in object recognition and virtual reality. A wide variety of techniques have been developed for modeling objects with opaque surfaces, whereas less attention has been paid to objects with transparent surfaces. A transparent surface has only surface reflection; it has little body reflection. We present a new method for obtaining surface orientations of transparent surfaces through analysis of the degree of polarization in surface reflection and emission in visible and far-infrared wavelengths, respectively. This parameter, the polarization degree of reflected light at the visible wavelengths, is used for determining the surface orientation at a surface point. The polarization degree at visible wavelengths provides two possible solutions, and the proposed method uses the polarization degree at far-infrared wavelengths to resolve this ambiguity.
Uploads
Papers by K. Ikeuchi