Papers by Matteo Dellepiane
In this paper we present how technologies developed in the context of 3D graphics (3D scanning an... more In this paper we present how technologies developed in the context of 3D graphics (3D scanning and image-to-3D-model mapping) have been used to evaluate in a metrically objective way the plausibility of an attribution hypothesis of a Renaissance artifact. The artifact considered is a small bronze horse (Archeological Museum, Florence, Italy), which was discovered to be very similar to a silverpoint drawing by Leonardo (Leonardo drawing #358, Windsor Royal Library, UK), thus originating an attribution hypothesis. A highly accurate digital replica of this small bronze statue was reconstructed by means of triangulation-based 3D scanning technology. Using image-to-3D-model mapping techniques the Leonardo's drawing depicting a very similar horse has been aligned to the digital 3D model, reconstructing in a virtual manner the ideal vantage points which could have been used to sketch the drawings of the horse (in the hypothesis that Leonardo had drawn it by directly looking at the bronze). The same approach has been also tried with other Leonardo's drawings depicting a similar subject, but none of those was sufficiently compatible in shape with the bronze horse to allow convergence of the image-to-3D mapping process. The approach proposed allows us to give some visually objective evidences about the shape similarity issue, which was the origin of this attribution attempt. At the same time, a purely technical evaluation does not close the attribution issue. Further archival research and expertise will be needed to solve and assess the issue of this disputed hypothesis.
eHeritage 2010: 2nd ACM workshop on eHeritage and digital art preservation
Recent years have witnessed a great increase in the interest in digitally capturing and represent... more Recent years have witnessed a great increase in the interest in digitally capturing and representing world heritage sites for preservation and access. The “UNESCO World Heritage Programme” now has 851 listed sites, all considered as sites of “outstanding cultural or natural importance to the common heritage of humanity.” Of these, 660 sites are cultural sites, 166 are natural and 25 have mixed properties. But the concept of Heritage is much vaster than the tangible places around the Earth: there is the need to preserve not ...
Real-time sound rendering engines often render occlusion and early sound reflection effects using... more Real-time sound rendering engines often render occlusion and early sound reflection effects using geometrical techniques such as ray or beam tracing. They can only achieve interactive rendering for environments of low local complexity resulting in crude effects which can degrade the sense of immersion. However, surface detail or complex dynamic geometry has a strong influence on sound propagation and the resulting auditory perception. This paper focuses on high-quality modeling of first-order sound scattering. Based on a surface-integral formulation and the Kirchhoff approximation, we propose an efficient evaluation of scattering effects, including both diffraction and reflection, that leverages programmable graphics hardware for dense sampling of complex surfaces. We evaluate possible surface simplification techniques and show that combined normal and displacement maps can be successfully used for audio scattering calculations. We present an auralization framework that can render scattering effects interactively thus providing a more compelling experience. We demonstrate that, while only considering first order phenomena, our approach can provide realistic results for a number of practical interactive applications. It can also process highly detailed models containing millions of unorganized triangles in minutes, generating high-quality scattering filters. Resulting simulations compare well with on-site recordings showing that the Kirchhoff approximation can be used for complex scattering problems.
VAST11: The 12th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage - Short Papers
Improving 2D-3D Registration by Mutual Information using Gradient Maps
Abstract: In this paper we propose an extension for the algorithms of image-to-geometry registrat... more Abstract: In this paper we propose an extension for the algorithms of image-to-geometry registration by Mutual Information (MI) to improve the performance and the quality of the alignment. Proposed for the registration of multi modal medical images, in the last years MI has been adapted to align a 3D model to a given image by using different renderings of the model and a gray-scale version of the input image. A key aspect is the choice of the rendering process to correlate the 3D model to the image without taking into account the ...
Artifacts Removal for Color Projection on 3D Models using Flash Light
Abstract Lighting artifacts are one of the main issues in digital photography: complex light setu... more Abstract Lighting artifacts are one of the main issues in digital photography: complex light setups are needed to attenuate or remove them. Flash light is a very easy way to illuminate an object or an environment, but it is rarely considered in most of the Computer Graphics and Computer Vision applications. This is due to the big amount of artifacts introduced by this lighting, and to the difficulty in modeling its behavior. In this paper we present a simple method to use flash light in the context of color acquisition and mapping on 3D models. ...
Multiscale acquisition and presentation of very large artifacts: The case of portalada
Journal on Computing and Cultural Heritage, 2011
Abstract The dichotomy between full detail representation and the efficient management of data di... more Abstract The dichotomy between full detail representation and the efficient management of data digitization is still a big issue in the context of the acquisition and visualization of 3D objects, especially in the field of the cultural heritage. Modern scanning devices enable very detailed geometry to be acquired, but it is usually quite hard to apply these technologies to large artifacts. In this article we present a project aimed at virtually reconstructing the impressive (7× 11 m.) portal of the Ripoll Monastery, Spain. The monument was acquired ...
Reflection Transformation Imaging has proved to be a powerful method to acquire and represent the... more Reflection Transformation Imaging has proved to be a powerful method to acquire and represent the 3D reflectance properties of an object, displaying them as a 2D image. Recently, Polynomial Texture Maps (PTM), which are relightable images created from a set of photos of the object taken under several different lighting conditions, have been used in Cultural Heritage field to document and virtually inspect several sets of small objects, such as cuneiform tablets and coins. In this paper we explore the possibility of producing high quality PTM of medium or large size objects. The aim is to analyze the acquisition pipeline, resolving all the issues related to the size of the object, and the conditions of acquisition. We will discuss issues regarding acquisition planning and data gathering. We also present a new tool to interactively browse high resolution PTMs. Moreover, we perform some quality assessment considerations, in order to study the degradation of quality of the PTMs respect to the number and position of lights used to acquire the PTM. The results of our acquisition system are presented with some examples of PTMs of large artifacts like a sarcophagus of 2.4 × 1 m size. PTM can be a good alternative to 3D scanning for capturing and representing certain class of objects, like bas-relieves, having lower costs in terms of acquisition equipment and data processing time.
Image-to-Geometry Registration: a Mutual Information Method exploiting Illumination-related Geometric Properties
Computer Graphics Forum, 2009
This work concerns a novel study in the field of image-to-geometry registration. Our approach tak... more This work concerns a novel study in the field of image-to-geometry registration. Our approach takes inspiration from medical imaging, in particular from multi-modal image registration. Most of the algorithms developed in this domain, where the images to register come from different sensors (CT, X-ray, PET), are based on Mutual Information, a statistical measure of non-linear correlation between two data sources. The main idea is to use mutual information as a similarity measure between the image to be registered and renderings of the model geometry, in order to drive the registration in an iterative optimization framework. We demonstrate that some illumination-related geometric properties, such as surface normals, ambient occlusion and reflection directions can be used for this purpose. After a comprehensive analysis of such properties we propose a way to combine these sources of information in order to improve the performance of our automatic registration algorithm. The proposed approach can robustly cover a wide range of real cases and can be easily extended.
Image registration is an important task in several applications of Computer Graphics and Computer... more Image registration is an important task in several applications of Computer Graphics and Computer Vision. Among the large number of proposed approaches, currently there is no solution which is automatic and robust enough to handle any general case. The most robust methods usually require a significant intervention by the user to specify many 2D-3D correspondences, while automatic techniques often rely on strong assumptions about the quality of 2D and 3D data. In this paper we present Mutual Correspondences, which is based on a minimization function which combines correspondences based and Mutual Information based approaches, and takes advantage of the strong points of both. Mutual Correspondences give the user the possibility to "guide" Mutual Information with only a few 2D-3D correspondences. The proposed approach results in a wider convergence range and in higher registration accuracy, regardless of the quality of both the image and the 3D model. Mutual Correspondences were applied on some practical cases, where state-of-the-art approaches tended to fail, and they provided a mean to obtain accurate results. This led to a simple, robust and practical approach that can provide a way to register images in a few seconds.
Multiple Uses of 3D Scanning for the Valorization of an Artistic Site: the Case of Luni
Abstract: The heritage site of Luni is a very interesting location, which tells the story of an a... more Abstract: The heritage site of Luni is a very interesting location, which tells the story of an ancient roman colony, and of an area where different cultures left their heritage. In particular, the remains of the pediments of an ancient temple represent a very interesting case and an open field of study for art historians. The scanning of a group of statue belonging to the pediments showed that the produced 3D models could be used in several ways, not only for archival and presentation purposes but also to provide interactive support for the work of ...
Removing Shadows for Color Projection Using Sun Position Estimation
Abstract: The result of an outdoors 3D scanning acquisition campaign is usually an accurate 3D mo... more Abstract: The result of an outdoors 3D scanning acquisition campaign is usually an accurate 3D model of the site, but in most of the cases the quality of the color acquired by the scanner is not satisfying. Alternative solutions, like the projection of a photographic dataset acquired in a different stage, are still dependent on the quality of the initial images. The short time for the acquisition campaigns and the weather conditions often force the shooting of images taken under a strong direct sun illumination. This generates colored models of poor quality ...
IEEE Transactions on Visualization and Computer Graphics, 2012
The projection of a photographic data set on a 3D model is a robust and widely applicable way to ... more The projection of a photographic data set on a 3D model is a robust and widely applicable way to acquire appearance information of an object. The first step of this procedure is the alignment of the images on the 3D model. While any reconstruction pipeline aims at avoiding misregistration by improving camera calibrations and geometry, in practice a perfect alignment cannot always be reached. Depending on the way multiple camera images are fused on the object surface, remaining misregistrations show up either as ghosting or as discontinuities at transitions from one camera view to another. In this paper we propose a method, based on the computation of Optical Flow between overlapping images, to correct the local misalignment by determining the necessary displacement. The goal is to correct the symptoms of misregistration, instead of searching for a globally consistent mapping, which might not exist. The method scales up well with the size of the data set (both photographic and geometric) and is quite independent of the characteristics of the 3D model (topology cleanliness, parametrization, density). The method is robust and can handle real world cases that have different characteristics: low level geometric details and images that lack enough features for global optimization or manual methods. It can be applied to different mapping strategies, such as texture or per-vertex attribute encoding.
Mapping Highly Detailed Colour Information on Extremely Dense 3D Models: The Case of David's Restoration
Computer Graphics Forum, 2008
The support of advanced information technology (IT) to preservation, restoration and documentatio... more The support of advanced information technology (IT) to preservation, restoration and documentation of Cultural Heritage (CH) is becoming a very important goal for the research community. Michelangelo's David was one of the first applications of 3D scanning technology on a highly popular work of art. The subsequent restoration campaign, started in 2002 and concluded in 2004, was also a milestone for the adoption of modern scientific analysis procedures and IT tools in the framework of a restoration process. One of the focuses in this restoration was also methodological, i.e. to plan and adopt innovative ways to document the restoration process. In this paper, we present the results of an integration of different restoration data (2D and 3D datasets) which has been concluded recently. The recent evolution of HW and SW graphics technologies gave us the possibility to interactively visualize an extremely dense 3D model which incorporates the colour information provided by two professional photographic campaigns, made before and after the restoration. Moreover, we present the results concerning the mapping, in this case on the 2D media, of the reliefs produced by restorers to assess and document the status of the marble surface before the restoration took place. This result could lead to new and fascinating applications of computer graphics for preservation, restoration and documentation of CH.
Computer Graphics Forum, 2008
Visual fidelity and interactivity are the main goals in Computer Graphics research, but recently ... more Visual fidelity and interactivity are the main goals in Computer Graphics research, but recently also audio is assuming an important role. Binaural rendering can provide extremely pleasing and realistic three-dimensional sound, but to achieve best results it's necessary either to measure or to estimate individual Head Related Transfer Function (HRTF). This function is strictly related to the peculiar features of ears and face of the listener. Recent sound scattering simulation techniques can calculate HRTF starting from an accurate 3D model of a human head. Hence, the use of binaural rendering on large scale (i.e. video games, entertainment) could depend on the possibility to produce a sufficiently accurate 3D model of a human head, starting from the smallest possible input. In this paper we present a completely automatic system, which produces a 3D model of a head starting from simple input data (five photos and some key-points indicated by user). The geometry is generated by extracting information from images and accordingly deforming a 3D dummy to reproduce user head features. The system proves to be fast, automatic, robust and reliable: geometric validation and preliminary assessments show that it can be accurate enough for HRTF calculation.
We present a statistical method for the estimation of the Spatially Varying Bidirectional Reflect... more We present a statistical method for the estimation of the Spatially Varying Bidirectional Reflectance Distribution Function (SVBRDF) of an object with complex geometry, starting from video sequences acquired with fixed but general lighting conditions. The aim of this work is to define a method that simplifies the acquisition phase of the object surface appearance and allows to reconstruct an approximated SVBRDF. The final output is suitable to be used with a 3D model of the object to obtain accurate and photo-realistic renderings. The method is composed by three steps: the approximation of the environment map of the acquisition scene, using the same object as a probe; the estimation of the diffuse color of the object; the estimation of the specular components of the main materials of the object, by using a Phong model. All the steps are based on statistical analysis of the color samples projected by the video sequences on the surface of the object. Although the method presents some limitations, the trade-off between the easiness of acquisition and the obtained results makes it useful for practical applications.
Using 3D Scanning to Analyze a Proposal for the Attribution of a Bronze Horse to Leonardo da Vinci
Abstract: In this paper we present how technologies developed in the context of 3D graphics (3D s... more Abstract: In this paper we present how technologies developed in the context of 3D graphics (3D scanning and image-to-3D-model mapping) have been used to evaluate in a metrically objective way the plausibility of an attribution hypothesis of a Renaissance artifact. The artifact considered is a small bronze horse (Archeological Museum, Florence, Italy), which was discovered to be very similar to a silverpoint drawing by Leonardo (Leonardo drawing# 358, Windsor Royal Library, UK), thus originating an attribution hypothesis. A highly ...
The paper describes a pipeline for 3D scanning acquisition and processing that allow to exploit t... more The paper describes a pipeline for 3D scanning acquisition and processing that allow to exploit the utmost precision and quality out of ToF scanners. The proposed approach capitalize on the knowledge of the distribution of the noise to apply sophisticated fairing techniques for cleaning up the data. Leveraging on the very dense sampling of this kind of scanners we show that is possible to attain high accuracy. We present a practical application of the proposed approach for the scanning of a large (5mt) statue with millimetric precision.
IEEE Computer, 2011
Digital technologies are transforming the way cultural heritage researchers, archaeologists, and ... more Digital technologies are transforming the way cultural heritage researchers, archaeologists, and curators work by providing new ways to collaborate, record excavations, and restore artifacts.
Uploads
Papers by Matteo Dellepiane