A variational approach to multi-modal image matching

Computational Imaging VI, 2008

In the past decade, information theory has been studied extensively in computational imaging. In particular, image matching by maximizing mutual information has been shown to yield good results in multimodal image registration. However, there have been few rigorous studies to date that investigate the statistical aspect of the resulting deformation fields. Different regularization techniques have been proposed, sometimes generating deformations very different from one another. In this paper, we present a novel model for multimodal image registration. The proposed method minimizes a purely information-theoretic functional consisting of mutual information matching and unbiased regularization. The unbiased regularization term measures the magnitude of deformations using either asymmetric Kullback-Leibler divergence or its symmetric version. The new multimodal unbiased matching method, which allows for large topology preserving deformations, was tested using pairs of two and three dimensional serial MRI images. We compared the results obtained using the proposed model to those computed with a well-known mutual information based viscous fluid registration. A thorough statistical analysis demonstrated the advantages of the proposed model over the multimodal fluid registration method when recovering deformation fields and corresponding Jacobian maps.

2000

We propose a novel MRF-based model for image matching. Given two images, the task is to estimate a mapping from one image to another, in order to maximize the matching quality. We consider mappings defined by discrete deformation field constrained to preserve 2-dimensional continuity. We approach the corresponding optimization problem by the TRW-S (sequential Tree-reweighted message passing) algorithm . Our model design allows for a considerably wider class of natural transformation and yields a compact representation of the optimization task. For this model TRW-S algorithm demonstrated nice practical performance in experiments. We also propose a concise derivation of the TRW-S algorithm as a sequential maximization of the lower bound on the energy function.

2007

We present a variational approach for the computation of dense cor- respondence between three rectified images. Using rectification allows reducing the problem to a single parameter functional. Classically, the functional is composed of a data term and a regularization term. We introduce an improved model, where the pixel wise data term is com- bined with a template matching approach, which

Computer Vision and Image Understanding, 2008

We propose a novel MRF-based model for deformable image matching. Given two images, the task is to estimate a mapping from one image to the other maximizing the quality of the match. We consider mappings defined by a discrete deformation field constrained to preserve 2D continuity. We pose the task as finding MAP configurations of a pairwise MRF. We propose a more compact MRF representation of the problem which leads to a weaker, though computationally more tractable, linear programming relaxation -the approximation technique we choose to apply. The number of dual LP variables grows linearly with the search window side, rather than quadratically as in previous approaches. To solve the relaxed problem (suboptimally), we apply TRW-S (Sequential Tree-Reweighted Message passing) algorithm . Using our representation and the chosen optimization scheme, we are able to match much wider deformations than was considered previously in global optimization framework. We further elaborate on continuity and data terms to achieve more appropriate description of smooth deformations. The performance of our technique is demonstrated on both synthetic and real-world experiments.

IET Image Processing, 2017

In this study, a new framework for multimodal image registration is proposed based on the expectation-maximisation (EM) methodology. This framework allows to address simultaneously parametric and elastic registrations independently on the modality of the target and source images without making any assumptions about their intensity relationship. The EM formulation for the image registration problem leads to a regularised quadratic optimisation scheme to compute the displacement vector field (DVF) that aligns the images and depends on their joint intensity distribution. At the first stage, a parametric transformation is assumed for the DVF, where the resulting quadratic optimisation is computed recursively to calculate its optimal parameters. Next, a general unknown deformation models the elastic part of the DVF, which is represented by an additive structure. The resulting optimisation process by the EM formulation results in a cost function that involves data and regularisation terms, which is also solved recursively. A comprehensive evaluation of the parametric and elastic proposals is carried out by comparing to state-of-the-art algorithms and images from different application fields, where an advantage is visualised by the authors' proposal in terms of a compromise between accuracy and robustness.

1997

assumption. In fact, by simply disabling the I component of We describe a novel technique for matching and recognition our deformations we can obtain a standard 2D deformable based on deformable intensity surfaces which incorporates both mesh which yields correspondences similar to an optical the shape (x, y) and the texture (I(x, y)) components of a 2D flow technique with thin-plate regularizers.

Lecture Notes in Computer Science, 2006

A particular problem in image registration arises for multimodal images taken from different imaging devices and/or modalities. Starting in 1995, mutual information has shown to be a very successful distance measure for multi-modal image registration. However, mutual information has also a number of well-known drawbacks. Its main disadvantage is that it is known to be highly non-convex and has typically many local maxima. This observation motivate us to seek a different image similarity measure which is better suited for optimization but as well capable to handle multi-modal images. In this work we investigate an alternative distance measure which is based on normalized gradients and compare its performance to Mutual Information. We call the new distance measure Normalized Gradient Fields (NGF).

In this paper, a new multi-modal non-rigid registration technique for medical images is presented. Firstly, the reg- istration problem is outlined and some of the most com- mon approaches reported, then, the proposed algorithm is presented. The proposed technique is based on mutual information maximization and computes a deformation field through a suitable globally smoothed affine piecewise transformation. The algorithm has been conceived with particu- lar attention to computational load and accuracy of results. Experimental results involving intra-patient, inter-patients and atlas images on brain CT and MR (T1, T2 and PD modalities) are reported.

Procedings of the British Machine Vision Conference 2003, 2003

We introduce a compact coding of image information which explicitly separates visual information into geometric information (orientation) and structural information (phase and colour) and temporal information (optic flow). We investigate the importance of these visual attributes for stereo matching on a large data set. From these investigation we can conclude that it is the combination of different attributes that gives the best results. Concrete weights for the relative importance of different visual attributes are statistically determined.

Image registration is central to many challenges in medical imaging and therefore it has a vast range of applications. The purpose of this note is to provide a unified but extremely flexible framework for image registration. This framework is based on a variational formulation of the registration problem. We discuss the framework as well as some of its most important building blocks. These include some of the most promising non-linear registration strategies used in today medical imaging. The overall goal of image registration is to compute a deformation, such that a deformed version of an image becomes similar to a so-called reference image. Hence, the similarity measure is an important building block. Depending on the application at hand, it is inevitable to constrain the wanted deformation in an appropriate way. Thus, regularization is also a main building block. Finally, it is often desirable to incorporate higher level information about the expected deformation. We show how such constraints or information can easily be integrated in our general framework and discuss some examples. Moreover, the proposed general framework allows for a unified algorithmic treatment of the various building blocks.

HAL (Le Centre pour la Communication Scientifique Directe), 2022

In this work, we use the geometric information, such as edges and thin structures, to build a similarity measure for deformable registration models of multi-modality images. The idea is to extract a geometric information from the images and then use it to build a robust and efficient similarity term. In order to extract this information, we use the Blake-Zisserman's energy that is well suited for detecting discontinuities at different scales, i.e. of first and second order. In addition, we present a theoretical analysis of the proposed model. For the numerical solution of the model, we use a gradient descent method and iteratively solve corresponding the Euler-Lagrangian equation. We present some numerical results that demonstrate the efficiency of the proposed model.

Proceedings Icip International Conference on Image Processing, 2010

This paper presents the combined use of gradient and mutual information for infrared and intensity templates matching. We propose to joint: (i) feature matching in a multiresolution context and (ii) information propagation through scale-space representations. Our method consists in combining mutual information with a shape descriptor based on gradient, and propagate them following a coarseto-fine strategy. The main contributions of this work are: to offer a theoretical formulation towards a multimodal stereo matching; to show that gradient and mutual information can be reinforced while they are propagated between consecutive levels; and to show that they are valid cost functions in multimodal template matchings. Comparisons are presented showing the improvements and viability of the proposed approach.

Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention, 2009

In this paper we present a novel method for performing image registration of different modalities. Mutual Information (MI) is an established method for performing such registration. However, it is recognised that standard MI is not without some problems, in particular it does not utilise spatial information within the images. Various modifications have been proposed to resolve this, however these only offer slight improvement to the accuracy of registration. We present Feature Neighbourhood Mutual Information (FNMI) that combines both image structure and spatial neighbourhood information which is efficiently incorporated into Mutual Information by approximating the joint distribution with a covariance matrix (c.f. Russakoff's Regional Mutual Information). Results show that our approach offers a very high level of accuracy that improves greatly on previous methods. In comparison to Regional MI, our method also improves runtime for more demanding registration problems where a high...

In this paper, we present a novel methodology for multi-modal non-rigid image registration. The proposed approach is formulated by using the Expectation-Maximization (EM) technique in order to estimate a displacement vector field that aligns the images to register. In this approach, the image alignment relies on hidden stochastic random variables which allow to compare the intensity values between images of different modality. The methodology is basically composed of two steps: first, we provide an initial estimation of the the global deformation vector field by using a rigid registration technique based on particle filtering, obtaining, at the same time, an initial estimation of the joint conditional intensity distribution of the registered images; second, we approximate the remaining deformations by applying an iterative EM-technique approach , where at each step, a new estimation of the joint conditional intensity distribution and the displacement vector field are computed. The proposed algorithm was tested with different kinds of medical images ; preliminary results show that the methodology is a good alternative for non-rigid multimodal registration.

1999

We present a new approach for the computation of the deformation field between three dimensional (3D) images. The deformation field minimizes the sum of the squared differences between the images to be matched and is constrained by the physical properties of the different objects represented by the image. The objects are modeled as elastic bodies. Compared to optical flow methods, this approach distinguishes itself by three main characteristics: it can account for the actual physical properties of the objects to be deformed, it can provide us with physical properties of the deformed objects (i.e. stress tensors), and computes a global solution to the deformation instead of a set of local solutions. This latter characteristic is achieved through a finite-element based scheme. The finite element approach requires the different objects in the images to be meshed. Therefore, a tetrahedral mesh generator using a pre-computed case table and specifically suited for segmented images has been developed. Preliminary experiments on simulated data as well as on medical data have been carried out successfully. Tested medical applications included muscle exercise imaging and ventricular deformation in multiple sclerosis.