Partially linear models on Riemannian manifolds
Daniela Rodriguez2010
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
In partially linear models the dependence of the response y on (x^T,t) is modeled through the relationship y=\x^T \beta+g(t)+\epsilon where \epsilon is independent of (x^T,t). In this paper, estimators of \beta and g are constructed when the explanatory variables t take values on a Riemannian manifold. Our proposal combine the flexibility of these models with the complex structure of a set of explanatory variables. We prove that the resulting estimator of \beta is asymptotically normal under the suitable conditions. Through a simulation study, we explored the performance of the estimators. Finally, we applied the studied model to an example based on real dataset.
Related papers
Robust Estimators in Partly Linear Regression Models on Riemannian Manifolds
2010
Under a partially linear models we study a family of robust estimates for the regression parameter and the regression function when some of the predictor variables take values on a Riemannian manifold. We obtain the consistency and the asymptotic normality of the proposed estimators. Also, we consider a robust cross validation procedure to select the smoothing parameter. Simulations and application to real data show the performance of our proposal under small samples and contamination.
Generalized partially linear models on Riemannian manifolds
The Journal of the Royal Statistical Society, Series C (Applied Statistics), 2020
The generalized partially linear models on Riemannian manifolds are introduced. These models, like ordinary generalized linear models, are a generalization of partially linear models on Riemannian manifolds that allow for response variables with error distribution models other than a normal distribution. Partially linear models are particularly useful when some of the covariates of the model are elements of a Riemannian man-ifold, because the curvature of these spaces makes it difficult to define parametric models. The model was developed to address an interesting application, the prediction of children's garment fit based on 3D scanning of their body. For this reason, we focus on logistic and ordinal models and on the important and difficult case where the Riemannian manifold is the three-dimensional case of Kendall's shape space. An experimental study with a well-known 3D database is carried out to check the goodness of the procedure. Finally it is applied to a 3D database obtained from an anthropometric survey of the Spanish child population. A comparative study with related techniques is carried out.
Multivariate General Linear Models (MGLM) on Riemannian Manifolds with Applications to Statistical Analysis of Diffusion Weighted Images
Proceedings / CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014
Linear regression is a parametric model which is ubiquitous in scientific analysis. The classical setup where the observations and responses, i.e., (xi , yi ) pairs, are Euclidean is well studied. The setting where yi is manifold valued is a topic of much interest, motivated by applications in shape analysis, topic modeling, and medical imaging. Recent work gives strategies for max-margin classifiers, principal components analysis, and dictionary learning on certain types of manifolds. For parametric regression specifically, results within the last year provide mechanisms to regress one real-valued parameter, xi ∈ R, against a manifold-valued variable, yi ∈ . We seek to substantially extend the operating range of such methods by deriving schemes for multivariate multiple linear regression -a manifold-valued dependent variable against multiple independent variables, i.e., f : R (n) → . Our variational algorithm efficiently solves for multiple geodesic bases on the manifold concurrent...
Robust nonparametric regression on Riemannian manifolds
Journal of Nonparametric Statistics, 2009
Nonlinear Regression on Riemannian Manifolds and Its Applications to Neuro-Image Analysis
Lecture Notes in Computer Science, 2015
Regression in its most common form where independent and dependent variables are in ℝ n is a ubiquitous tool in Sciences and Engineering. Recent advances in Medical Imaging has lead to a wide spread availability of manifold-valued data leading to problems where the independent variables are manifold-valued and dependent are real-valued or vice-versa. The most common method of regression on a manifold is the geodesic regression, which is the counterpart of linear regression in Euclidean space. Often, the relation between the variables is highly complex, and existing most commonly used geodesic regression can prove to be inaccurate. Thus, it is necessary to resort to a non-linear model for regression. In this work we present a novel Kernel based nonlinear regression method when the mapping to be estimated is either from M → ℝ n or ℝ n → M, where M is a Riemannian manifold. A key advantage of this approach is that there is no requirement for the manifold-valued data to necessarily inherit an ordering from the data in ℝ n . We present several synthetic and real data experiments along with comparisons to the state-of-theart geodesic regression method in literature and thus validating the effectiveness of the proposed algorithm.
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach
Journal of Machine Learning Research, 2011
The paper addresses the problem of learning a regression model parameterized by a fixed-rank positive semidefinite matrix. The focus is on the nonlinear nature of the search space and on scalability to high-dimensional problems. The mathematical developments rely on the theory of gradient descent algorithms adapted to the Riemannian geometry that underlies the set of fixed-rank positive semidefinite matrices. In contrast with previous contributions in the literature, no restrictions are imposed on the range space of the learned matrix. The resulting algorithms maintain a linear complexity in the problem size and enjoy important invariance properties. We apply the proposed algorithms to the problem of learning a distance function parameterized by a positive semidefinite matrix. Good performance is observed on classical benchmarks.
Local polynomial regression on unknown manifolds
Complex Datasets and Inverse Problems
We reveal the phenomenon that "naive" multivariate local polynomial regression can adapt to local smooth lower dimensional structure in the sense that it achieves the optimal convergence rate for nonparametric estimation of regression functions belonging to a Sobolev space when the predictor variables live on or close to a lower dimensional manifold.
Tangent space estimation for smooth embeddings of Riemannian manifolds
Information and Inference, 2013
Numerous dimensionality reduction problems in data analysis involve the recovery of lowdimensional models or the learning of manifolds underlying sets of data. Many manifold learning methods require the estimation of the tangent space of the manifold at a point from locally available data samples. Local sampling conditions such as (i) the size of the neighborhood (sampling width) and (ii) the number of samples in the neighborhood (sampling density) affect the performance of learning algorithms. In this work, we propose a theoretical analysis of local sampling conditions for the estimation of the tangent space at a point P lying on a m-dimensional Riemannian manifold S in R n. Assuming a smooth embedding of S in R n , we estimate the tangent space T P S by performing a Principal Component Analysis (PCA) on points sampled from the neighborhood of P on S. Our analysis explicitly takes into account the second order properties of the manifold at P , namely the principal curvatures as well as the higher order terms. We consider a random sampling framework and leverage recent results from random matrix theory to derive conditions on the sampling width and the local sampling density for an accurate estimation of tangent subspaces. We measure the estimation accuracy by the angle between the estimated tangent space T P S and the true tangent space T P S and we give conditions for this angle to be bounded with high probability. In particular, we observe that the local sampling conditions are highly dependent on the correlation between the components in the second-order local approximation of the manifold. We finally provide numerical simulations to validate our theoretical findings.
Non-Parametric Estimation of Manifolds from Noisy Data
ArXiv, 2021
A common observation in data-driven applications is that high dimensional data has a low intrinsic dimension, at least locally. In this work, we consider the problem of estimating a $d$ dimensional sub-manifold of $\mathbb{R}^D$ from a finite set of noisy samples. Assuming that the data was sampled uniformly from a tubular neighborhood of $\mathcal{M}\in \mathcal{C}^k$, a compact manifold without boundary, we present an algorithm that takes a point $r$ from the tubular neighborhood and outputs $\hat p_n\in \mathbb{R}^D$, and $\widehat{T_{\hat p_n}\mathcal{M}}$ an element in the Grassmanian $Gr(d, D)$. We prove that as the number of samples $n\to\infty$ the point $\hat p_n$ converges to $p\in \mathcal{M}$ and $\widehat{T_{\hat p_n}\mathcal{M}}$ converges to $T_p\mathcal{M}$ (the tangent space at that point) with high probability. Furthermore, we show that the estimation yields asymptotic rates of convergence of $n^{-\frac{k}{2k + d}}$ for the point estimation and $n^{-\frac{k-1}{2k +...
Related papers
Partial Least Squares Regression on Riemannian Manifolds and Its Application in Classifications
ArXiv, 2016
Partial least squares regression (PLSR) has been a popular technique to explore the linear relationship between two datasets. However, most of algorithm implementations of PLSR may only achieve a suboptimal solution through an optimization on the Euclidean space. In this paper, we propose several novel PLSR models on Riemannian manifolds and develop optimization algorithms based on Riemannian geometry of manifolds. This algorithm can calculate all the factors of PLSR globally to avoid suboptimal solutions. In a number of experiments, we have demonstrated the benefits of applying the proposed model and algorithm to a variety of learning tasks in pattern recognition and object classification.
Intrinsic regression models for manifold-valued data
2009
Abstract. In medical imaging analysis and computer vision, there is a growing interest in analyzing various manifold-valued data including 3D rotations, planar shapes, oriented or directed directions, the Grassmann manifold, deformation field, symmetric positive definite (SPD) matrices and medial shape representations (m-rep) of subcortical structures.
Localized regression on principal manifolds
We consider nonparametric dimension reduction techniques for multivariate regression problems in which the variables constituting the predictor space are strongly nonlinearly related. Specifically, the predictor space is approximated via "local" principal manifolds, based on which a kernel regression is carried out.
Bayesian manifold regression
The Annals of Statistics, 2016
There is increasing interest in the problem of nonparametric regression with high-dimensional predictors. When the number of predictors D is large, one encounters a daunting problem in attempting to estimate a D-dimensional surface based on limited data. Fortunately, in many applications, the support of the data is concentrated on a d-dimensional subspace with d ≪ D. Manifold learning attempts to estimate this subspace. Our focus is on developing computationally tractable and theoretically supported Bayesian nonparametric regression methods in this context. When the subspace corresponds to a locally-Euclidean Riemannian manifold, we show that a Gaussian process regression approach can be applied that leads to the minimax optimal adaptive rate in estimating the regression function under some conditions. The proposed model bypasses the need to estimate the manifold, and can be implemented using standard algorithms for posterior computation in Gaussian processes. Finite sample performance is illustrated in an example data analysis.
A bayesian approach to the estimation of maps between riemannian manifolds, II: examples
2009
Let M be a smooth compact oriented manifold without boundary, imbedded in a euclidean space E and let f be a smooth map of M into a Riemannian manifold N. An unknown state x in M is observed via X=x+su where s>0 is a small parameter and u is a white Gaussian noise. For a given smooth prior on M and smooth estimators g of the map f we have derived a second-order asymptotic expansion for the related Bayesian risk (see arXiv:0705.2540). In this paper, we apply this technique to a variety of examples. The second part examines the first-order conditions for equality-constrained regression problems. The geometric tools that are utilised in our earlier paper are naturally applicable to these regression problems.
The Riemannian geometry of certain parameter estimation problems with singular Fisher information matrices
2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004
Many parametric statistical models suffer from "intrinsic ambiguities" in the sense that the distribution of the observation vector is invariant to smooth, structured changes in the model's parameters. The fact that certain members of the parametric statistical family are locally undistinguishable makes the Fisher information matrix (FIM) associated to the given statistical model singular. We examine such degenerate deterministic parameter estimation problems from a Riemannian geometric perspective. We start by replacing the original (ambiguous) parameter set by a lower-dimensional Riemannian (non-ambiguous) parameter set. The new parameter set comes in the form of a quotient space and is obtained by identifying equivalent family members in the initial parameterization. We specialize recently developed extensions of the Cramér-Rao bound (CRB) for the Riemannian setup to this particular setting. This offers a re-interpretation of the CRB inequality involving the pseudo-inverse of the FIM. Also, we present a lower bound on the variance (computed with respect to the geodesic distance) of unbiased estimators taking values in the quotient space. Geometrically, this corresponds to a fundamental limit on the capability of these estimators in discriminating adjacent parameter equivalence classes in the original problem parameterization. A numerical example involving the blind identification of single-input multipleoutput (SIMO) channels driven by a Gaussian source is worked out.
Bayesian Geodesic Regression on Riemannian Manifolds
2020
Geodesic regression has been proposed for fitting the geodesic curve. However, it cannot automatically choose the dimensionality of data. In this paper, we develop a Bayesian geodesic regression model on Riemannian manifolds (BGRM) model. To avoid the overfitting problem, we add a regularization term to control the effectiveness of the model. To automatically select the dimensionality, we develop a prior for the geodesic regression model, which can automatically select the number of relevant dimensions by driving unnecessary tangent vectors to zero. To show the validation of our model, we first apply it in the 3D synthetic sphere and 2D pentagon data. We then demonstrate the effectiveness of our model in reducing the dimensionality and analyzing shape variations of human corpus callosum and mandible data.
Maximum Likelihood Estimators on Manifolds
Lecture Notes in Computer Science, 2017
Maximum likelihood estimator (MLE) is a well known estimator in statistics. The popularity of this estimator stems from its asymptotic and universal properties. While asymptotic properties of MLEs on Euclidean spaces attracted a lot of interest, their studies on manifolds are still insufficient. The present paper aims to give a unified study of the subject. Its contributions are twofold. First it proposes a framework of asymptotic results for MLEs on manifolds: consistency, asymptotic normality and asymptotic efficiency. Second, it extends popular testing problems on manifolds. Some examples are discussed.
Multi-output regression on the output manifold
Pattern Recognition, 2009
Multi-output regression aims at learning a mapping from an input feature space to a multivariate output space. Previous algorithms define the loss functions using a fixed global coordinate of the output space, which is equivalent to assuming that the output space is a whole Euclidean space with a dimension equal to the number of the outputs. So the underlying structure of the output space is completely ignored. In this paper, we consider the output space as a Riemannian submanifold to incorporate its geometric structure into the regression process. To this end, we propose a novel mechanism, called locally linear transformation (LLT), to define the loss functions on the output manifold. In this way, currently existing regression algorithms can be improved. In particular, we propose an algorithm under the support vector regression framework. Our experimental results on synthetic and real-life data are satisfactory.
Regression on manifolds using kernel dimension reduction
2007
Abstract We study the problem of discovering a manifold that best preserves information relevant to a nonlinear regression. Solving this problem involves extending and uniting two threads of research. On the one hand, the literature on sufficient dimension reduction has focused on methods for finding the best linear subspace for nonlinear regression; we extend this to manifolds. On the other hand, the literature on manifold learning has focused on unsupervised dimensionality reduction; we extend this to the supervised setting.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.