This repo contains papers, books, tutorials and resources on Riemannian optimization.
⚡ This repo is currently under-developed. Any suggestions are welcome!⚡
- 1. Books
- 2. Papers
- Acceleration
- Conjugate Gradient Methods
- Quasi-Newton Methods
- Second-order Methods
- Zeroth-Order Optimization
- Stochastic Optimization
- Geodesic Convex and Nonconvex Optimization
- Nonsmooth Optimization
- Constrained Optimization
- Bilevel and Min-max Optimization
- Scalable Riemannian Optimization
- Differentially Private Optimization
- Federated and Distributed Optimization
- Multi-objective Optimization
- Modern Applications to Large Models
- 3. Software
- Optimization Algorithms on Matrix Manifolds Cambridge University Press. 2023
- Convex Analysis and Optimization in Hadamard Spaces De Gruyter. 2014
- Riemannian Optimization and Its Applications Spinger. 2021.
- An introduction to Optimization on smooth manifolds Princeton University Press. 2008
- Convex Functions and Optimization Methods on Riemannian Manifolds Kluwer Academic Publishers Group. 1994
Accelerated gradient methods and analysis on Riemannian manifolds.
Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space Arxiv. 2025.
Accelerated Gradient Dynamics on Riemannian Manifolds: Faster Rate and Trajectory Convergence Arxiv. 2023.
Riemannian accelerated gradient methods via extrapolation AISTATS. 2023.
Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penalties COLT. 2023.
Riemannian Anderson Mixing Methods for Minimizing C2-Functions on Riemannian Manifolds Arxiv. 2023.
An Accelerated First-Order Method for Non-convex Optimization on Manifolds Found. Comput. Math. 2022.
Understanding Riemannian acceleration via a proximal extragradient framework COLT. 2022.
A variational formulation of accelerated optimization on Riemannian manifolds SIAM J. Math. Data Sci. 2022.
Global Riemannian acceleration in hyperbolic and spherical spaces ALT. 2022.
Accelerated optimization with orthogonality constraints J. Comp. Math. 2021.
Momentum Improves Optimization on Riemannian Manifolds AISTATS. 2021.
A Nesterov-type Acceleration with Adaptive Localized Cayley Parametrization for Optimization over the Stiefel Manifold EUSIPCO 2020.
From Nesterov’s Estimate Sequence to Riemannian Acceleration COLT. 2020.
A continuous-time perspective for modeling acceleration in Riemannian optimization AISTATS. 2020.
Accelerated First-order Methods for Geodesically Convex Optimization on Riemannian Manifolds NeurIPS. 2017.
Towards Riemannian accelerated gradient methods Arxiv. 2018.
Riemannian conjugate gradient methods
Practical gradient and conjugate gradient methods on flag manifolds Comput. Optim. Appl. 2024.
Nonlinear conjugate gradient method for vector optimization on Riemannian manifolds with retraction and vector transport Appl. Math. Comput.. 2024.
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold ICLR. 2024.
Two efficient nonlinear conjugate gradient methods for Riemannian manifolds Comput. Optim. Appl. 2024.
An improved Riemannian conjugate gradient method and its application to robust matrix completion Numer. Algorithms. 2024.
Conjugate gradient methods for optimization problems on symplectic Stiefel manifold IEEE Control Syst. Lett. 2023.
A hybrid Riemannian conjugate gradient method for nonconvex optimization problems J. Appl. Math. 2023.
A class of spectral conjugate gradient methods for Riemannian optimization Numer. Algorithms. 2023.
Global convergence of Hager–Zhang type Riemannian conjugate gradient method Appl. Math. Comput. 2023.
Riemannian Conjugate Gradient Methods: General Framework and Specific Algorithms with Convergence Analyses SIAM J. Optim. 2022.
Sufficient descent Riemannian conjugate gradient methods J. Optim. Theory Appl. 2021.
Riemannian Modified Polak--Ribière--Polyak Conjugate Gradient Order Reduced Model by Tensor Techniques SIAM J. Matrix Anal. Appl. 2020.
Hybrid Riemannian conjugate gradient methods with global convergence properties Comput. Optim. Appl. 2020.
Riemannian conjugate gradient methods with inverse retraction Comput. Optim. Appl. 2020.
A Riemannian Fletcher--Reeves Conjugate Gradient Method for Doubly Stochastic Inverse Eigenvalue Problems SIAM J. Matrix Anal. Appl. 2016.
A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions Comput. Optim. Appl. 2016.
A new, globally convergent Riemannian conjugate gradient method Optim. 2013.
Conjugate gradient on Grassmann manifolds for robust subspace estimation Image Vis. Comput. 2012.
Conjugate gradient algorithm for optimization under unitary matrix constraint Signal Process. 2009.
Quasi-Newton methods on Riemannian manifolds.
A restricted memory quasi-Newton bundle method for nonsmooth optimization on Riemannian manifolds Arxiv. 2024.
Modified Memoryless Spectral-Scaling Broyden Family on Riemannian Manifolds J. Optim. Theory Appl. 2024.
A Riemannian subspace BFGS trust region method Optim. Lett. 2023.
Memoryless Quasi-Newton Methods Based on the Spectral-Scaling Broyden Family for Riemannian Optimization J. Optim. Theory Appl. 2023.
Generalization of Quasi-Newton methods: application to robust symmetric multisecant updates AISTATS. 2021.
Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds ACML. 2021.
Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints SIAM J. Sci. Comput. 2019.
Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis AISTATS. 2018.
A Riemannian BFGS method without differentiated retraction for nonconvex optimization problems SIAM J. Optim. 2018.
A Riemannian Limited-Memory BFGS Algorithm for Computing the Matrix Geometric Mean ICCS. 2016.
A Riemannian BFGS method for nonconvex optimization problems ENUMATH. 2015.
A Broyden Class of Quasi-Newton Methods for Riemannian Optimization SIAM J. Optim. 2015.
Riemannian BFGS Algorithm with Applications Recent Advances in Optimization and its Applications in Engineering. 2010.
Second-order methods on Riemannian manifolds.
Sub-sampled adaptive trust region method on Riemannian manifolds Numer. Linear Algebra Appl. 2024.
Inexact Adaptive Cubic Regularization Algorithms on Riemannian Manifolds and Application Arxiv. 2024.
A Riemannian Dimension-Reduced Second-Order Method with Application in Sensor Network Localization SIAM J. Sci. Comput. 2024.
Riemannian Trust Region Methods for SC1 Minimization J. Sci. Comput. 2024.
Adaptive trust-region method on Riemannian manifold J. Sci. Comput. 2023.
Faster Riemannian Newton-type optimization by subsampling and cubic regularization Mach. Learn. 2023.
A Limited-Memory Riemannian Symmetric Rank-One Trust-Region Method with a Restart Strategy J. Sci. Comput. 2022.
Adaptive regularization with cubics on manifolds Math. Program. 2021.
A Nonmonotone Trust Region Method for Unconstrained Optimization Problems on Riemannian Manifolds J. Optim. Theory Appl. 2021.
Damped Newton’s method on Riemannian manifolds J. Glob. Optim. 2020.
A Riemannian trust-region method for low-rank tensor completion Numer. Linear Algebra Appl. 2018.
Adaptive Quadratically Regularized Newton Method for Riemannian Optimization SIAM J. Matrix Anal. Appl. 2018.
Inexact trust-region algorithms on Riemannian manifolds NeurIPS. 2018.
On the Superlinear Convergence of Newton’s Method on Riemannian Manifolds J. Optim. Theory Appl. 2017.
A Riemannian symmetric rank-one trust-region method Math. Program. 2015.
Low-rank matrix completion via preconditioned optimization on the Grassmann manifold Linear Algebra Its Appl. 2015.
Riemannian Trust Regions with Finite-Difference Hessian Approximations are Globally Convergent GSI. 2015.
RTRMC: A Riemannian trust-region method for low-rank matrix completion NeurIPS. 2011.
Trust-Region Methods on Riemannian Manifolds Found. Comput. Math. 2006.
Derivative-free and zeroth-order optimization on manifolds
Riemannian Accelerated Zeroth-order Algorithm: Improved Robustness and Lower Query Complexity ICML. 2024.
Zeroth-order Riemannian averaging stochastic approximation algorithms SIAM J. Optim. 2024.
Stochastic Zeroth-Order Riemannian Derivative Estimation and Optimization Math. Oper. Res. 2022.
A Modified Particle Swarm Optimization Algorithm for the Best Low Multilinear Rank Approximation of Higher-Order Tensors, Swarm Intelligence. 2010.
Direct Search Methods on Riemannian Manifolds Optim. Online 2007.
Methods for stochastic and finite-sum optimization on Riemannian manifolds
Continuous-time Riemannian SGD and SVRG Flows on Wasserstein Probabilistic Space NeurIPS. 2025.
Stochastic Modified Flows for Riemannian Stochastic Gradient Descent Arxiv. 2024.
Stochastic approximation on riemannian manifolds Appl. Math. Optim. 2021.
On Riemannian Stochastic Approximation Schemes with Fixed Step-Size AISTATS. 2021.
Averaging Stochastic Gradient Descent on Riemannian Manifolds COLT. 2018.
Stochastic gradient descent on Riemannian manifolds IEEE Trans. Autom. Control. 2013.
Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction Arxiv. 2024.
Riemannian SVRG Using Barzilai–Borwein Method as Second-Order Approximation for Federated Learning Symmetry. 2024.
Improved Variance Reduction Methods for Riemannian Non-Convex Optimization IEEE Trans. Pattern Anal. Mach. Intell. 2022.
Riemannian stochastic recursive momentum method for non-convex optimization IJCAI. 2021.
Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifolds AISTATS. 2019. IEEE Trans. Pattern Anal. Mach. Intell. 2021.
Escape saddle points faster on manifolds via perturbed riemannian stochastic recursive gradient Arxiv. 2020.
Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport SIAM J. Optim. 2019.
R-SPIDER: A fast Riemannian stochastic optimization algorithm with curvature independent rate Arxiv. 2018.
Riemannian stochastic recursive gradient algorithm ICML. 2018.
MASAGA: A Linearly-Convergent Stochastic First-Order Method for Optimization on Manifolds ECML. 2018.
Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds NeurIPS. 2016.
Adaptive Gradient Descent on Riemannian Manifolds with Nonnegative Curvature Arxiv. 2025.
A general framework of Riemannian adaptive optimization methods with a convergence analysis Arxiv. 2024.
Riemannian Adaptive Optimization Algorithm and its Application to Natural Language Processing IEEE Trans. Cybern. 2022.
Riemannian adaptive stochastic gradient algorithms on matrix manifolds ICML. 2019.
Riemannian Adaptive Optimization Methods ICLR. 2019.
Convergence theory, analysis and lower bound for geodesic convex and nonconvex optimization on Riemannian manifolds
Non-parametric Online Change Point Detection on Riemannian Manifolds. ICML. 2024.
Geodesic Convexity of the Symmetric Eigenvalue Problem and Convergence of Steepest Descent J. Optim. Theory Appl. 2024.
Curvature and complexity: Better lower bounds for geodesically convex optimization COLT. 2023.
A No-go Theorem for Robust Acceleration in the Hyperbolic Plane NeurIPS. 2021.
Global rates of convergence for nonconvex optimization on manifolds IMA J. Numer. Anal. 2019.
Geodesic convex optimization: Differentiation on manifolds, geodesics, and convexity Arxiv. 2018.
First-order Methods for Geodesically Convex Optimization COLT. 2016.
Methods for solving nonsmooth optimization on manifolds
Two Nonmonotone Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold. 2026
A Single-loop Stochastic Riemannian ADMM for Nonsmooth Optimization Arxiv. 2025.
Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds NeurIPS. 2025.
Adaptive Riemannian ADMM for Nonsmooth Optimization: Optimal Complexity without Smoothing NeurIPS. 2025.
On Relatively Smooth Optimization over Riemannian Manifolds Arxiv. 2025.
A Riemannian Proximal Newton-CG Method Arxiv. 2024.
Nonsmooth nonconvex optimization on Riemannian manifolds via bundle trust region algorithm Comput. Optim. Appl. 2024.
A Riemannian ADMM Arxiv. 2023.
An inexact Riemannian proximal gradient method Comput. Optim. Appl. 2023.
A manifold inexact augmented Lagrangian method for nonsmooth optimization on Riemannian submanifolds in Euclidean space IMA J. Numer. Anal. 2023.
A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds IMA J. Numer. Anal. 2023.
A Riemannian Smoothing Steepest Descent Method for Non-Lipschitz Optimization on Embedded Submanifolds of Rn Math. Oper. Res.. 2022.
Riemannian proximal gradient methods Math. Program. 2022.
Fenchel duality theory and a primal-dual algorithm on Riemannian manifolds, Found. Comput. Math. 2021
Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions SIAM J. Optim. 2021.
Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods SIAM J. Optim. 2021.
Proximal gradient method for nonsmooth optimization over the Stiefel manifold SIAM J. Optim. 2020.
Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds SIAM J. Optim. 2018.
A Collection of Nonsmooth Riemannian Optimization Problems Nonsmooth Optim. Appl. 2017.
A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization on Manifolds SIAM J. Optim. 2017.
A parallel Douglas Rachford algorithm for minimizing ROF-like functionals on images with values in symmetric Hadamard manifolds SIAM J. Imag. Sci 2016
Computing medians and means in Hadamard spaces SIAM J. Optim. 2014
Proximal point algorithm on Riemannian manifolds Optimization. 2002
Subgradient algorithm on Riemannian manifolds J. Optim. Theory Appl. 1998
Methods for solving optimization problems on manifolds with constraints
Local near-quadratic convergence of Riemannian interior point methods Arxiv. 2025.
Structured Regularization for Constrained Optimization on the SPD Manifold Arxiv. 2024.
Constraint Qualifications and Strong Global Convergence Properties of an Augmented Lagrangian Method on Riemannian Manifolds SIAM J. Optim. 2024.
Riemannian Interior Point Methods for Constrained Optimization on Manifolds J. Optim. Theory Appl. 2024.
A Riemannian Exponential Augmented Lagrangian Method for Computing the Projection Robust Wasserstein Distance NeurIPS. 2023.
Riemannian Projection-free Online Learning NeurIPS. 2023.
Riemannian Optimization via Frank-Wolfe Methods Math. Program. 2023.
Interior-point methods on manifolds: theory and applications FOCS. 2023.
Sequential optimality conditions for nonlinear optimization on Riemannian manifolds and a globally convergent augmented Lagrangian method Comput. Optim. Appl. 2022.
Projection-free nonconvex stochastic optimization on Riemannian manifolds IMA J. Numer. Anal. 2022.
An SQP method for equality constrained optimization on Hilbert manifolds SIAM J. Optim. 2021.
Intrinsic formulation of KKT conditions and constraint qualifications on smooth manifolds SIAM J. Optim. 2021.
Simple algorithms for optimization on Riemannian manifolds with constraints Appl. Math. Optim. 2020.
Methods for solving bilevel and min-max optimization on manifolds
An Adaptive Algorithm for Bilevel Optimization on Riemannian Manifolds Arxiv. 2025.
A framework for bilevel optimization on Riemannian manifolds NeurIPS. 2024.
Riemannian Bilevel Optimization Arxiv. 2024.
Riemannian Bilevel Optimization Arxiv. 2024.
Semivectorial bilevel optimization on Riemannian manifolds J. Optim. Theory Appl. 2015.
Proximal Gradient Descent Ascent Methods for Nonsmooth Nonconvex-Concave Minimax Problems on Riemannian Manifolds Arxiv. 2025.
Local convergence of min-max algorithms to differentiable equilibrium on Riemannian manifold Arxiv. 2024.
Extragradient Type Methods for Riemannian Variational Inequality Problems AISTATS. 2024.
Accelerated methods for riemannian min-max optimization ensuring bounded geometric penalties Arxiv. 2023.
Nonconvex-nonconcave min-max optimization on Riemannian manifolds TMLR. 2023.
Riemannian optimistic algorithms Arxiv. 2023.
Curvature-independent last-iterate convergence for games on riemannian manifolds Arxiv. 2023.
Decentralized riemannian algorithm for nonconvex minimax problems AAAI. 2023.
Riemannian Hamiltonian methods for min-max optimization on manifolds SIAM J. Optim. 2023.
Sion's minimax theorem in geodesic metric spaces and a Riemannian extragradient algorithm SIAM J. Optim. 2023.
Gradient Descent Ascent for Minimax Problems on Riemannian Manifolds IEEE Trans. Pattern Anal. Mach. Intell. 2023.
First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces NeurIPS. 2022.
Tackling expensive retraction for scaling Riemannian optimization
Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method ICML. 2025.
Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions J. Comp. Math. 2024.
A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints Arxiv. 2023.
Fast and accurate optimization on the orthogonal manifold without retraction AISTATS. 2022.
A Class of Smooth Exact Penalty Function Methods for Optimization Problems with Orthogonality Constraints Optim. Methods Softw. 2020.
Parallelizable algorithms for optimization problems with orthogonality constraints SIAM J. Sci. Comput. 2019.
Coordinate-descent for learning orthogonal matrices through Givens rotations ICML. 2014.
Low-complexity subspace-descent over symmetric positive definite manifold Arxiv. 2023.
Riemannian Coordinate Descent Algorithms on Matrix Manifolds ICML. 2024.
Dissolving Constraints for Riemannian Optimization Math. Oper. Res. 2023.
Coordinate Descent Without Coordinates: Tangent Subspace Descent on Riemannian Manifolds Math. Oper. Res. 2022.
Trivializations for Gradient-Based Optimization on Manifolds NeurIPS. 2019.
Riemannian optimization that preserves privacy in data
Federated learning on Riemannian manifolds with differential privacy Arxiv. 2024.
Differentially private Riemannian optimization Mach. Learn. 2024.
Improved Differentially Private Riemannian Optimization: Fast Sampling and Variance Reduction TMLR. 2023.
Optimization on manifolds spanning multiple deviced or agents.
Riemannian Diffusion Adaptation for Distributed Optimization on Manifolds ICML. 2025.
Distributed sparsity constrained optimization over the Stiefel manifold Neurocomputing. 2024.
A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints Arxiv. 2024.
Riemannian Diffusion Adaptation over Graphs with Application to Online Distributed PCA ICASSP. 2024.
Distributed Riemannian Stochastic Gradient Tracking Algorithm on the Stiefel Manifold Arxiv. 2024.
Riemannian SVRG Using Barzilai–Borwein Method as Second-Order Approximation for Federated Learning Symmetry. 2024.
A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds Arxiv. 2024.
Global Convergence of Decentralized Retraction-Free Optimization on the Stiefel Manifold Arxiv. 2024.
Decentralized Online Riemannian Optimization with Dynamic Environments Arxiv. 2024.
Riemannian Federated Learning via Averaging Gradient Stream Arxiv. 2024.
Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data Arxiv. 2024.
Federated learning on Riemannian manifolds with differential privacy Arxiv. 2024.
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold ICLR. 2024.
A variance-reduced stochastic gradient tracking algorithm for decentralized optimization with orthogonality constraints J. Ind. Manag. Optim.. 2023.
Decentralized Douglas-Rachford splitting methods for smooth optimization over compact submanifolds Arxiv. 2023.
Decentralized Riemannian natural gradient methods with Kronecker-product approximations Arxiv. 2023.
On the Local Linear Rate of Consensus on the Stiefel Manifold IEEE Trans. Autom. Control.. 2023.
Incremental Aggregated Riemannian Gradient Method for Distributed PCA AISTATS. 2023.
Distributed Consensus on Manifolds using the Riemannian Center of Mass CCTA. 2023.
Federated Learning on Riemannian Manifolds Appl. Set-Valued Anal. Optim. 2023.
Decentralized weakly convex optimization over the Stiefel manifold Arxiv. 2023.
Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds Arxiv. 2023.
Decentralized optimization over the Stiefel manifold by an approximate augmented Lagrangian function IEEE Trans. Signal Process. 2022.
Distributed Riemannian Optimization with Lazy Communication for Collaborative Geometric Estimation IROS. 2022.
Decentralized Riemannian gradient descent on the Stiefel manifold ICML. 2021.
Communication-Efficient Distributed PCA by Riemannian Optimization ICML. 2020.
A Riemannian gossip approach to subspace learning on Grassmann manifold Mach. Learn. 2019.
Riemannian Consensus for Manifolds With Bounded Curvature IEEE Trans. Autom. Control.. 2013.
Study optimization on manifolds with multiple objective
Multiobjective BFGS method for optimization on Riemannian manifolds Comput. Optim. Appl. 2024.
Multiobjective Conjugate Gradient Methods on Riemannian Manifolds J. Optim. Theory Appl. 2023.
Proximal algorithm with quasidistances for multiobjective quasiconvex minimization in Riemannian manifolds RAIRO-Oper. Res. 2023.
A Trust Region Method for Solving Multicriteria Optimization Problems on Riemannian Manifolds J. Optim. Theory Appl. 2023.
The generalized conditional gradient method for composite multiobjective optimization problems on Riemannian manifolds J. Nonlinear Var. Anal. 2023.
Batched Data-Driven Evolutionary Multiobjective Optimization Based on Manifold Interpolation IEEE Trans. Evol. Comput. 2023.
Iteration-Complexity and Asymptotic Analysis of Steepest Descent Method for Multiobjective Optimization on Riemannian Manifolds J. Optim. Theory Appl. 2020.
Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints Optim Lett. 2019.
Convergence analysis of a nonmonotone projected gradient method for multiobjective optimization problems Optim Lett. 2019.
Proximal Point Method for Locally Lipschitz Functions in Multiobjective Optimization of Hadamard Manifolds J. Optim. Theory Appl. 2018.
A Subgradient Method for Multiobjective Optimization on Riemannian Manifolds J. Optim. Theory Appl. 2013.
Applications of Riemannian geometry and optimization to large foundation models
mHC: Manifold-Constrained Hyper-Connections Arxiv. 2026.
LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters Arxiv. 2025.
Parameter and memory efficient pretraining via low-rank riemannian optimization ICLR. 2025.
Riemannian preconditioned lora for fine-tuning foundation models ICML. 2024.
- ManOpt: A Matlab toolbox for optimization on manifolds.
- Pymanopt: A Python toolbox for optimization on Riemannian manifolds.
- Manopt.jl: A Julia toolbox for optimization on manifolds.
- ROPTLIB: An object-oriented C++ library for optimization on Riemannian manifolds.
- McTorch: A manifold optimization library for deep learning with PyTorch.
- Geoopt: A manifold optimization and sampling toolbox with PyTorch.
- Rieoptax: A Riemannian optimization toolbox in jax.
