There are open PRs for adaptive solvers that need review, revision, and merge:

• RMSprop RMSprop implementation based on G. Hinton Lecture 6  #1890: finished and ready for review in RMSprop clean up and rebase #2867
• AdaDelta AdaDelta Solver (v3) #2782: need to be aligned with the new solver interface Deduplicate solver regularization, logging, and local rates and decays #2518 Decouple the computational batch size and minibatch size by accumulating gradients #1977
• ADAM Adam solver #2856: finished and ready for review in Adam solver #2918

As a last step AdaGrad, AdaDelta, and RMSprop could likely be improved by refactoring out shared code into an AdaptiveSGDSolver

@ronghanghu these could be good warm-up projects for development.

@longjon @jeffdonahue thoughts on posting tracking issues of this kind? Does this help navigate or is it noise?