For the future PhD students that are deciding in these days which offer to accept, let me give you a very simple advice:
Avoid choosing a**holes as supervisors.
It does not matter how famous they are or how prestigious is the school, you'll regret it and they don't deserve you.
New blog post:
Neural Networks (Maybe) Evolved to Make ADAM the Best Optimizer
For the first time, I wrote a blog post without math :)
I discuss a *conjecture* I have regarding Adam and the way the deep learning community produces new ideas
As promised, I compiled all my lecture notes on Online Learning in a single PDF. Feedback is welcome!
"A Modern Introduction to Online Learning"
arxiv.org/abs/1912.13213
PS Happy New Year!
It is now official: I got the @NSF CAREER award for "Parameter-free Optimization Algorithms for Machine Learning"!
Super happy :)
Soon, I'll tweet about an exciting new result about parameter-free algorithms for non-convex functions: I had this idea writing this proposal :)
New version of my book on Online Learning!
arxiv.org/abs/1912.13213
This is a BIG update, with 43 pages of additional content.
Let me list some of the changes.
🧵1/8
Dear PhD students with a submission to @NeurIPSConf, soon ~20% of you will receive a desk reject. Here some suggestions to deal with it in a healthy way.
First, this is not personal: a paper written by you was rejected, not you. Keep your self-worth unlinked from your work. 1/5
New blog post: Yet Another ICML Award Fiasco
The story of the @icmlconf 2023 Outstanding Paper Award to the D-Adaptation paper with worse results that the ones from 9 years ago
Please share it to start a needed conversation on mistakenly granted awards
As promised, we put on Arxiv the proof we did with Gemini. arxiv.org/pdf/2505.20219
This shows that the Polyak stepsize not only will not reach the optimum, but it can cycle, when used without the knowledge of f*.
Gemini failed when prompted directly ("Find an example where the
Unpopular opinion: "Optimization for Deep Learning" should be removed as a subtopic in ML conferences.
Numerical optimization is estabilished field: we don't need a worse variant of it with algorithms justified by "intuition", rarely converging, and not properly evaluated.
Given that even @BU_ece changed my title on the website, I guess it is now official:
I have been awarded tenure and promoted to Associate Professor
This was a long and stressful journey, but I f*cking did it!!! 💪🥳🎉
