Expose optimizer options as attributes when there's a single param group #1736
Description
For example, I hope I can accesses its lr!
But I read the source code, it just process the default, it does not assign them to the opposite data members.
cc @vincentqb