Skip to content

Conversation

@n-poulsen
Copy link
Contributor

When switching to the Adam optimizer on Apple computers (as SGD crashes on Apple Silicon), the learning rate was not updated. This often (almost always) caused the models to diverge.

This PR updates the learning rate, to the same values (and schedule) as the default for multi-animal projects (which use Adam):

cfg["multi_step"] = [[1e-4, 7500], [5e-5, 12000], [1e-5, 200000]]

This was tested on an M2 MacBook Air, on a dataset which diverge in less than 5 iterations using the previous schedule, and now converges nicely.

@MMathisLab MMathisLab merged commit c0dba2b into main Sep 21, 2023
@MMathisLab MMathisLab deleted the update_adam_lr branch September 21, 2023 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants