This is currently not really a bug, but thought I would share either way.

The implemented cauchy loss formula is:

Instead of:

Mathematically I don't think this has any effect, since the residuals/jacobians will just be scaled differently.
But in my use case, I was using multiple loss functions, and a threshold schedule for some of them. Other loss functions approximate r^2 below the threshold, but this implementation has a different much smaller scale (the bigger the threshold, the smaller the scale), which made its contribution negligible.