BackPACK

BackPACK is a library built on top of PyTorch to extract more information from a backward pass.

pip install backpack-for-pytorch

For a quick overview of the features, check backpack.pt. The code lives on Github.

Getting started

• How to use BackPACK
  • Extending the model and loss function
  • Calling the extension
• Example using all extensions
  • First order extensions
  • Second order extensions
  • Block-diagonal curvature products

Backpack

• Supported models
  • For first-order extensions
  • For second-order extensions
• Available Extensions
  • First order extensions
  • BatchGrad()
  • BatchL2Grad()
  • SumGradSquared()
  • Variance()
  • Second order extensions
  • DiagGGNMC()
  • DiagGGNExact()
  • BatchDiagGGNMC()
  • BatchDiagGGNExact()
  • KFAC()
  • KFLR()
  • KFRA()
  • DiagHessian()
  • BatchDiagHessian()
  • SqrtGGNExact()
  • SqrtGGNMC()
  • Block-diagonal curvature products
  • HMP()
  • GGNMP()
  • PCHMP()
• Good to know
  • Check that BackPACK does what you think it should do
  • Aggregating quantities and zero_grad()
  • extend-ing for first and second-order extension
  • Not (yet) supported models
• Code examples and use cases