I discovered that the case newLatLngBounds which is under the hood using fitBounds has an comment that package:google_maps: ^4.0.0 is required to implement it.

package:google_maps is currently at 6.2.0 so it should be possible to implement the padding for fitBounds

Here is the comment in the code which I discovered:
https://github.com/flutter/packages/blob/9bfef9513cae3e84ddf3abc0b9c7f2cc99e8d2f4/[…]oogle_maps_flutter/google_maps_flutter_web/lib/src/convert.dart