What happened:
Offshoot of #96565. We had to revert
#84206 because it revealed that secret volume directories sometimes don't get cleaned up.

We should also investigate if other atomic writer volumes are impacted, like configmap, downwardAPI, etc.

What you expected to happen:
Volume directories should be cleaned up

How to reproduce it (as minimally and precisely as possible):
Patch with #84206 and run the failing test in #96565