Feature Description

This was raised by @paaljoachim in a support topic:

I turned on many of the Performance Feature modules. I do not see any deactivate buttons though. How would I go about deactivating?

When we were implementing the Performance screen for standalone plugins I recall discussing whether there should be deactivate functionality, but it was ruled out. I didn't really understand why, so I'm opening this issue to discuss further. There was early discussion of adding a deactivate button at #651 (comment).