The use of the object-cache drop-in Server-Timing is clever but also clearly a hack. Ideally we wouldn't need to use this. I recall users reporting issues with the approach, for example today.

What would we lose if we didn't the Server-Timing API wasn't available before plugins_loaded?

As opposed to object-cache.php could the plugin install an mu-plugin instead?