Right. It's already sort of not very efficient now, though. It reads the entire cache file just for the last-modified property. Could be done a lot more efficiently by storing meta data either in a file alongside the actual cache file - or using some JSON streaming stuff which as I wrote, might be hard to find re-usable libraries with our current minimum supported PHP version 😊

So I tried storing cache files as <?php return [...] and include them instead of file_get_contents+json_decode.. But for some reason this is much slower.. It does save a bit of memory but this is not worth it really.

I don't fully grasp why this is.. But anyway this isn't helping so I'll investigate your patch closer.

Interestingly I don't see a memory improvement, just an almost 2x slowdown with your patch.

I tried just reading the last-modified flag by doing a fopen+fseek to the end, and reading the last 80 chars to extract it, but that doesn't improve anything either.. Not sure what is going on today :D

Different approach: Simply chunk the loading of packages in the PoolBuilder. This also has the advantage that it covers all sorts of memory/cache build up cases within all kinds of repositories, not just the ComposerRepository specifically.
All it does is instead of loading e.g. 250 packages at once, we now chunk into batches of 50 packages giving the GC the opportunity to jump in and free memory.
This fixes my issue (from 100 MB down to 47 MB) and I couldn't see any negative effects on a general composer update - performance is not affected as far as I could tell.

Diff best viewed with hiding whitespace - it's basically just one more loop.

Thanks