Reproduce static content concurrency issue

For magento/magento2#39954

Reproduce a bug in which multiple concurrent processes spin up to handle the same theme package, depending on how the themes are defined with their parents.

We saw a lot of issues like

DEBUG: STDERR:Compilation from source /path/to/some.js failed
Magento\Framework\Exception\FileSystemException: The file or directory "/app/var/view_preprocessed/pub/static/frontend/path/to/some.js" cannot be copied to "/app/pub/static/frontend/path/to/some.js"

We also saw less compilation intermittently failing to generate complete styles-m.css and styles-l.css files, as multiple processes were fighting and doing work on the same file at the same time. This cause the frontend to be deployed in a broken manner which needed a redeploy to solve.

The fix

--- a/vendor/magento/module-deploy/Process/Queue.php	2024-03-01 15:00:04.455947481 +0000
+++ b/vendor/magento/module-deploy/Process/Queue.php	2024-03-05 09:06:20.203928736 +0000
@@ -263,6 +263,10 @@
             && !$this->isDeployed($package)
             && ($this->maxProcesses < 2 || (count($this->inProgress) < $this->maxProcesses))
         ) {
+            if (!isset($packages[$name])) {
+                $this->logger->debug('Preventing duplicate execution of package as it is in progress: ' . $package->getPath() . ' (pid: ' . $this->getPid($package) . ')');
+                return;
+            }
             unset($packages[$name]);
             $this->execute($package);
         }

Reproduce

Clone this repo then run composer install, you can now generate static content.

Without the bugfix we can see multiple executions of the same package, see that frontend/Custom/ztheme/default is executed multiple times

$ rm -rf pub/static var/view_preprocessed/ var/cache ; php bin/magento > /dev/null ;  php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB en_IE -vvv | grep -E 'Execute: frontend/Custom/ztheme|Prevent'
Execute: frontend/Custom/ztheme/default
Execute: frontend/Custom/ztheme/default
Execute: frontend/Custom/ztheme/en_US
Execute: frontend/Custom/ztheme/en_GB
Execute: frontend/Custom/ztheme/en_IE

With the bugfix we can see each is only executed once

$ rm -rf pub/static var/view_preprocessed/ var/cache ; php bin/magento > /dev/null ;  php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB en_IE -vvv | grep -E 'Execute: frontend/Custom/ztheme|Prevent'
Execute: frontend/Custom/ztheme/default
Preventing duplicate execution of package as it is in progress: frontend/Custom/ztheme/default (pid: 67830)
Execute: frontend/Custom/ztheme/en_US
Execute: frontend/Custom/ztheme/en_GB
Execute: frontend/Custom/ztheme/en_IE

Verification

Get a baseline

rm -rf pub/static/
php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB
find ./pub/static -type f -exec md5sum {} + | awk '{print $1, $2}' > before.txt

Apply the bugfix then do the following

rm -rf pub/static/
php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB
find ./pub/static -type f -exec md5sum {} + | awk '{print $1, $2}' > after.txt

See that all files (that matter) were generated the same

$ diff  before.txt after.txt
1c1
< 04d5dfe8b06b35834eeb526128ef11ba ./pub/static/deployed_version.txt
---
> b5e5f1122730b93197d32286a8dc65ab ./pub/static/deployed_version.txt