👋 Thanks for your first Pull Request and for helping build the future of Gutenberg and WordPress, @tomalec! In case you missed it, we'd love to have you join us in our Slack community, where we hold regularly weekly meetings open to anyone to coordinate with each other.

If you want to learn more about WordPress development in general, check out the Core Handbook full of helpful information.

Thank you for opening this Pull Request.

I need it as I try to build a tool that helps to inspect, analyze and manage the versions being used for local unit tests, and that runs with WordPress (See articles: pb0Spc-1LK-p2, p9oQ9f-zJ-p2).

How people can access those articles? It's hard to understand how the changes proposed would benefit the WordPress project and the community without this context.

The build process in Gutenberg and WordPress core replaces all WP dependencies with wp.* globals. In practice, the versions of WP packages or core libraries like lodash or react used in the development will always differ from those in WordPress because they depend on the version of WordPress core and whether a Gutenberg plugin is installed.

How people can access those articles? It's hard to understand how the changes proposed would benefit the WordPress project and the community without this context.

I created an issue in this repo #35630
I hope it gives enough context and rationale to investigate this problem, and partial solution.

This PR tries to address problem 5. stated there

As a plugin developer, I don't even know what dependencies are extracted, as the list is maintained in the package repo, and not reported while bundling.
That may result in unexpected behavior and error. Checking that again requires even more manual effort. As it requires digging into the source code of DEWP at a given version and manually comparing packages list.

The build process in Gutenberg and WordPress core replaces all WP dependencies with wp.* globals. In practice, the versions of WP packages or core libraries like lodash or react used in the development will always differ from those in WordPress because they depend on the version of WordPress core and whether a Gutenberg plugin is installed.

That's exactly the DevX problem I'm trying to address here.

Thanks to this PR a developer would at least know which of their local dependencies are extracted and which are not. So for which packages do they need to anticipate different versions.

I run this code against
https://github.com/woocommerce/google-listings-and-ads repo's webpack config with and without the new option.

What do I have to provide to enable this option? Can you share a full example?

What do I have to provide to enable this option? Can you share a full example?

I tested this new option by modifying the tools/webpack/shared.js config file and running npm run build. A report.json file was created.

Thinking a bit more about it, if you want to see a full list of externalized dependencies you can enable combineAssets flag and it will report all of them in a single file grouped by chunks. It is also what WordPress core uses:

https://github.com/WordPress/wordpress-develop/blob/master/src/wp-includes/assets/script-loader-packages.php

What do I have to provide to enable this option? Can you share a full example?

I added more elaborate steps to reproduce in the OP #35106 (comment)

Is that clearer now?

Thinking a bit more about it, if you want to see a full list of externalized dependencies you can enable combineAssets flag and it will report all of them in a single file grouped by chunks. It is also what WordPress core uses:

WordPress/wordpress-develop@master/src/wp-includes/assets/script-loader-packages.php

Interesting. I didn't think combineAssets flag could serve to report dependency.
After reading the API docs:

By default, one asset file is created for each entry point. When this flag is set to true, all information about assets is combined into a single assets.(json|php) file generated in the output directory.

I thought it is to change the behavior of generating assets separately for each entry point to generating a single file. So to change the way it actually exposes the assets to be used on runtime.

Also, for

new DependencyExtractionWebpackPlugin( {
	  combineAssets: false,
	  outputFormat: 'json',
} );

it generates this list of internal identifiers

{
	"index.js": {
		"dependencies": [
			"lodash",
			"moment",
			"react",
			"react-dom",
			"wc-components",
			"wc-navigation",
			"wc-settings",
			"wp-a11y",
			"wp-api-fetch",
			"wp-data",
			"wp-data-controls",
			"wp-date",
			"wp-deprecated",
			"wp-dom",
			"wp-element",
			"wp-hooks",
			"wp-html-entities",
			"wp-i18n",
			"wp-is-shallow-equal",
			"wp-keycodes",
			"wp-polyfill",
			"wp-priority-queue",
			"wp-rich-text",
			"wp-url",
			"wp-warning"
		],
	// …

instead of npm package names module specifiers used in the local (plugin's) codebase

[
	"@wordpress/i18n",
	"@wordpress/hooks",
	"@woocommerce/wc-admin-settings",
	"@woocommerce/navigation",
	"@wordpress/element",
	"@woocommerce/components",
	"@wordpress/data-controls",
	"@wordpress/data",
	"@wordpress/api-fetch",
	"@babel/runtime/regenerator",
	"@wordpress/url",
	"@wordpress/date",
	"lodash",
	"moment",
	"@wordpress/deprecated",
	"@wordpress/a11y",
	"@wordpress/html-entities",
	"react",
	"@wordpress/keycodes",
	"@wordpress/priority-queue",
	"react-dom",
	"@wordpress/dom",
	"@wordpress/is-shallow-equal",
	"@wordpress/rich-text",
	"@wordpress/warning"
]

But that's pretty close, do you think we can somewhat merge those two together?
Like serializing the actual Map< externalizedDependency, scriptDependency >?

if you want to see a full list of externalized dependencies you can enable combineAssets flag

Seems to me that the purposes of combineAssets and externalizedReportFile are quite different and it's not advisable to mix them:

• combineAssets specifies how that *.asset.php files should look like, and these files are part of the published artifact. WordPress PHP code that loads the scripts depends on them.
• externalizedReportFile is not supposed to be published -- it's an intermediate report/log/stats file that's supposed to be an input to some next step in the build pipeline. Or helps the developer with debugging their dependencies.

@gziolo What's your take on the above comments?

One of the Continues Integration jobs fails even on subsequent re-tries: https://github.com/WordPress/gutenberg/runs/4261297083?check_suite_focus=true

Can you check if rebasing with trunk resolves it? I doubt there's anything wrong with this branch.

Can you check if rebasing with trunk resolves it? I doubt there's anything wrong with this branch.

Thanks, @gziolo . Rebased, everything is green now.

@gziolo is there any date when this (and two other PRs) could be released to npm? - so it could be easily used outside

See: https://github.com/WordPress/gutenberg/blob/trunk/docs/contributors/code/release.md#synchronizing-wordpress-trunk

For each Gutenberg plugin release, WordPress trunk should be synchronized. Note that the WordPress trunk branch can be closed or in "feature-freeze" mode. Usually, this happens between the first beta and the first RC of the WordPress release cycle. During this period, the Gutenberg plugin releases should not be synchronized with WordPress Core.

We are in the process of releasing the next major WordPress version (5.9) so we don't do regular npm publishing every two weeks.

I plan to run the development release with the next dist-tag later this week in case someone wants to use the most recent version earlier.

Thank you for the info!

BTW, I made a small improvement for this feature in #37377, maybe it could get into that release as well?

It's now available for testing on npm with:

npm install --save-dev @wordpress/dependency-extraction-webpack-plugin@next

The stable version should be available in the second half of January.

Awesome, thanks for the heads-up :)