Reduce dependencies according to semver

I haven't tried it yet, but does webpack reduce dependencies according to the package version? So if a module with the same version number is included from two different directories, e.g. A/node_modules/[email protected] and B/node_modules/[email protected], webpack should include it only once into the bundle.

It would be even better if webpack includes only one version as long as the major version of those two dependencies don't differ. So if myLib1 depends on [email protected] and myLib2 depends on [email protected], webpack should include only [email protected] because - according to semver - it won't break myLib2. Maybe this should be opt-in as it is potential risky.

In any case webpack should log a warning if two versions of the same module are included.

You can try npm dedupe, according to the documentation it should merge modules with the same (similar) version number. (Not tested yet)

Merging modules with different versions is very risky.

A similar feature is in my mind for a long time: Finding modules with the same code, but maybe different depedencies and merging them. At runtime I would include only a reference to the shared code and a dependency mapping table. It is a bit complex to implement, but this wouldn't break anything.

A small note: even merging modules with exact the same version can break something. If you rely on being the only user of you local module and the module has some static state. (very rar and bad style)

Definitly not in 0.9

It is risky - that's why it should be opt-in. But I don't think that bundling two different versions of e.g. underscore for the client is the right thing to do.

But you're right: This could also be the job of npm when deploying the application. You may close this one or keep it open as a reminder.

Btw: components keeps the dependency hierarchy flat. I think, both styles - deep (npm) and flat (component) - are appropriate for their targeted environment.

This will now become relevant as we're trying to split alamid in different modules.

This will come...

Here is the planned deduplication code. See it as riddle...

(function(m/*odules*/){
    for(var i in m) {
        switch(typeof m[i]) {
        case "number":
            m[i] = m[m[i]];
            break;
        case "object":
            m[i] = (function(fn) {
                var args = [].slice.call(arguments, 1);
                return function(a,b,c) {
                    m[fn].apply(null, [a,b,c].concat(args));
                };
            }.apply(null, m[i]));
            break;
        }
    }
})

EDIT: use Array.prototype.concat

Looks good! Slice, unshift, apply - that's the way we go! 👍

Should be concat.

I thought Array.prototype.concat isn't supported by IE6, but I think I was wrong... So it could be optimized a bit 😄 thanks

Could you tell us a bit about your plans? I think this is a hot topic that needs to be discussed as it's not trivial.

I'll only perform safe optimization. If enabled a hash over each module source code (before dependencies) will detect duplicates. Dependencies can and will be different.

An example without dependencies:

node_modules/a/file.js -> module id 1
node_mdoules/b/node_modules/a/file.js -> module id 2

With the source:

module.exports = { value: 123 };

would result in

{
 1: function(module, exports, require) {
  module.exports = { value: 123 };
 },
 2: 1
}

The above code fragment than transform this array.

With dependencies it is a bit complexer:

node_modules/a/index.js -> module id 3
node_mdoules/b/node_modules/a/index.js -> module id 4

With the source:

module.exports = require("./file");

Would result in

{
 3: [5, 1],
 4: [5, 2],
 5: function(module, exports, require, __webpack_dependency_xyz__) {
  module.exports = require(/*! ./file */__webpack_dependency_xyz__);
 }
}

assert(require("a") !== require("b/node_modules/a"))

So the code is not duplicated but require("a") returns a different instance than require("b/node_modules/a")? Is semver taking into account?

yes. No.

I think semver is too high level for webpack... It's working on file level, while semver is package level... semver is the job of npm.

So how do you decide that two modules are the same? String comparison?

yes, maybe optimized with a hash. Maybe I'll compare after minimizing...

added experimental deduplication support #47

0.10.0-beta20 has experimental support. --optimize-dedupe

There is some stuff missing. Webpack may exit with a "Template cannot be applied as TemplateArgument" exception.

Nice .. 👍

Here is an example: https://github.com/webpack/webpack/tree/master/examples/dedupe

And another with code splitting: https://github.com/webpack/webpack/tree/master/examples/code-splitted-dedupe

So, first using npm dedupe to reduce the dependencies according to semver and then webpack's dedupe to remove duplication on file-level (if there are some remaining) will probably have the best result.

fix(html): read a script type through the parser's own decoding

fix(html): read a `<script type>` decoded, and trim an inline script

fix(html): drop the inline-script trim, keep the decoded type

fix(html): read a <script type> decoded, and trim an inline script (#…

Navigation Menu

Repository navigation

Reduce dependencies according to semver #47

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Reduce dependencies according to semver #47

Description

Activity

sokra commented on Jan 14, 2013

sokra commented on Jan 14, 2013

jhnns commented on Jan 14, 2013

jhnns commented on Apr 4, 2013

sokra commented on Apr 4, 2013

sokra commented on Apr 4, 2013

jhnns commented on Apr 4, 2013

michaelficarra commented on Apr 4, 2013

sokra commented on Apr 4, 2013

jhnns commented on Apr 8, 2013

sokra commented on Apr 8, 2013

jhnns commented on Apr 8, 2013

sokra commented on Apr 8, 2013

jhnns commented on Apr 8, 2013

sokra commented on Apr 8, 2013

sokra commented on Jun 12, 2013

jhnns commented on Jun 12, 2013

sokra commented on Jun 13, 2013

sokra commented on Jun 13, 2013

jhnns commented on Jun 14, 2013

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions