Please talk to @rxaviers about this instead of building one from scratch.

Please, whoever is going to work on this, get in touch with me. I do have ideas to share.

This issue can probably be closed? It's duplicate of #1780.

I'd say it's the other way around. This issue is older. Right?

Actually, #1780 was older as it was a port from http://bugs.jquery.com but that doesn't really matter; there are comments here.

Is an online builder the answer? The more I think about it, the more I question its purpose. What problem are we really solving with an online download builder? Who are our target users?

People do want custom builds. If you look at github.com for example, they're using a custom build. Ember can work with a small subset as well. Those are pretty advanced users though.

Our hypothesis is that requiring the user to pull the whole repo, npm install with the possibility that the dependencies will be broken since the time we released, and understanding the grunt command line options for a custom build is more work than most users are willing to go through.

You could be right that it still won't be what people are looking for. I'm not sure how we determine this without building it.

Dave, thanks for the answers. I completely understand and agree that "people do want custom builds". Although, I'd like to dig in a little bit in our current hypothesis.

npm install jquery takes 7s in my brazilian internet connection and it's 2MB big. It's ran once by a developer and it currently allows one to cherry-pick whatever jQuery piece they need (src/ is there). Do we want to improve these numbers by using a download builder?

I didn't understand the dependencies problem (aren't they locked)?

Goals (TBD - I will edit this comment as points are clarified)

• Allow developers not willing to use CLI or understand the grunt options or use AMD to customize jQuery.
• Provide stable (freezed dependencies) build environment. (Does it?)

OBS: These goals are going to be changed in case we decide to distribute consumable modules instead of the monolithic --- like we do in UI, Mobile and Globalize.

I didn't understand the dependencies problem (aren't they locked)?

They're not. We specify concrete versions of our direct dependencies but dependencies of dependencies are vague. We have no control if they break in the future; the only way to control that would be to use npm shrinkwrap which is in many ways broken and even then someone may unpublish a version of a library which we depend on and it'll break.

The only way to be sure would be to commit dependencies but that would boost the repo size extremely to every dev. There's no perfect solution here.

Got it thanks. Goals. updated.

@rxaviers It's not just about dependencies of dependencies (but mostly); I wrote: "even then someone may unpublish a version of a library which we depend on and it'll break". Such a scenario is rare but possible.

Simplified that in the above goals. Thanks

If we have a separate repo used for a download builder, we could shrinkwrap or commit dependencies in that repo and it would only need to be committed every release (final or betas). I agree that if we tried committing dependencies and publishing that to Bower or npm people would come after us with torches and pitchforks, since their own projects would be massively bloated when most only want one thing--the minified file.

About the dependencies issue, I just want to point out the download builder falls into the same problem developers (using CLI) do.

Download Builder will fetch dependencies during deploy and is exposed to the same risks pointed out here (broken or unpublished dependencies). The only difference is that we manage the download builder environment and we'll be responsible to provide a fix somehow. We still need a way to prevent this problem if we really think it is a problem.

@mzgol

use npm shrinkwrap which is in many ways broken

Could you point to the issue(s) about this?

@markelog

use npm shrinkwrap which is in many ways broken

Could you point to the issue(s) about this?

Off the top of my head:
npm/npm#3581
npm/npm#4516

Generally:

1. Creating mismatches between npm-shrinkwrap.json and package.json causes silent failures.
2. npm install package --save ignores the shrinkwrap file.
3. Having the shrinkwrap in the project may cause npm install to trigger excessive network requests.
4. npm shrinkwrap isn't idempotent.

Ad. 2: to update the dependency instead of doing just npm install package@newVersion --save and commit we'd have to:

1. Ensure we have installed exactly what we have specified: npm prune && npm i.
2. Remove npm-shrinkwrap.json.
3. Update the package: npm install package@newVersion --save[-dev].
4. Re-generate the shrinkwrap file: npm shrinkwrap --dev.
5. Commit changes.

Ad. 3 in the first list: If we wanted to avoid the problem, we'd have to insert running a special cleanup scrip after the 4th point in the last list. The Angular team has sth like that: https://github.com/angular/angular.js/blob/master/scripts/clean-shrinkwrap.js

I'm pretty sure we'd be making mistakes in the process over & over again.

Fixing shrinkwrap is in the pipeline, I believe @othiym23 is involved in the process. But it hasn't happened yet; npm 3.0.0 the earliest.

None of that matters. We'd only be shrinkwrapping at the time of release and there wouldn't be an existing npm-shrinkwrap.json.

Wow, synchronicity. This outlines the problems and proposes a solution. bower/bower#1606

Unless the plan is to make jQuery a meta-project and ship smaller standalone libraries like sizzle, events, animation (which I think would be a great idea) bower/bower#1606 is a suggested solution to a slightly different problem.

I hate to be #thatguy, but I frequently point to hand-crafted per-library build tools as an anti-pattern – "[h]andcrafted per-library build tools only add complexity to an already complex problem" – and it's one of the few things I say about modularity in JavaScript where nearly every library consumer agrees vehemently.

People don't want to ship code they don't use. One solution is custom builds via a build tools. It is an especially useful solution in world where globals are dominant as a module system and you happen to have a small dependency graph.

Once apps get more complicated and the dependency graph grows custom builds start to add to, rather than subtract from, complexity. Here's an example:

Imagine a library X is really composed of smaller parts: A, B, C, D, E, and F You ship X as a standalone package, but hear that some projects only use parts A, B, and C, some only use A, D, and F, etc.

A build tool lets one project define a smaller subset of X as a dependency. When you begin to combine multiple projects that depend on different subsets of X, you run into trouble.

Let's say:

• Project 1 depends on (or possibly ships with) X(A,B,C)
• Project 2 depends on (or possibly ships with) X(A,D,F)

Do you leave it to the consumer to know they need a custom X(A,B,C,D,F) build? Do you just eat the cost of shipping the bits of A twice? The more dependencies that depend on X, the more likely you'll ship > 100% by mistake.

The best idea I've heard for resolving this problem goes like this: author in ES6 modules, transpile to builds that support ES6, AMD, CJS but retain their modularity. Also ship a globals build for people who prefer that.

For people who need to ship minimal code allow them to tree-shake (in their preferred module syntax if it supports tree-shaking) at application build time instead of library build time.

A "module first" approach, where the monolithic build is just an exception like any other sounds good to me.

@trek, is there a node-based tool chain that would do what you describe out of the box?

For the module approach, Core could take a look at Globalize, where modules are not as granular as source files, but assembled in functional groups. Similarly, there could be jquery.js (base), jquery/ajax.js, jquery/deferred.js, ...

👍 module approach. 👍 not pushing our handcrafted build tools onto consumers, but using it internally to assemble consumable/dist modules.

@jaubourg All the tooling exists. There are several options: https://github.com/esnext/es6-module-transpiler is the one I recommend. Integrating it into an existing build process is trivial: here's a sample one using Broccoli https://github.com/trek/multi-publish-example/blob/master/Brocfile.js

That repo is an example of authoring a small project in ES6 and transpiling to AMD, CJS, and Globals. npm publishing is also there. See the README for more info.

Note that if we wanted to made the src folder contain files authored in
ES6 modules, it would be a breaking change since we publish src/ to npm.
This would require bumping the major version so if we're going to do that,
3.0.0 seems like a good moment.

One way to be compatible with other module formats would be to transpile to
AMD and CommonJS and put output files in src-amd/ and src-commonjs/.
This would make our package larger which will surely make people complain
about the size (they already are). But it's a thing to consider.

And I'm happy to help however I can. Ember is already using ES6 modules so being able to do

import {request} from "jquery/ajax/xhr";

And have everything else shake out as part of an application's build process is HUGELY valuable.

We already tried a grunt-based custom build a while back and ran into the problem I described above, so I can say with 100% confidence that we wouldn't use a custom build tool. Angular 2.0 is also ES6 module based, so I imagine they'll be in the same boat.

There probably is a subset of users who would use a build tool, but I'd suggesting making that small wrapper around a real modules setup.

One way to be compatible with other module formats would be to transpile to
AMD and CommonJS and put output files in src-amd/ and src-commonjs/.
This would make our package larger which will surely make people complain
about the size (they already are). But it's a thing to consider.

Some of you should chime in on bower/bower#1606 about having a place to publish modules. I'd strongly urge you to reconsider putting compile source into version control for future versions. I'm sure you're keenly aware of the many annoying problems this causes. I'd really love for some larger projects to come together and help Bower to solve this for us.

Note that if we wanted to made the src folder contain files authored in
ES6 modules, it would be a breaking change since we publish src/ to npm.
This would require bumping the major version so if we're going to do that,
3.0.0 seems like a good moment.

Not to worry! You can get this today without breaking public API and needing a version bump:

• start authoring as ES6 modules somewhere else (e.g. lib/)

• add 'src/' to your .gitignore so it won't end up in version control

• add lib/ to your .npmignore so it it won't up up in npm

• compile from lib/ into src/

• add pre- and post-publish scripts to your package.json:

    {
      "scripts": {
        "prepublish": "some-compile-script --in ./lib --out ./src",
        "postpublish": "rm -rf ./src"
      }
    }

• Your source is in source control (Github) and your npm build is in npm

• Source control doesn't contain built assets and npm doesn't contain authoring files

This is precisely what https://github.com/trek/multi-publish-example demonstrates: I'm building from src/ to dist/. You'll notice dist/doesn't appear anywhere in the source because it's being gitignored. The relevant package.json entry is here and the main entry points to dist/cjs/index.js.

The authoring code doesn't pollute npm because it's being npm ignored

https://github.com/6to5/6to5 is also a popular ES6 to ES5 converter if you want to author in both ES6 modules and use ES6 language features.

https://github.com/brianleroux/random-number-in-range is an example of using 6to5

Because jQuery is in AMD format, we can leverage that to get custom builds.

This could be optimized:

1. Use requirejs to calculate dependencies and get the functions.
2. AMD naming conventions names variables after the filename (define(["./var/rnotwhite"], function(rnotwhite){...});). jQuery follows this with only 2 exceptions ./core -> jQuery & sizzle -> Sizzle. With this knowledge, we can wrap the callbacks in IIFEs and assign their returned values to variables named after the script name. This yields something like:

var rnotwhite = (function() {
    return (/\S+/g);
})();

Now when further functions are looking for the rnotwhite variable, it has the value of /\S+/g.
3. Add the IIFE wrapped functions to an output variable.
4. Because requirejs will only load the needed functions, we have ourselves a custom build.
5. Allow the user to download the custom build.

Proof of concept: (I wrote this just for fun. I'm not expecting anything.)
http://carameljs.com

Settling on AMD was more of a marriage of convenience for us. At the time there wasn't any standard that seemed better, plus our previous custom build system was definitely ad-hoc and much uglier. At this point I would lean towards using ES6 modules and a tool like 6to5 to transpile that into AMD modules to something like /dist/amd/*.js.

We could still use the AMD files to do the custom build though! Did you check out the jQuery UI builder? Anything we can learn/borrow from that?

I don't think it's likely we'll get this out at the same time as the 3.0 release, but it's not strictly tied to a release.

Closing in favor of tracking in the Roadmap.

Atleast if not a builder, jquery should be divided in libraries. For example, I just wanted to use the DOM manipulation part of jquery. I didn't want ajax included to increase the file size. Using a module like jquery/libs/core-dom would be great to reduce the file size.

@srahulprdxn You can use the AMD modules in src/ for that purposes.

@mgol Thanks a lot for addressing. Exactly what I needed.