The problem

Ever since Meteor 1.3, we have assumed that code published to npm and installed inside node_modules should be consumed by Meteor without any further compilation.

However, as the comments in meteor/meteor-feature-requests#6 make clear, not every npm package author compiles their code for full compatibility with older browsers, and there are cases where you might like to use Meteor compiler plugins to compile resources installed in node_modules, even if that's not something the package author intended.

A blanket policy of recompiling all packages in node_modules would be extraordinarily expensive, not to mention error-prone, because no single Babel configuration can safely process the immense variety of npm package code. Not even if you use the package's own .babelrc configuration, as the Parcel project has discovered: parcel-bundler/parcel#13

The obvious alternative to compiling everything or nothing is to let the developer specify which packages should be compiled, but that approach increases the configuration burden of Meteor applications and steepens the learning curve for Meteor developers, which is something we take very seriously. Meteor was a "zero-configuration" build tool long before that terminology was hip.

With that background in mind, we are proud to offer a technique for selectively recompiling packages and modules within node_modules that is simultaneously easy to use, efficient, and flexible enough for any conceivable use case.

Our solution

Because Meteor refuses to compile code within node_modules by default, the trick is to expose npm package code outside of node_modules using symbolic links, so that the code will be compiled as part of your application, using whatever compiler plugins you have installed.

This trick has worked for a long time, but it was never really satisfying, because you'd have to import the exposed code using a relative import like import stuff from "./imports/linked/the-package" rather than just doing import stuff from "the-package".

Even if you were very careful to stick to the relative import style, you couldn't control the way other npm packages imported the-package, so this approach was always doomed to failure—until now!

This pull request makes both of these import styles work in exactly the same way.

Technically speaking, if two modules share the same fs.realpath, but one of them is compiled by Meteor, this pull request ensures that both modules will use the compiled code.

In other words, if Meteor compiles some code as part of your application, you can create symbolic links to that code elsewhere in your application (including inside node_modules), and the same compiled code will be used everywhere.

Packages that need modification before compilation

If you want to use an npm package that needs to be recompiled, but you have to make some modifications to the package source code before the compilation happens, first clone the package repository somewhere in your application:

cd path/to/application
mkdir -p imports/linked
cd imports/linked
git clone [email protected]:acornjs/acorn.git
rm -rf acorn/.git
git add acorn

Tip: you can use git submodules to include other git repositories in your application, instead of committing the clone.

Now you can edit the package however you like, perhaps by replacing the main field of acorn/package.json with src/index.js instead of dist/acorn.js, or adding a custom .babelrc file to the root of the package.

Once you've modified the package to your liking, simply npm link it into your node_modules directory:

cd path/to/application
meteor npm link imports/linked/acorn

The npm link command works by creating a symbolic link from node_modules/acorn to the package source directory (imports/linked/acorn).

Thanks to this linkage, import { parse } from "acorn", require("acorn"), and import("acorn") will resolve to code within node_modules/acorn/..., but that code will be compiled by the Meteor, because it was also exposed via imports/linked/acorn/....

If you accidentally require("/imports/linked/acorn") instead of just require("acorn"), that's fine, because the Meteor module system conveniently aliases the modules together. To see how this aliasing works, try checking the following relationships:

require.resolve("/imports/linked/acorn") === require.resolve("acorn")
require("acorn").parse === require("./imports/linked/acorn").parse

Packages that only need to be compiled

If you want to use an npm package that isn't adequately compiled for older browsers, but you don't need to make any modifications to the package, that's even easier!

Instead of using npm link to create a symbolic link from node_modules/... to somewhere else in your application, first npm install the package into node_modules as usual, then create a symbolic link to the installed package from somewhere else in the application:

meteor npm install lodash-es
mkdir imports
cd imports
ln -s ../node_modules/loash-es .

As you might imagine, lodash-es makes liberal use of ECMAScript import and export declarations, which need to be compiled in most JS environments. Because imports/lodash-es is now compiled as Meteor application code, code within node_modules/lodash-es/... will also be compiled, and can be imported as usual.

In some sense, this is the opposite of the previous example, because you're making a symbolic link to node_modules/lodash-es instead of making node_modules/lodash-es a link to somewhere else, but the common idea is the same: every module inside lodash-es shares the same fs.realpath with a module that was compiled by Meteor, so the direction of the symbolic linkage doesn't really matter. The module will be compiled by Meteor regardless of how you import it.

Packages that only need, like, one file to be compiled

Just as you can make a symbolic link between directories in node_modules and directories elsewhere in your application, you can also make symbolic links to individual files or subdirectories:

meteor npm install some-npm-package
mkdir -p imports/linked
cd imports/linked
ln -s ../../node_modules/some-npm-package/that/file/with/class/syntax.js .

If that/file/with/class/syntax.js is the only file in some-npm-package that needs to be recompiled, you can compile it by exposing it as imports/linked/syntax.js without compiling any of the other modules in the package.

tl;dr

To compile specific npm packages, use symbolic links to expose node_modules code within your application, outside of node_modules. Meteor will compile the exposed code as if it was part of your application, using whatever compiler plugins you have installed, and also guarantee that you get the compiled code when you import from node_modules (this is they key new feature).