I'm noticing extreme performance problems comparing a tsc --build to webpack with ts-loader of the same project. Sysinternals Process Monitor is showing about 11k file reads from a 1 minute tsc --build and nearly 10x ReadFile events through ts-loader, with matchingly awful CPU and memory work before an ultimately successful build hours later. I am pursuing this as a ts-loader optimization problem, but I'm not sure if the ts api gives enough control to succeed.
|
const { path: candidate, parts } = normalizePathAndParts(combinePaths(containingDirectory, moduleName)); |
|
const resolved = nodeLoadModuleByRelativeName(extensions, candidate, /*onlyRecordFailures*/ false, state, /*considerPackageJson*/ true); |
|
// Treat explicit "node_modules" import as an external library import. |
|
return resolved && toSearchResult({ resolved, isExternalLibraryImport: contains(parts, "node_modules") }); |
Perhaps it's exactly what it needs to do, but this /*considerPackageJson*/ true here is causing repeated reads/parsing of package.json files from dependencies in a row. I can override the injected readFile() and possibly avoid the read with a cache, but I don't think we can avoid the json parse because that is done internal to typescript.
Common dependencies with many internal .d.ts files are costing 6000+ read/parse of their package.json files in a run, with a long tail down to single reads.
Here is the call to compiler.resolveModuleName()
https://github.com/TypeStrong/ts-loader/blob/b8a70f91aa4a450603342e62b8c03bdd09c7a979/src/servicesHost.ts#L1175-L1199
Here is the injected readFile that is called back to read the contents of many package.json again and again, with each relative path import of a .d.ts file within its library:
https://github.com/TypeStrong/ts-loader/blob/b8a70f91aa4a450603342e62b8c03bdd09c7a979/src/servicesHost.ts#L96-L104
Once the packageJsonInfo is extracted, I suspect additional waste but one step at a time.
CC @sheetalkamat
I'm noticing extreme performance problems comparing a
tsc --buildto webpack with ts-loader of the same project. Sysinternals Process Monitor is showing about 11k file reads from a 1 minutetsc --buildand nearly 10x ReadFile events through ts-loader, with matchingly awful CPU and memory work before an ultimately successful build hours later. I am pursuing this as a ts-loader optimization problem, but I'm not sure if the ts api gives enough control to succeed.TypeScript/src/compiler/moduleNameResolver.ts
Lines 963 to 966 in d36df0d
Perhaps it's exactly what it needs to do, but this
/*considerPackageJson*/ truehere is causing repeated reads/parsing of package.json files from dependencies in a row. I can override the injectedreadFile()and possibly avoid the read with a cache, but I don't think we can avoid the json parse because that is done internal to typescript.Common dependencies with many internal .d.ts files are costing 6000+ read/parse of their package.json files in a run, with a long tail down to single reads.
Here is the call to
compiler.resolveModuleName()https://github.com/TypeStrong/ts-loader/blob/b8a70f91aa4a450603342e62b8c03bdd09c7a979/src/servicesHost.ts#L1175-L1199
Here is the injected readFile that is called back to read the contents of many package.json again and again, with each relative path import of a .d.ts file within its library:
https://github.com/TypeStrong/ts-loader/blob/b8a70f91aa4a450603342e62b8c03bdd09c7a979/src/servicesHost.ts#L96-L104
Once the packageJsonInfo is extracted, I suspect additional waste but one step at a time.
CC @sheetalkamat