fix(npm): slim cached full packument when minimumDependencyAge is set#35285
Merged
Conversation
When `minimumDependencyAge` is configured, Deno requests the full npm packument instead of the abbreviated install manifest, because the age check needs the `time` field (publish dates) that the abbreviated format omits. The full packument additionally carries a complete `scripts` map for every version. For frequently published packages (e.g. drizzle-orm) that bloats the cached `registry.json` to ~100MB, and reparsing it on every `deno run` spikes RSS to ~900MB. Slim the full packument down to the abbreviated shape before caching it: drop each version's `scripts` map, replacing it with a `hasInstallScript` flag derived from the install lifecycle scripts, while keeping the `time` field. This matches what the registry returns for the abbreviated format, so the installer reads the real scripts from the extracted package.json exactly as it already does for abbreviated packuments.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Setting
minimumDependencyAgeindeno.jsonforces Deno to request thefull npm packument instead of the abbreviated install manifest, because
the age check needs each version's publish date (the
timefield) andthe abbreviated format omits it. The full packument also carries a
complete
scriptsmap for every published version. For packages thatpublish very frequently (the reporter hit this with
drizzle-orm) thatinflates the cached
registry.jsonto around 100MB, and because thatfile is re-read and deserialized on every
deno run(not just oninstall), parsing it spikes resident memory to roughly 900MB. With the
setting off the same app sits around 80MB. In memory-constrained
environments like a container with a 512MB limit this is enough to
OOM-crash the process on boot.
This slims the full packument down to the abbreviated shape before it is
written to the cache: for each version the
scriptsmap is dropped andreplaced with a
hasInstallScriptflag derived from the installlifecycle scripts (
preinstall/install/postinstall), while thetimefield is kept. That is exactly the representation the registryreturns for the abbreviated format, so the only thing the cache now
retains beyond a normal abbreviated packument is the
timefield thatminimumDependencyAgeactually needs.The change is safe because it makes a full-format cache behave the same
as an abbreviated-format cache, which is already the default. Resolution
only uses versions, dependencies and dist info, none of which are
touched. The installer reads real lifecycle scripts from the extracted
package's
package.json(get_package_extra_info/fetch_from_package_json), falling back to a registry refetch only forthe deprecated string, a missing
bin, or abinding.gypnode-gypcase, which is identical to how the abbreviated format already works.
has_scriptsis computed from eitherhasInstallScriptor the presenceof install scripts in the map, so detection is preserved.
A spec test adds a registry package that has both a
publishDateand ascriptssection, installs it withminimumDependencyAgeset, andasserts that the cached
registry.jsonkeeps thetimefield, drops theper-version
scriptsmap, and recordshasInstallScript.Fixes #35281