Improve Library Loading of many DYFs#16255
Closed
dimven-adsk wants to merge 2 commits intoDynamoDS:masterfrom
Closed
Improve Library Loading of many DYFs#16255dimven-adsk wants to merge 2 commits intoDynamoDS:masterfrom
dimven-adsk wants to merge 2 commits intoDynamoDS:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR introduces performance improvements for loading many DYF files by streaming JSON content rather than loading and parsing entire files into memory. Key changes include:
- Using a new streaming JSON method in CustomNodeManager to retrieve custom node info.
- Adding a GetFromJsonDocument static method in CustomNodeDefinition to efficiently extract only the required JSON properties.
- Minor adjustments to CustomNodeInfo properties and constructors for improved deserialization.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/DynamoCore/Core/CustomNodeManager.cs | Replaces full-file JSON parsing with streaming and simplifies error handling. |
| src/DynamoCore/Core/CustomNodeDefinition.cs | Introduces a streaming JSON reader method and updates property initializations. |
reddyashish
reviewed
Jun 5, 2025
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
If you install the top 10 packages from the package manager, you'll end up with over 1300 DYFs in total. Every single one of those gets loaded in memory and processed not once but twice when reading its
CustomNodeInfo. That can be avoided completely by creating a file stream and reading the json content line by line. The DYNs are usually pretty small in size, but when there's thousands of them, this can quickly add up. The optimization proposed here is very similar to the one in #15759 and brings the exact same benefits:Current performance:
A lot of time is spent on reading and parsing the JSON data.
Proposed performance:
By streaming the files, we can improve the performance of the DYF scan function more than twice.
Declarations
Check these if you believe they are true
*.resxfilesRelease Notes
(FILL ME IN) Brief description of the fix / enhancement. Use N/A to indicate that the changes in this pull request do not apply to Release Notes. Mandatory section
Reviewers
(FILL ME IN) Reviewer 1 (If possible, assign the Reviewer for the PR)
(FILL ME IN, optional) Any additional notes to reviewers or testers.
FYIs
(FILL ME IN, Optional) Names of anyone else you wish to be notified of