-
Notifications
You must be signed in to change notification settings - Fork 118
Closed
Labels
FeatureNew feature or requestNew feature or requestPerformanceChanges that improve performanceChanges that improve performance
Description
In components where the chooser table is copied and/or merged with other data tables (e.g. interaction-simulate and interaction-sample-simulate), we would like the option to copy ONLY the required columns/variables. Doing so should greatly reduce memory requirements when there are many un-used columns.
There are two potential approaches to this:
- Manual. Allow the user to manually specify columns to keep or columns to drop. This has already been implemented in a handful of particularly problematic components, but having a more generic / widely applicable interface for this capability would be better.
- Automatic. Scan the specification file and have the program decide in advance of copy/merge what columns will be needed.
Some concerns and complications:
- Tracing. It may be desirable for an analyst to have access to all variables in a trace file, not just the retained variables.
- Non-static variable names. It might be difficult to extract all variable name references, especially if some variable names are constructed programmatically ("on the fly") inside the model spec file, instead of appearing as a literal string.
- Estimation mode. It is desirable for an analyst to have access to all variables in estimation mode.
Metadata
Metadata
Assignees
Labels
FeatureNew feature or requestNew feature or requestPerformanceChanges that improve performanceChanges that improve performance
Type
Projects
Status
Done