-
-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Cloud Formation Engine v2: Improve Computation of Ref Functions and PhysicalResourceIDs Listing #12535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
LocalStack Community integration with Pro 2 files ± 0 2 suites ±0 20m 36s ⏱️ - 1h 35m 50s Results for commit 0587ff3. ± Comparison against base commit 96a0216. This pull request removes 3910 tests.♻️ This comment has been updated with latest results. |
simonrw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general I'm ok with this but I have a couple of questions
| after_resource = delta.after | ||
| if after_resource is not None and delta.before != delta.after: | ||
| after_logical_id = after_resource.logical_id | ||
| after_physical_id = self._current_resource_physical_id( | ||
| resource_logical_id=after_logical_id | ||
| ) | ||
| after_resource.physical_resource_id = after_physical_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is mutating the delta object right? I'm not keen on mutation - part of the reason the old engine was frustrating to work with was because of the mutation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we are overriding the logic to compose the delta for resources and mutating the delta before it is returned upstream. I understand mutations can become tedious though. To avoid mutations I would probably mark the members of these deltas as finals, and then compute the references on demand rather than binding it to the resource delta. I moved away from this as it meant that downstream from the preproc are responsible for choosing where to sample this physical id from given the logical id (before and after). But it's definitely a good alternative to consider.
I think I am happy with this as we mutate the delta for a resource type in the one function that computes it; and it's also true that all the logics upstream from this point only need this entity as read-only (the resource was executed and others may depend on its computed values).
Can I propose I attempt the previous design again in a follow-up branch for us to reevaluate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After discussion we think this approach is ok as the mutation is really hidden from the caller, and is an implementation detail. A low effort alternative would be to treat delta objects as immutable and construct new ones but at this stage this change is not critical.
| after_physical_id = self._after_resource_physical_id(resource_logical_id) | ||
| if before_physical_id is None and after_physical_id is None: | ||
| raise RuntimeError(f"No PhysicalResourceId for '{resource_logical_id}'") | ||
| current_physical_id = after_physical_id or before_physical_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worry that this might lead to us incorrectly getting the physical resource id for a resource that has been deleted. Should this not enforce that the physical id is of a resource in the current stack? What is the use case for getting the physical resource of an object that is no longer in the stack?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should not happen. In a delta, deleted resources are represented as the current stack value in the before , and null values in the after. Therefore, we should never be able to sample the a physical id to assign it to the after of a resource delta, as the after of a delete resource delta does not need computing: it is null, doesn't exists (also in l77).
The executor may still record a duplicate of the deleted resource in the resources, which may be useful for the provider after execution (I'm not sure about this atm). Regardless, this is also fine for two reasons as far I can see as: (1) before definitions of resources are sampled strictly from before stores of resolved resources (as they should), and (2) even in the incorrect scenario that these are sampled from the 'current' or 'most updated value' (_current_resource_physical_id), the physical id can only be fore one for the resource being deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’ve excluded the physical ID from the delta’s comparison logic and now explicitly sample the physical resource ID from the after object of updated resources where necessary. This should make the intent behind which version of the physical ID is being used more clear in the code.
| class ChangeSetModelExecutor(ChangeSetModelPreproc): | ||
| change_set: Final[ChangeSet] | ||
| # TODO: add typing. | ||
| _change_set: Final[ChangeSet] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm personally not too fond of "hiding" these properties. What is the motivation here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here (and in general) I make member variables private when they exist solely to support the class’s (or extensions) internal logic, and exposing them externally has no semantic purpose. By restricting visibility this way, I see it helps maintain clear abstraction boundaries it is in an effort to adhere to standard practices in this context where sensible. That being said, I'd be happy to revise this if you feel it's too different from the rest of this area of code, or it is simply needed
Motivation
The current CloudFormation v2 engine has limitations in handling Ref intrinsic functions and does not populate PhysicalResourceId values during describe operations (e.g., DescribeChangeSet). The existing implementation is constrained to resource types only, and the logic for computing Ref values, both before and after changes, is complex and difficult to extend.
This PR introduces a redesigned approach to computing Ref intrinsic functions that generalizes the pattern beyond AWS::Resource types, making the engine more extensible and less convoluted in this area. It also adds support in the describer for including PhysicalResourceId values in describe steps.
Changes