add support for Fn::Tranform in CFnV2 #12966

pinzon · 2025-08-06T20:47:55Z

Motivation

This PR implements the capability to resolve FnTransform in the new CFnV2 engine in multiple locations of a template.

Changes

Add properties (parent, jsonpath) to Scope class
NodeResources and NodeProperties define a possible fn::transformation
New NodeIntrinsicFunctionFnTransform class to model a Fn::Transform
_visit_intrinsic_function can return a NodeIntrinsicFunctionFnTransform
Refactor _excute_macro and AWS::Include into transformer
Add events when a transformation fails
Add template body character limit assertion
Transformer adds some extra overrides due to missing resources during transformations

Testing

Removed skip_if_v2_provider in related tests

Notes

This PR only implements the resolution of the transformations in specific locations. More locations are going to be needed in the future

github-actions · 2025-08-06T21:05:14Z

Test Results - Preflight, Unit

22 107 tests ±0 20 372 ✅ ±0 6m 20s ⏱️ -14s
1 suites ±0 1 735 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

github-actions · 2025-08-06T21:12:54Z

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 6s ⏱️ -8s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

github-actions · 2025-08-06T21:43:50Z

Test Results (amd64) - Integration, Bootstrap

5 files ±0 5 suites ±0 2h 20m 42s ⏱️ - 1m 10s
4 987 tests ±0 4 395 ✅ ±0 592 💤 ±0 0 ❌ ±0
4 993 runs ±0 4 395 ✅ ±0 598 💤 ±0 0 ❌ ±0

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

github-actions · 2025-08-06T21:58:55Z

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 1h 45m 8s ⏱️ + 2m 14s
4 628 tests ±0 4 188 ✅ ±0 440 💤 ±0 0 ❌ ±0
4 630 runs ±0 4 188 ✅ ±0 442 💤 ±0 0 ❌ ±0

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model.py

pinzon · 2025-08-13T18:39:20Z

localstack-core/localstack/services/cloudformation/v2/provider.py

+        except FailedTransformationException as e:
+            change_set.status = ChangeSetStatus.FAILED
+            change_set.status_reason = e.message
+            change_set.stack.set_stack_status(
+                status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
+            )
+            change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
+            return


@simonrw This is the section I'm not liking that much

I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.

Adding these docstrings helped me understand what the test did

simonrw

Amazing stuff, this was a really hard problem and it's solution is actually pretty clean. I am still not 100% on board with using the scope to manipulate the template structure however it is still quite a neat solution, and it prevents us from needing to add checks for Fn::Transform in parent nodes as well 👍

simonrw · 2025-08-14T21:07:31Z

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model.py

+        before_siblings: list[Any],
+        after_siblings: list[Any],


question: I'm slightly confused why the arguments can be a ChangeSetEntity but the siblings have to be separate values. When I try to change it I end up re-visiting the transform and hitting the "Invalid: Fn::Transforms cannot be nested inside another Fn::Transform" case. I understand why (the Fn::Transform is included in the siblings) and I don't think it's worth it but perhaps we can try to switch to this type (i.e. make these a single siblings: ChangeSetEntity property) in a follow up

simonrw · 2025-08-14T21:08:46Z

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model.py

+                    "Invalid: Fn::Transforms cannot be nested inside another Fn::Transform"
+                )
+
+            path = "$" + ".".join(scope.split("/")[:-1])


suggestion: could this be replaced with

Suggested change

path = "$" + ".".join(scope.split("/")[:-1])

path = scope.parent.jsonpath

simonrw · 2025-08-14T22:46:40Z

localstack-core/localstack/services/cloudformation/v2/provider.py

+        except FailedTransformationException as e:
+            change_set.status = ChangeSetStatus.FAILED
+            change_set.status_reason = e.message
+            change_set.stack.set_stack_status(
+                status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
+            )
+            change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
+            return


I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.

simonrw · 2025-08-14T22:47:12Z

tests/aws/services/cloudformation/test_change_set_fn_transform.py



-@pytest.mark.skip(reason="Not implemented with either provider")
+@skip_if_v1_provider("change sets")


simonrw · 2025-08-14T22:47:41Z

tests/aws/services/cloudformation/test_template_engine.py


    @markers.aws.validated
-    @pytest.mark.skip(reason="Fn::Transform does not support array of transformations")
+    @skip_if_v1_provider("V1 is unable to resolve fn::transform with lists")


pinzon added the semver: patch Non-breaking changes which can be included in patch releases label Aug 6, 2025

pinzon force-pushed the cp/cfn/v2/transform branch from a85bc97 to aae7310 Compare August 7, 2025 19:40

simonrw reviewed Aug 8, 2025

View reviewed changes

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model.py Outdated Show resolved Hide resolved

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model.py Show resolved Hide resolved

pinzon force-pushed the cp/cfn/v2/transform branch 3 times, most recently from 8703ae2 to b22a894 Compare August 12, 2025 20:36

pinzon requested a review from simonrw August 12, 2025 22:08

pinzon added 19 commits August 12, 2025 17:11

fix snapshot

0b13503

fix tests

1eead41

fix tests and remove skips

25f517a

basic transformation works

8e2cba5

fix snapshot typo

37dba46

move transform resolution to transformer

cc7ca75

transform as property supported

396141a

Remove transform section of a template

aacc869

90%

069965e

all tests green

3b8c3c2

unskip tests

45b86d1

make stack also return the proccesed template

7c2902f

add validations for template size

e953ab7

unused test

08f6072

resolve parameters during transformations

5047d8e

refactoring of macro execution

7a417d0

refactor macro execution to avoid old utilities

659d43a

fix getAtt during transformation

6d590f5

wip: unsuccesful transform

34356e9

pinzon added 5 commits August 12, 2025 17:16

conditionally allow if a macro can return a single str

2e2f068

add fn transform to modeling

9fc17c5

requested changes

d458d27

fix error message

cfa6f88

fix linting issue

c962e2f

pinzon force-pushed the cp/cfn/v2/transform branch from daa7743 to c962e2f Compare August 12, 2025 22:18

fix markers

edf1947

pinzon marked this pull request as ready for review August 12, 2025 22:24

pinzon requested a review from dominikschubert as a code owner August 12, 2025 22:24

pinzon added 3 commits August 12, 2025 17:43

fix markers

ea47df7

fix cdk tests

d5b8972

skip test for v1

49ecf00

pinzon commented Aug 13, 2025

View reviewed changes

fn-sub partially works during transformation

3409251

pinzon force-pushed the cp/cfn/v2/transform branch from e078fad to 3409251 Compare August 13, 2025 21:38

pinzon and others added 3 commits August 13, 2025 19:58

another override

bab886a

Add clarity to the tests

38d869a

Adding these docstrings helped me understand what the test did

Look up template outcome in cache if possible

1b4f91b

simonrw mentioned this pull request Aug 14, 2025

CFNV2: fix remaining transform tests #13010

Merged

simonrw approved these changes Aug 14, 2025

View reviewed changes

simonrw merged commit c417544 into main Aug 14, 2025
43 of 44 checks passed

simonrw deleted the cp/cfn/v2/transform branch August 14, 2025 22:49

simonrw mentioned this pull request Sep 5, 2025

CFNV2: Switch provider defaults to new engine #13098

Merged

	path = "$" + ".".join(scope.split("/")[:-1])
	path = scope.parent.jsonpath



		@pytest.mark.skip(reason="Not implemented with either provider")
		@skip_if_v1_provider("change sets")

Uh oh!

add support for Fn::Tranform in CFnV2 #12966

add support for Fn::Tranform in CFnV2 #12966

Uh oh!

Conversation

pinzon commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Testing

Notes

Uh oh!

github-actions bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Preflight, Unit

Uh oh!

github-actions bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Acceptance

Uh oh!

github-actions bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Integration, Bootstrap

Uh oh!

github-actions bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

Uh oh!

Uh oh!

pinzon Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw left a comment

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

simonrw Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pinzon commented Aug 6, 2025 •

edited

Loading

github-actions bot commented Aug 6, 2025 •

edited

Loading

github-actions bot commented Aug 6, 2025 •

edited

Loading

github-actions bot commented Aug 6, 2025 •

edited

Loading

github-actions bot commented Aug 6, 2025 •

edited

Loading