Repository navigation
chore: hide diffs of mangled unicode strings - #25912
Conversation
CloudFormation's `GetStackTemplate` irrecoverably mangles any character not in the 7-bit ASCII range. This causes noisy output from `cdk diff` when a template contains non-English languages or emoji. We can detect this case and consider these strings equal. This can be disabled by passing `--strict`. *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
|
github-actions
Bot
added
the
beginning-contributor
github-actions
Bot
added
bug
aws-cdk-automation
added
the
pr/needs-community-review
otaviomacedo
left a comment
otaviomacedo
left a comment
There was a problem hiding this comment.
Hi, @laverdet. Sorry for missing your previous PR. I understand that CloudFormation is lossy in this respect, by putting all non-ASCII characters in the same equivalence class. But this fact would become more evident to both users and developers if we separated transformation from comparison:
- Transform the original local template into its mangled version (in memory).
- Compare the mangled version with the CFN result using
deepEquals()as it is today. - If original !== mangled, display a warning to let users know that the result of this diff may contain false negatives.
corymhall
removed
the
pr/needs-community-review
|
@otaviomacedo thank you for following up. I agree that the suggested approach is more informative and now has no downsides. I've updated the PR to implement that idea, and piggy-backed on the |
Pull request has been modified.
aws-cdk-automation
left a comment
aws-cdk-automation
left a comment
There was a problem hiding this comment.
The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.
A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.
aws-cdk-automation
added
the
pr/needs-cli-test-run
otaviomacedo
added
the
pr-linter/cli-integ-tested
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
aws-cdk-automation
removed
the
pr/needs-cli-test-run
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
I am reopening this from aws#25525 and following up on my comments here: aws#24557 (comment) aws#24557 (comment) aws#25008 (comment) aws#25008 (comment) aws#25008 (comment) aws#25008 (comment) aws#25008 (comment) aws#25008 (comment) aws#25525 (comment) aws#25525 (comment) 🫠 aws#25525 (comment) 🫠 --- Fixes aws#25309 Fixes aws#22203 Fixes aws#20212 Fixes aws#13634 Fixes aws#10523 Fixes aws#10219 See also: aws-cloudformation/cloudformation-coverage-roadmap#1220 See also: aws-cloudformation/cloudformation-coverage-roadmap#814 --- 👻 I have retitled this PR as a `chore` instead of a `fix` because @aws-cdk-automation keeps closing my PRs as abandoned even though they are clearly not abandoned. > This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error. --- @otaviomacedo @rix0rrr @TheRealAmazonKendra - I'm happy to adjust the approach, add more tests, or do what else needs to be done. I'm not getting any feedback from the team so I'm not sure how to proceed. The diff noise with non-ASCII information in cdk diff makes it difficult to find meaningful changes to our stacks. 🗿🗞️📬 **Crucially, this change only affects the CLI output and therefore an integration test isn't possible.** --- CloudFormation's `GetStackTemplate` irrecoverably mangles any character not in the 7-bit ASCII range. This causes noisy output from `cdk diff` when a template contains non-English languages or emoji. We can detect this case and consider these strings equal. *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* Many AWS services accept non-ASCII input, eg many "description" fields. CloudFormation will correctly dispatch these templates but when invoking `GetStackTemplate` the result is mangled. This causes annoying noise in the output of `cdk diff`: ``` Resources [~] AWS::Lambda::Function Lambda/Resource └─ [~] Description ├─ [-] ????? └─ [+] 🤦🏻♂️ ``` This change modifies the diff algorithm to consider the string equal if the lvalue is a mangled version of the rvalue. Of course this runs the risk of hiding changesets which modify only a single non-ASCII character to another non-ASCII character, but these fields already tend to be informative in nature.
I am reopening this from #25525
and following up on my comments here:
#24557 (comment)
#24557 (comment)
#25008 (comment)
#25008 (comment)
#25008 (comment)
#25008 (comment)
#25008 (comment)
#25008 (comment)
#25525 (comment)
#25525 (comment)
🫠 #25525 (comment) 🫠
Fixes #25309
Fixes #22203
Fixes #20212
Fixes #13634
Fixes #10523
Fixes #10219
See also: aws-cloudformation/cloudformation-coverage-roadmap#1220
See also: aws-cloudformation/cloudformation-coverage-roadmap#814
👻 I have retitled this PR as a
choreinstead of afixbecause @aws-cdk-automation keeps closing my PRs as abandoned even though they are clearly not abandoned.@otaviomacedo @rix0rrr @TheRealAmazonKendra - I'm happy to adjust the approach, add more tests, or do what else needs to be done. I'm not getting any feedback from the team so I'm not sure how to proceed. The diff noise with non-ASCII information in cdk diff makes it difficult to find meaningful changes to our stacks.
🗿🗞️📬 Crucially, this change only affects the CLI output and therefore an integration test isn't possible.
CloudFormation's
GetStackTemplateirrecoverably mangles any character not in the 7-bit ASCII range. This causes noisy output fromcdk diffwhen a template contains non-English languages or emoji. We can detect this case and consider these strings equal.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license
Many AWS services accept non-ASCII input, eg many "description" fields. CloudFormation will correctly dispatch these templates but when invoking
GetStackTemplatethe result is mangled. This causes annoying noise in the output ofcdk diff:This change modifies the diff algorithm to consider the string equal if the lvalue is a mangled version of the rvalue.
Of course this runs the risk of hiding changesets which modify only a single non-ASCII character to another non-ASCII character, but these fields already tend to be informative in nature.