ASF: handle error serialization for Query-compatible services #13172

bentsku · 2025-09-19T19:10:15Z

Motivation

This PR is part of a bigger initiative (see #13028) to implement support for the new Smithy Protocol smithy-rpc-v2-cbor, as well as the new Botocore multi-protocol support for single services (before, a service could only define one protocol)

This PR implements the utility/parametrized fixture to start running the full CloudWatch test suite with multiple protocols. I already encountered a few issues, so decided to already open a PR and work in smaller chunks.

This PR tackles mostly the error serialization for query compatible services (they are indicated via a field in their specs).
Those service will need to return a special header (x-amzn-query-error) to help the client have the proper error serialization, see the boto handling here: https://github.com/boto/botocore/blob/de187f398ba981be728e3c69f23d6b634d555016/botocore/parsers.py#L399

For the AWS Query compatible service, when they are working with JSON, they will put the shape name as the __type field.

Looking at the spec, it seems the json protocol actually requires to use the shape name and not the error.code field when serializing error response: https://smithy.io/2.0/aws/protocols/aws-json-1_1-protocol.html#operation-error-serialization

The value of this component SHOULD contain the error's shape name.

I believe this might be the case of all json exception, but we did not realize it because for most services in their specs, we fallback to the shape.name in our scaffold script if the error.code field is not defined. For now, I'll contain this change to AWS query compatible services.

I also added a small fix in the CloudWatch service itself regarding the type of a return value that would fail in the JSON protocol.

Changes

add new parametrized fixture to run CloudWatch test with all 3 protocols
update the serializers to serialize the x-amzn-query-error when necessary, allowing client to properly handle errors
update the json serializer to use the shape name for query compatible service (to investigate if its actually all the time)
updated the SQS serializer to make us of the new x-amzn-query-error logic

…ests

github-actions · 2025-09-19T19:18:23Z

S3 Image Test Results (AMD64 / ARM64)

2 files ±0 2 suites ±0 8m 20s ⏱️ +7s
532 tests ±0 480 ✅ ±0 52 💤 ±0 0 ❌ ±0
1 064 runs ±0 960 ✅ ±0 104 💤 ±0 0 ❌ ±0

Results for commit d7d6848. ± Comparison against base commit 0809063.

♻️ This comment has been updated with latest results.

github-actions · 2025-09-19T19:30:01Z

Test Results - Preflight, Unit

22 277 tests ±0 20 539 ✅ ±0 16m 12s ⏱️ +58s
1 suites ±0 1 738 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit d7d6848. ± Comparison against base commit 0809063.

♻️ This comment has been updated with latest results.

github-actions · 2025-09-19T20:55:06Z

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 19s ⏱️ -20s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit d7d6848. ± Comparison against base commit 0809063.

♻️ This comment has been updated with latest results.

github-actions · 2025-09-19T20:56:04Z

Test Results - Alternative Providers

73 tests 39 ✅ 3m 32s ⏱️
1 suites 34 💤
1 files 0 ❌

Results for commit d7d6848.

♻️ This comment has been updated with latest results.

github-actions · 2025-09-19T21:39:04Z

Test Results (amd64) - Integration, Bootstrap

5 files 5 suites 2h 35m 38s ⏱️
5 048 tests 4 566 ✅ 482 💤 0 ❌
5 054 runs 4 566 ✅ 488 💤 0 ❌

Results for commit d7d6848.

♻️ This comment has been updated with latest results.

github-actions · 2025-09-19T21:48:07Z

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 1h 58m 5s ⏱️ +46s
4 674 tests +7 4 352 ✅ +7 322 💤 ±0 0 ❌ ±0
4 676 runs +7 4 352 ✅ +7 324 💤 ±0 0 ❌ ±0

Results for commit d7d6848. ± Comparison against base commit 0809063.

This pull request removes 3 and adds 10 tests. Note that renamed tests count towards both.

tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_gzip
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_validation
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_values_list

tests.aws.services.cloudwatch.test_cloudwatch.TestCloudWatchMultiProtocol ‑ test_multi_protocol_client_fixture[json]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudWatchMultiProtocol ‑ test_multi_protocol_client_fixture[query]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudWatchMultiProtocol ‑ test_multi_protocol_client_fixture[smithy-rpc-v2-cbor]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_gzip_with_query_protocol
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_validation[json]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_validation[query]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_validation[smithy-rpc-v2-cbor]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_values_list[json]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_values_list[query]
tests.aws.services.cloudwatch.test_cloudwatch.TestCloudwatch ‑ test_put_metric_data_values_list[smithy-rpc-v2-cbor]

♻️ This comment has been updated with latest results.

alexrashed

Nice! Really great findings, and great that you are going a step further to increase the generalization based on the findings in the project! 💯 🦸🏽

alexrashed · 2025-09-22T12:22:49Z

tests/aws/services/cloudwatch/conftest.py

+
+
+@pytest.fixture(params=["query", "json", "smithy-rpc-v2-cbor"])
+def aws_cloudwatch_client(aws_client, monkeypatch, request) -> "CloudWatchClient":


praise: Absolutely great! This means that we run all the tests which use this fixture against the different protocols! 🤩

alexrashed · 2025-09-22T12:25:15Z

localstack-core/localstack/aws/protocol/serializer.py

-        body["__type"] = error.code
+        # this depends on the JSON protocol version as well. If json-1.0 the Error should be the full shape ID, like
+        # com.amazon.coral.service#ExceptionName
+        # if json-1.1, it should only be the name


praise: Nice find! 💯
question: Is it maybe only json version specific, and the TODO is now actually done?

Right now, I left the behavior as json-1.1 like it somewhat was before. I'm not sure we will be able to fully fix this anytime soon, because the prefix is highly dependent on the exception itself and where it is raised. Example:
CloudWatch ResourceNotFound -> com.amazonaws.cloudwatch.v2010_08_01#ResourceNotFound
CloudWatch ValidationException -> com.amazon.coral.validate#ValidationException

Clients are supposed to ignore what is put before #, so maybe we can safely ignore, but I think it's good to have this behavior written down somewhere to keep track of it.

So it's not addressed yet, and I'll want to test more about this in the future 👍

alexrashed · 2025-09-22T12:27:41Z

localstack-core/localstack/aws/protocol/serializer.py

        "QueueNameExists": "QueueAlreadyExists",
    }

-    def _serialize_error(


praise: Nice! Removing this / moving this handling to the superclass means more generalization! 💯

bentsku added 5 commits September 19, 2025 19:30

implement multi-protocol client for CloudWatch and start re-running t…

95425ea

…ests

fix float encoding of get_metric_statistics

c88170d

re-add snapshots and do it iteratively

1062cc3

add query error serialization

68f18cf

fix error serialization for query compatible

7ea96d5

bentsku added this to the 4.9 milestone Sep 19, 2025

bentsku self-assigned this Sep 19, 2025

bentsku added aws:sqs Amazon Simple Queue Service aws:cloudwatch Amazon CloudWatch area: asf semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases docs: skip Pull request does not require documentation changes labels Sep 19, 2025

fix SQS sender fault

d8eb256

fix skipping for moto & fix missed error for RPC V2 CBOR

d7d6848

bentsku mentioned this pull request Sep 19, 2025

ASF: validate full CloudWatch suite with multi-protocol #13173

Merged

bentsku marked this pull request as ready for review September 19, 2025 23:39

bentsku requested review from alexrashed, baermat, gregfurman, pinzon, steffyP and thrau as code owners September 19, 2025 23:39

bentsku assigned alexrashed Sep 19, 2025

bentsku removed the request for review from steffyP September 19, 2025 23:40

bentsku removed request for baermat, gregfurman and thrau September 19, 2025 23:40

alexrashed approved these changes Sep 22, 2025

View reviewed changes

bentsku merged commit 4be950c into main Sep 22, 2025
49 checks passed

bentsku deleted the cloudwatch-multi-protocol-runs branch September 22, 2025 13:27

alexrashed added the notes: skip Pull request does not have to be mentioned in the release notes label Sep 24, 2025



		@pytest.fixture(params=["query", "json", "smithy-rpc-v2-cbor"])
		def aws_cloudwatch_client(aws_client, monkeypatch, request) -> "CloudWatchClient":

Uh oh!

ASF: handle error serialization for Query-compatible services #13172

ASF: handle error serialization for Query-compatible services #13172

Uh oh!

Conversation

bentsku commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

S3 Image Test Results (AMD64 / ARM64)

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Preflight, Unit

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Acceptance

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Alternative Providers

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Integration, Bootstrap

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

alexrashed left a comment

Choose a reason for hiding this comment

Uh oh!

alexrashed Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

alexrashed Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

bentsku Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

alexrashed Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bentsku commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading

github-actions bot commented Sep 19, 2025 •

edited

Loading