KMS: On Demand Key Rotation for Imported Key Material #13363

aidehn · 2025-11-10T20:53:58Z

Motivation

This year AWS made it possible to import custom key material into external keys making it possible to use KeyRotationOnDemand with external KMS keys.

Changes

Updates the KmsKey class to expose the CurrentKeyMaterialId property in it's metadata. Based on testing, it will only expose this key if its a symmetric default key. It will also only expose this field on key creation if its AWS managed (since external keys will have no key material)
The KeyMaterialId of key material depends on the KeyId of the key it's associated with, in addition to the key material. Creating a hexadecimal ID with the above requirements can be achieved using uuid5 which takes a UUID as input (for example a KeyId in addition to byte information (like key material).
- Multi-region keys needed to be handled differently since they start with mrk- which makes it an invalid UUID but based on the current implementation, the part of the key ID following mrk- is a valid UUID, and so we opted to use that instead.
- This uses an internal UUID to determine the KeyMaterialId as in theory since KeyId and KeyMaterialId is public information which could be used to determine the key material since UUID5 is deterministic (Maybe a naive implementation, so open to suggestions!).
From what I understand, we will want to update CurrentKeyMaterialId either on creation of the key or when we rotate the key to use new key material we have imported, so logic was added in these places to update these fields.
_get_all_key_ids needed to be updated as it only would query the first page, meaning tests which used it would fail once there were more keys than the page limit since it wouldn't show up on the first page.

Tests

Since we are now returning the CurrentKeyMaterialId and the KeyMaterialId fields for certain operations, all the existing parity tests which were affected were updated.
- This includes most KMS tests and some S3 tests which relied on KMS.
Removed the existing tests which tested the functionality when rotate_on_demand did not support external KMS keys.
Added test cases for the following paths:
- Creating an external KMS Key -> Importing key material -> Encrypting / Decrypting -> Importing more key material -> Rotating the key -> Encrypting / Decrypting with the new key material
- Creating an external KMS Key -> Attempting to rotate key material whilst the key is in the pending state
- Creating an external KMS Key -> Attempting to import key material when there is already material pending
- Creating an external KMS Key -> Attempting to rotate the key which has material, but none pending for rotation

resolves FLC-134
fixes #12801

github-actions · 2025-11-10T21:14:10Z

Test Results - Preflight, Unit

22 332 tests ±0 20 578 ✅ ±0 6m 12s ⏱️ -21s
1 suites ±0 1 754 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 9825b46. ± Comparison against base commit b374616.

♻️ This comment has been updated with latest results.

github-actions · 2025-11-10T21:23:08Z

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 10s ⏱️ -10s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 9825b46. ± Comparison against base commit b374616.

♻️ This comment has been updated with latest results.

github-actions · 2025-11-10T21:58:39Z

Test Results (amd64) - Integration, Bootstrap

5 files 5 suites 2h 40m 3s ⏱️
5 323 tests 4 785 ✅ 538 💤 0 ❌
5 329 runs 4 785 ✅ 544 💤 0 ❌

Results for commit 9825b46.

♻️ This comment has been updated with latest results.

github-actions · 2025-11-10T22:07:05Z

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 2h 3m 16s ⏱️ + 2m 13s
4 949 tests +7 4 571 ✅ +8 378 💤 - 1 0 ❌ ±0
4 951 runs +7 4 571 ✅ +8 380 💤 - 1 0 ❌ ±0

Results for commit 9825b46. ± Comparison against base commit b374616.

This pull request removes 2 and adds 9 tests. Note that renamed tests count towards both.

tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_raises_error_given_key_with_imported_key_material
tests.aws.services.kms.test_kms.TestKMS ‑ test_unsupported_rotate_key_on_demand_with_imported_key_material

tests.aws.services.kms.test_kms.TestKMS ‑ test_get_parameters_for_import_raises_for_non_external_kms_keys
tests.aws.services.kms.test_kms.TestKMS ‑ test_import_key_material_raises_if_there_is_already_key_material_pending
tests.aws.services.kms.test_kms.TestKMS ‑ test_key_rotation_updates_current_key_material_id_for_aws_symmetric_keys
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_for_external_keys_decrypts_with_previous_material
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_on_external_key_with_no_key_material[RSA_4096]
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_on_external_key_with_no_key_material[SYMMETRIC_DEFAULT]
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_raises_for_no_pending_key_material
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_key_on_demand_updates_current_key_material_for_external_keys
tests.aws.services.kms.test_kms.TestKMS ‑ test_rotate_on_demand_returns_arn_for_key_id

♻️ This comment has been updated with latest results.

github-actions · 2025-11-11T11:26:10Z

S3 Image Test Results (AMD64 / ARM64)

2 files 2 suites 7m 45s ⏱️
535 tests 483 ✅ 52 💤 0 ❌
1 070 runs 966 ✅ 104 💤 0 ❌

Results for commit 9825b46.

♻️ This comment has been updated with latest results.

k-a-il

It's great to see this functionality being added. Nice work on the PR 🚀 I have just a few questions, otherwise LGTM

localstack-core/localstack/services/kms/models.py

localstack-core/localstack/services/kms/provider.py

k-a-il · 2025-11-18T16:34:05Z

localstack-core/localstack/services/kms/provider.py

+        if existing_pending_material := key_to_import_material_to.crypto_key.pending_key_material:
+            pending_key_material_id = key_to_import_material_to.generate_key_material_id(
+                existing_pending_material
+            )
+            raise KMSInvalidStateException(
+                f"New key material (id: {key_material_id}) cannot be imported into KMS key "
+                f"{key_to_import_material_to.metadata['Arn']}, because another key material "
+                f"(id: {pending_key_material_id}) is pending rotation."
+            )
+


Nice additional valdiation! 🚀

…the key ID w/ tests

… key is created

…s in the response

… immediately and return the material id for symmetric keys

…resent at a time

… with the correct key material

…shots

k-a-il

Thank you for addressing the comments quickly, LGTM! 👍

aidehn force-pushed the aidehn/feature/on-demand-key-rotation-for-imported-key-material branch from da6ba8d to 7d061f5 Compare November 11, 2025 09:11

aidehn requested a review from k-a-il November 11, 2025 09:38

aidehn added docs: skip Pull request does not require documentation changes notes: needed Pull request should be mentioned in the release notes semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels Nov 11, 2025

aidehn force-pushed the aidehn/feature/on-demand-key-rotation-for-imported-key-material branch from 3a3f7c6 to 8d34956 Compare November 13, 2025 09:21

aidehn marked this pull request as ready for review November 13, 2025 09:24

aidehn requested review from bentsku, dominikschubert, sannya-singal and steffyP as code owners November 13, 2025 09:24

aidehn assigned sannya-singal and k-a-il Nov 13, 2025

aidehn force-pushed the aidehn/feature/on-demand-key-rotation-for-imported-key-material branch from 8d34956 to f3aa74e Compare November 17, 2025 09:09

aidehn assigned aidehn and unassigned sannya-singal and k-a-il Nov 18, 2025

k-a-il approved these changes Nov 18, 2025

View reviewed changes

aidehn added 7 commits November 19, 2025 11:04

support rotation of external key material, and raise on pending import

c8d4d80

add test for raising when rotating a key without key material

67dc46c

snapshot updates for key rotation on key without material

9fb7e4e

use the built in util for validating pending import state

dd87709

update rotate key functionality to return entire ARN instead of just …

312c161

…the key ID w/ tests

utility to generate the key material id

f839847

add key material id when importing custom key material

98672cb

aidehn added 23 commits November 19, 2025 11:04

poll for replicated key creation as some fields don't exist until the…

ec0e514

… key is created

update snapshots for kms parity tests

93b05ef

update transformer utility to handle key material ids

7bab0f8

poll for creation of a replicated key to ensure key material id exist…

e0e2556

…s in the response

importing key material should only update the current key material id…

79b25c5

… immediately and return the material id for symmetric keys

add test case for get parameters for import

6bd26f8

add test which validates there can only be one pending key material p…

e72e2d2

…resent at a time

fix test to store error in snapshot

5572ada

add a key material buffer for external symmetric keys

1b0ef1f

remove redundant transformers

affb916

update snapshot information for s3 tests which use kms keys

e172132

update get all key ids util to use marker instead of token

7ea9c43

formatting

d260d33

improve security surrounding key material id calculation

0046a23

update load key material to keep old behaviour for hmac keys

842790c

update comments and internal id naming

837842c

implement todo for list aliases test util

b86e580

add test to ensure after key material rotation, the data is decrypted…

5a3238c

… with the correct key material

comment update

769c3cd

validate for no pending material for external keys with test and snap…

8d9f759

…shots

formatting

16ed17b

update code to update key material even for aws managed keys

fb7b2b8

add tests to ensure that KMS key material ids are updated

28f0fac

aidehn force-pushed the aidehn/feature/on-demand-key-rotation-for-imported-key-material branch from f3aa74e to 28f0fac Compare November 19, 2025 11:04

aidehn added 2 commits November 19, 2025 11:12

format

8d4c710

safer way to remove the field from the response

9825b46

aidehn requested a review from k-a-il November 19, 2025 13:13

k-a-il approved these changes Nov 19, 2025

View reviewed changes

aidehn merged commit fedfb84 into main Nov 19, 2025
49 checks passed

aidehn deleted the aidehn/feature/on-demand-key-rotation-for-imported-key-material branch November 19, 2025 13:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

KMS: On Demand Key Rotation for Imported Key Material #13363

KMS: On Demand Key Rotation for Imported Key Material #13363

Uh oh!

aidehn commented Nov 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 11, 2025 •

edited

Loading

Uh oh!

k-a-il left a comment

Uh oh!

Uh oh!

Uh oh!

k-a-il Nov 18, 2025

Uh oh!

k-a-il left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

KMS: On Demand Key Rotation for Imported Key Material #13363

KMS: On Demand Key Rotation for Imported Key Material #13363

Uh oh!

Conversation

aidehn commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Tests

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Preflight, Unit

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Acceptance

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Integration, Bootstrap

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

github-actions bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

S3 Image Test Results (AMD64 / ARM64)

Uh oh!

k-a-il left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

k-a-il Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

k-a-il left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aidehn commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 11, 2025 •

edited

Loading