Skip to content

IaC: Load the AWS catalog from the platform or cached file #13179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
k-a-il merged 6 commits into from
Nov 6, 2025
Merged

IaC: Load the AWS catalog from the platform or cached file #13179

k-a-il merged 6 commits into from
Nov 6, 2025
+191 −3

Conversation

@k-a-il
Copy link
Contributor

@k-a-il k-a-il commented Sep 22, 2025 •
edited
Loading

Motivation

This PR adds a way to load the catalog either from a saved local file or from the platform. The RemoteCatalogLoader will be used only when the catalog plugin starts. If the platform returns a different version of LocalStack than the one in the cached catalog, the local catalog file is updated.

Changes

  • Adds RemoteCatalogLoader which either loads catalog from cached file or from platform

TODO

  • Implement _get_latest_localstack_version method

@k-a-il k-a-il self-assigned this Sep 22, 2025
@k-a-il k-a-il added semver: patch Non-breaking changes which can be included in patch releases docs: skip Pull request does not require documentation changes labels Sep 22, 2025
@k-a-il k-a-il requested a review from a team September 22, 2025 08:17
@github-actions
Copy link

github-actions bot commented Sep 22, 2025 •
edited
Loading

Test Results - Preflight, Unit

22 380 tests  +4   20 630 ✅ +4   15m 52s ⏱️ +4s
     1 suites ±0    1 750 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit d70567f. ± Comparison against base commit f829e69.

This pull request removes 13 and adds 17 tests. Note that renamed tests count towards both. 
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_build_cfn_catalog_resources
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::Athena::CapacitiesReservation-athena-SUPPORTED_WITH_LICENSE_UPGRADE]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::NonExistentService::NonExistent-nonexistentservice-NOT_SUPPORTED]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::S3::Bucket-s3-AVAILABLE]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::S3::NonExistent-s3-NOT_SUPPORTED]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_service_status[athena-SUPPORTED_WITH_LICENSE_UPGRADE]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_service_status[nonexistent-NOT_SUPPORTED]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_service_status[s3-SUPPORTED]
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_service_status_with_empty_operation
tests.unit.utils.test_catalog.TestAwsCatalog ‑ test_get_service_status_with_only_one_emulator_type
…

tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_build_cfn_catalog_resources
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::Athena::CapacitiesReservation-athena-SUPPORTED_WITH_LICENSE_UPGRADE]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::NonExistentService::NonExistent-nonexistentservice-NOT_SUPPORTED]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::S3::Bucket-s3-AVAILABLE]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_cfn_resource_status[AWS::S3::NonExistent-s3-NOT_SUPPORTED]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_service_status[athena-SUPPORTED_WITH_LICENSE_UPGRADE]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_service_status[nonexistent-NOT_SUPPORTED]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_service_status[s3-SUPPORTED]
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_service_status_with_empty_operation
tests.unit.utils.catalog.test_catalog.TestAwsCatalog ‑ test_get_service_status_with_only_one_emulator_type
…

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Sep 22, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 20s ⏱️ -4s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit d70567f. ± Comparison against base commit f829e69.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Sep 22, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 38m 33s ⏱️
5 268 tests 4 739 ✅ 529 💤 0 ❌
5 274 runs  4 739 ✅ 535 💤 0 ❌

Results for commit d70567f.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Sep 22, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   2h 1m 14s ⏱️ - 1m 49s
4 894 tests ±0  4 525 ✅ ±0  369 💤 ±0  0 ❌ ±0 
4 896 runs  ±0  4 525 ✅ ±0  371 💤 ±0  0 ❌ ±0 

Results for commit d70567f. ± Comparison against base commit f829e69.

♻️ This comment has been updated with latest results.

@k-a-il k-a-il changed the title IaC: add catalog loader from platform IaC: Load the AWS catalog from the platform or cached file Sep 22, 2025
@k-a-il k-a-il marked this pull request as ready for review September 22, 2025 15:50
@k-a-il k-a-il added the review: merge when ready Signals to the reviewer that a PR can be merged if accepted label Sep 23, 2025
@k-a-il k-a-il added this to the Playground milestone Sep 23, 2025
@k-a-il k-a-il requested a review from silv-io September 23, 2025 14:59
@alexrashed alexrashed added the notes: skip Pull request does not have to be mentioned in the release notes label Sep 24, 2025
silv-io
silv-io reviewed Oct 3, 2025
View reviewed changes
Copy link
Member

@silv-io silv-io left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general! Just the latest version check is missing and some minor nitpicks in the tests.

Also kudos for finding an existing solution to the cache rather than implementing something new unnecessarily 🚀

localstack-core/localstack/utils/catalog/catalog_loader.py Outdated
Comment on lines 38 to 39
def _get_latest_localstack_version(self) -> str:
pass
Copy link
Member

@silv-io silv-io Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note We can add this functionality now that the endpoint for it will exist on the next deployment

localstack-core/localstack/utils/catalog/catalog_loader.py
try:
latest_version = self._get_latest_localstack_version()
return latest_version != current_catalog_version
except Exception as e:
Copy link
Member

@silv-io silv-io Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: After the latest version check is implemented, this should be scoped down to the actual Exceptions we'd want to handle IMO

@k-a-il k-a-il requested a review from silv-io October 20, 2025 09:01
silv-io
silv-io approved these changes Oct 24, 2025
View reviewed changes
Copy link
Member

@silv-io silv-io left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for addressing the comments and adding the latest version request

@silv-io silv-io force-pushed the iac/catalog-loader branch from d0c979c to d70567f Compare October 31, 2025 12:16
@k-a-il k-a-il merged commit bfcc671 into main Nov 6, 2025
43 checks passed
@k-a-il k-a-il deleted the iac/catalog-loader branch November 6, 2025 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@silv-io silv-io silv-io approved these changes

Assignees

@k-a-il k-a-il

Labels

docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes review: merge when ready Signals to the reviewer that a PR can be merged if accepted semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Milestone

Playground

Development

Successfully merging this pull request may close these issues.

4 participants

@k-a-il @silv-io @alexrashed