Skip to content

DynamoDB: Upgrade DDBLocal to v2 #11064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
viren-nadkarni merged 12 commits into from
Nov 5, 2024
Merged

DynamoDB: Upgrade DDBLocal to v2 #11064

viren-nadkarni merged 12 commits into from
Nov 5, 2024

Conversation

@viren-nadkarni
Copy link
Member

@viren-nadkarni viren-nadkarni commented Jun 20, 2024 •
edited
Loading

Background

AWS released DDB Local v2 in July 2023.

https://aws.amazon.com/about-aws/whats-new/2023/07/dynamodb-local-version-2-0/

LocalStack currently uses DDB Local v1. The EOL for DDB Local v1 which is January 2025.

Among others, DDB Local v2 supports:

  • Added support for table deletion protection and the ReturnValuesOnConditionCheckFailure parameter
  • Support for configurable maximum throughput for on-demand tables, ReturnValuesOnConditionCheckFailure, BatchExecuteStatement, and ExecuteTransactionRequest

Changes

This PR upgrades DDB Local to v2.

To do

  • Restore certain skipped snapshot attributes (like /Table/DeletionProtectionEnabled)
  • Test on Mac OS M1 (verified on MacOS 14.6.1; openjdk 21.0.5)

Related

Supersedes #9909

Depends on #11139, #11462

Closes #11038

@viren-nadkarni viren-nadkarni self-assigned this Jun 20, 2024
@viren-nadkarni viren-nadkarni added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Jun 20, 2024
@viren-nadkarni viren-nadkarni changed the title DynamoDB: Bump DDBLocal to v2 DynamoDB: Upgrade DDBLocal to v2 Jul 4, 2024
@viren-nadkarni viren-nadkarni added this to the Playground milestone Jul 22, 2024
@github-actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

LocalStack Community integration with Pro

  2 files  ±    0    2 suites  ±0   26m 14s ⏱️ - 1h 17m 7s
862 tests  - 2 647  804 ✅  - 2 292  58 💤  - 355  0 ❌ ±0 
864 runs   - 2 647  804 ✅  - 2 292  60 💤  - 355  0 ❌ ±0 

Results for commit 8faf393. ± Comparison against base commit 56e0b77.

This pull request removes 2647 tests. 
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

♻️ This comment has been updated with latest results.

@viren-nadkarni viren-nadkarni changed the base branch from master to use-java-package September 9, 2024 11:49
@viren-nadkarni viren-nadkarni force-pushed the bump-ddblocal branch 2 times, most recently from 795e4f6 to 3219d6d Compare September 9, 2024 11:52
@viren-nadkarni viren-nadkarni changed the base branch from use-java-package to master September 9, 2024 11:53
@viren-nadkarni viren-nadkarni force-pushed the bump-ddblocal branch 13 times, most recently from be62112 to 94883b6 Compare September 10, 2024 13:55
@viren-nadkarni viren-nadkarni changed the base branch from master to use-java-package September 10, 2024 13:56
@viren-nadkarni viren-nadkarni changed the base branch from use-java-package to master September 12, 2024 13:44
@localstack localstack deleted a comment from localstack-bot Sep 16, 2024
@viren-nadkarni viren-nadkarni modified the milestones: Playground, 4.0 Sep 23, 2024
@viren-nadkarni viren-nadkarni mentioned this pull request Oct 9, 2024
Closed
1 task
@viren-nadkarni
Remove JavaInstallerMixin
0e6117b 
It can not handle non-default versions
@viren-nadkarni viren-nadkarni mentioned this pull request Oct 18, 2024
Merged
1 task
viren-nadkarni
viren-nadkarni commented Nov 1, 2024
View reviewed changes
localstack-core/localstack/services/dynamodb/packages.py
if is_mac_os() and get_arch() == "arm64":
target_path = os.path.join(ddb_local_lib_dir, "libsqlite4java-osx-aarch64.dylib")
if not file_exists_not_empty(target_path):
download(LIBSQLITE_AARCH64_URL, target_path)
Copy link
Member Author

@viren-nadkarni viren-nadkarni Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These libraries are bundled with DDBv2

@viren-nadkarni viren-nadkarni marked this pull request as ready for review November 1, 2024 10:21
giograno
giograno approved these changes Nov 4, 2024
View reviewed changes
Copy link
Member

@giograno giograno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good! Thanks for taking care of the migration.

Do you have any idea if upgrading to V2 immediately breaks pods/state compatibility with the previous version?

@viren-nadkarni
Copy link
Member Author

viren-nadkarni commented Nov 5, 2024

Do you have any idea if upgrading to V2 immediately breaks pods/state compatibility with the previous version?

@giograno I'm not sure, I did not test this aspect. Would you suggest merging this closer to the v4 release than now?

@giograno
Copy link
Member

giograno commented Nov 5, 2024

@viren-nadkarni I just gave it a try by creating a pod with v1 and restoring it with v2 and it seems to work fine.
I would move forward with merging this asap, also because this PR seems to serendipitously solve also this issue.

@viren-nadkarni viren-nadkarni merged commit 5687ed4 into master Nov 5, 2024
2 checks passed
@viren-nadkarni viren-nadkarni deleted the bump-ddblocal branch November 5, 2024 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@giograno giograno giograno approved these changes

Assignees

@viren-nadkarni viren-nadkarni

Labels

semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases

Projects

None yet

Milestone

4.0

Development

Successfully merging this pull request may close these issues.

Using "old" version of DDB local

2 participants

@viren-nadkarni @giograno