Skip to content

Add CLI Argument to Override Snapshot Backup Encryption #8661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sfc-gh-nwijetunga merged 8 commits into from
Nov 8, 2022
Merged

Add CLI Argument to Override Snapshot Backup Encryption #8661

sfc-gh-nwijetunga merged 8 commits into from
Nov 8, 2022

Conversation

@sfc-gh-nwijetunga
Copy link
Collaborator

@sfc-gh-nwijetunga sfc-gh-nwijetunga commented Nov 1, 2022

Add a CLI argument to allow users of fdbbackup to override the encryption cluster state for snapshot backups

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 1188dc7
  • Duration 0:12:21
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 1188dc7
  • Duration 0:12:24
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

  • Commit ID: 1188dc7
  • Duration 0:13:30
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 1188dc7
  • Duration 0:16:35
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@sfc-gh-nwijetunga
Merge branch 'main' into nim/add-option-disable-backups
cdf426f 
* main:
  Apply clang-format
  Make PROXY_MAX_TAG_THROTTLE_DURATION a server knob
  Add GrvProxyTransactionTagThrottler::maxThrottleDuration field
@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Nov 1, 2022

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 1, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-macos-m1 on macOS BigSur 11.5.2

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 1080f28
  • Duration 0:40:43
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-macos on macOS Monterey 12.x

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr on Linux CentOS 7

@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Nov 2, 2022

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 1080f28
  • Duration 1:58:19
  • Result: ❌ FAILED
  • Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: cdf426f
  • Duration 1:56:48
  • Result: ❌ FAILED
  • Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@sfc-gh-nwijetunga
Merge branch 'main' into nim/add-option-disable-backups
ebc2bac 
* main:
  s/GrvProxyTransactionTagThrottler/GrvProxyTagThrottler
  Fix segmentation fault when parsing json path in Mako (apple#8650)
  Add SSD_ROCKSDB_EXPERIMENTAL for RocksDBLogForwarder.*
  Fix some clang warnings on unused variables
  Fix the compiling error
  Update PML comments and variable names to rename "launch limits" to "weights" to reflect the new scheduling logic.  No logic changes, only renames.
sfc-gh-yiwu
sfc-gh-yiwu reviewed Nov 7, 2022
View reviewed changes
@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: b5cb4c1
  • Duration 0:47:03
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Nov 7, 2022

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: b5cb4c1
  • Duration 1:58:07
  • Result: ❌ FAILED
  • Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 7, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: f69af7d
  • Duration 0:26:42
  • Result: ❌ FAILED
  • Error: Error while executing command: docker build --label "org.foundationdb.version=${FDB_VERSION}" --label "org.foundationdb.build_date=${BUILD_DATE}" --label "org.foundationdb.commit=${COMMIT_SHA}" --progress plain --build-arg FDB_VERSION="${FDB_VERSION}" --build-arg FDB_LIBRARY_VERSIONS="${FDB_VERSION}" --build-arg FDB_WEBSITE="${FDB_WEBSITE}" --tag foundationdb/base:${FDB_VERSION}-${COMMIT_SHA} --file Dockerfile --target base .. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: f69af7d
  • Duration 0:30:39
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-macos-m1 on macOS BigSur 11.5.2

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-macos on macOS Monterey 12.x

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: f69af7d
  • Duration 0:52:45
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Nov 8, 2022

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 5851568
  • Duration 1:19:05
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

@sfc-gh-nwijetunga
Merge branch 'main' into nim/add-option-disable-backups
c0e8052 
* main: (67 commits)
  Do not set timeout for RocksDB reads in simulation. (apple#8716)
  Update BlobManager epoc after restore
  Checkpoint restore with shard (apple#8667)
  More nit changes around DD
  Add some comments in DD and fix some nit
  Mako: Initialize Arguments::num_report_files to 0 (apple#8717)
  Use a network option for retaining temporary client lib copies instead of a client knob (apple#8630)
  Fix unintended behavior in Mako (apple#8635)
  Update TLogServer in main
  Change ASSERT_WE_THINK to ASSERT when checking that peek reply start version must be greater than latest pop version
  Apply clang format
  Fix a race condition between batched peek and pop, where the server removal pop may be lost
  revert sys.path placement
  revert sys.path placement
  fixing build vs with that caused compiling aws sdk to not work properly
  Fix /GlobalTagThrottler/TagLimit unit test
  fix update txn options
  moving fdbblob in sim folder for simulations (apple#8701)
  Set minimum average cost to 1 page in GlobalTagThrottler
  Add metrics for read range. (apple#8692)
  ...
@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 5851568
  • Duration 2:05:16
  • Result: ❌ FAILED
  • Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-macos-m1 on macOS BigSur 11.5.2

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr on Linux CentOS 7

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Nov 8, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: c0e8052
  • Duration 1:58:32
  • Result: ❌ FAILED
  • Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)
  • Build Artifact (available for 30 days)

@sfc-gh-nwijetunga sfc-gh-nwijetunga merged commit 70d30d9 into apple:main Nov 8, 2022
@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Nov 8, 2022

Doxense CI Report for Windows 10

sfc-gh-nwijetunga added a commit to sfc-gh-nwijetunga/foundationdb that referenced this pull request Nov 9, 2022
@sfc-gh-nwijetunga
Merge branch 'main' into nim/fix-bulk-load
7624d3f 
* main:
  Blob single part upload (apple#8703)
  Addressed review comments
  Extend Transaction Window during Restore in Simulation (apple#8729)
  Increase default GLOBAL_TAG_THROTTLING_FOLDING_TIME
  EAR: fix Redwood encryption system key space prefix (apple#8698)
  Rocksdb stats level knob. (apple#8711)
  Add CLI Argument to Override Snapshot Backup Encryption (apple#8661)
  Disable taking the ss read lock due to a 5% read perf regression in 100% cached read, CPU-bound scenario.
  Fix go usage in mock_kms cmake commands.
  make trackExcludedServers PRIORITY_SYSTEM_IMMEDIATE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sfc-gh-yiwu sfc-gh-yiwu sfc-gh-yiwu approved these changes

@sfc-gh-ahusain sfc-gh-ahusain Awaiting requested review from sfc-gh-ahusain

Assignees

@sfc-gh-yiwu sfc-gh-yiwu

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sfc-gh-nwijetunga @foundationdb-ci @fdb-windows-ci @sfc-gh-yiwu @sfc-gh-ahusain