Skip to content

Don't stop iterating over all storage processes in exclusion check #9846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
jzhou77 merged 1 commit into from
Mar 31, 2023
Merged

Don't stop iterating over all storage processes in exclusion check #9846

jzhou77 merged 1 commit into from
Mar 31, 2023

Conversation

@johscheuer
Copy link
Contributor

@johscheuer johscheuer commented Mar 30, 2023

Code-Reviewer Section

In the current code we stop iterating over all storage servers once we find a storage server that is in the newly excluded list.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@johscheuer johscheuer requested review from jzhou77 and sbodagala March 30, 2023 11:53
@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr-clang-ide on Linux CentOS 7

  • Commit ID: 6a612dd
  • Duration 0:15:59
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Mar 30, 2023

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 6a612dd
  • Duration 0:28:52
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 6a612dd
  • Duration 0:37:13
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 6a612dd
  • Duration 0:46:33
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 6a612dd
  • Duration 0:47:40
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 30, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 6a612dd
  • Duration 2:05:06
  • Result: ❌ FAILED
  • Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@johscheuer
Copy link
Contributor Author

johscheuer commented Mar 31, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 6a612dd
  • Duration 0:46:33
  • Result: ❌ FAILED
  • Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

I'm not sure if failure is related:

<Trace><Test TestUID="30cbbbb0-cb45-444c-938c-71734bbeb742" Statistics="TwQAAE0AAADEAAAAoAEAALQBAAD9AgAA7hMAADICAACoAgAASwAAACMBAABXCgAAyAEAAKUEAAACAwAAsAAAAFcBAAACAQAAZQAAALsAAAAiIQAAZAAAAGcCAACYLAAAxAkAAN4AAABdAgAAfwAAACABAAB+AgAAaQAAAK8SAAA6AQAAqQEAADMBAADSAAAA7AQAAHwAAAAlDAAAIgkAABYLAABKAAAAOgIAAEMAAABrBwAABAUAAEgAAABxBwAA1g4AANcAAADpAQAA8AAAAIcAAADHAAAAEAgAALMCAAAaAgAAfwAAAAEDAAAABgAA9wQAAPwAAAAPAAAAGwEAABcBAADoAAAAzQAAAAUMAABaAAAAqwcAAJIAAABCBgAA3QEAACQBAAA9iQAA0QEAAE8AAADJAwAAHQUAAMINAAC/JwAALy4AALUEAAB/AAAA5AYAAJgAAAACCgAAkAIAAPwNAACRAQAAXgAAAKAEAADGAAAAIwUAABIJAACUEwAANjgAAM8CAABJAQAAnwAAACwCAAB+CAAASQYAAGoEAACmAwAAjw0AAJwLAAACBAAAHQUAADYIAAADAwAACgoAAF4AAACdCwAAWwIAAFcAAAAbAgAAPgIAAMoBAAB7BQAA" JoshuaSeed="4920346416444127619" WillRestart="0" NegativeTest="0" RandomSeed="3794303038" SourceVersion="6a612dd85f3198bb577ae6decdd22a7f0b5b3be9" Time="1680178735" BuggifyEnabled="1" DeterminismCheck="0" FaultInjectionEnabled="1" TestFile="tests/fast/SpecialKeySpaceRobustness.toml" ConfigString="new backup_worker_enabled:=1 blob_granules_enabled:=0 commit_proxies:=1 encryption_at_rest_mode=disabled log_anti_quorum:=1 log_engine=ssd-2 log_replicas:=4 log_spill:=2 perpetual_storage_wiggle:=0 proxies:=1 storage_engine=memory storage_migration_type=disabled storage_replicas:=4 tenant_mode=required_experimental tss_count:=2 usable_regions:=1 tss_storage_engine:=1" PeakMemory="233640" Ok="0" Runtime="1028.5552582740784" FailReason="ProducedErrors" TotalTestTime="1096" TestRunCount="4"><BuggifySection File="/codebuild/output/src893364893/src/github.com/apple/foundationdb/flow/Knobs.cpp" Line="51"/>

@jzhou77 jzhou77 merged commit 3560117 into apple:main Mar 31, 2023
@jzhou77
Copy link
Contributor

jzhou77 commented Mar 31, 2023 •
edited
Loading

We probably need this for 7.1 branch as well, since #9789 is merged.

@johscheuer johscheuer deleted the fix-exclusion-check branch April 1, 2023 04:55
This was referenced Apr 1, 2023
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jzhou77 jzhou77 jzhou77 approved these changes

@sbodagala sbodagala sbodagala approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@johscheuer @foundationdb-ci @fdb-windows-ci @jzhou77 @sbodagala