feat: define test session semantics#1513
Merged
mdelapenya merged 60 commits intotestcontainers:mainfrom Sep 18, 2023
Merged
Conversation
✅ Deploy Preview for testcontainers-go ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Instead read the already shared value
* main: (32 commits) fix: remove wrong example from workspace (testcontainers#1556) chore(deps): bump the all group in /modules/localstack with 1 update (testcontainers#1552) modulegen: generate code-workspace with json marshal (testcontainers#1551) chore(deps): bump the all group in /modules/compose with 2 updates (testcontainers#1553) feat: add mariadb module (testcontainers#1548) feat(modulegen): print out VSCode workspace file if needed (testcontainers#1549) modulegen: generate md file inside internal/mkdocs (testcontainers#1543) modulegen: create internal/module and internal/modfile (testcontainers#1539) [Enhancement]: add ability to set repo:tag for ContainerRequest FromDockerfile (testcontainers#1508) Fix module generator for examples (testcontainers#1545) Update pulsar.md (testcontainers#1542) modulegen: create internal/make (testcontainers#1537) chore: fix workflow (testcontainers#1538) chore(deps): bump the all group in /examples/cockroachdb with 1 update (testcontainers#1522) chore(deps): bump the all group in /examples/bigtable with 1 update (testcontainers#1534) chore(deps): bump the all group in /modules/localstack with 4 updates (testcontainers#1535) chore(deps): bump the all group in /modules/k3s with 2 updates (testcontainers#1526) chore(deps): bump the all group in /examples/spanner with 2 updates (testcontainers#1532) chore(deps): bump the all group in /examples/firestore with 1 update (testcontainers#1523) chore(deps): bump the all group in /modules/redis with 1 update (testcontainers#1524) ...
* main: chore: refine fail-fast strategy on CI (testcontainers#1555)
* main: feat: add kafka (KRaft mode only) module (testcontainers#1610)
mdelapenya
added a commit
to mdelapenya/testcontainers-go
that referenced
this pull request
Sep 20, 2023
* main: fix: proper next version chore: prepare for next minor development cycle () chore: use new version (v0.24.0) in modules and examples fix: include sonarcloud file into the release commit modulegen: generate sonar configuration (testcontainers#1644) feat: define test session semantics (testcontainers#1513) chore(deps): bump actions/checkout from 3 to 4 (testcontainers#1623)
Merged
ttruongatl
pushed a commit
to ttruongatl/testcontainers-go
that referenced
this pull request
Feb 15, 2025
* feat: aggregate test executions at the sessionID level * chore: initialise ID in a simple manner, using the init function * fix: update tests * fix: mod tidy modules * chore: check if ryuk is running inspecting the container * chore: remove usage of deprecated labels * chore: do not pass sessionID everywhere Instead read the already shared value * chore: rename ID to SessionID * chore: define RunID as an identifier of the current test process * chore: pass RunID to reaper * feat: use parent pid creation date as part of the hash * chore: print out the test session and run IDs * fix: remove unused import * fix: remove blank line * chore: rename to processID * chore: go mod tidy modules * chore: protect sessionID and processID from undesired changes * chore: add label to Ryuk * chore: get reaper instance from the Docker response * fix: use sessionID as HTTP header * fix: proper format * chore: initialise reaper labels in a single manner * chore: do not add labels twice * fix: assertion was the opposite * fix: update test * chore: mod tidy * fix: right logic to bootstrap the reaper if it does not exist * fix: update test * chore: randomise the lookup of the reaper a little bit * chore: leverage sync.singleflight for starting the reaper just once * chore: do not call newReaper but reuseOrRecreate reaper * chore: back to sync.Once * fix: make sure there is one Ryuk instance * fix: properly set initial state for tests * fix: properly set initial state for tests * chore: make mockProvider restore the testing state * fix: mockReaperProvider must reset the state of the reaper * fix: apply reaper session ID when it exists * chore: remove unused code * chore: use reaper's sessionID * chore: use reaper's sessionID * chore: skip tests that need the reaper * chore: mod tidy * chore: move reaper tests to reaper_test.go * chore: proper order * chore: use reaper container in reused log * chore: do not panic but instead keep returning an error * chore: consistently read sessionID from session package of the reaper * fix: bring back check for reaper state if found * chore: mod tidy kafka module
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR refactors how the session ID is obtained. Instead of simply getting an UUID from the Google's library, which we will still use as default, we are going to get the current process' parent ID (parent pid) in order to make sure that any test package in the project shares the same unique identifier.
This is needed to synchronise how multiple test packages in Go are seen from the test execution standpoint: because each Go package will be compiled, packaged and tested in a separate binary, and because "go test" will call all of those test binaries for running tests, we need a way to aggregate the execution into one single hash representing that test execution. By test execution we mean:
As a consequence, with the sole goal of aggregating test execution across multiple packages, the new code will use the parent process ID (pid) of the current process and use it to generate a unique session ID. We are using the parent pid because the current process will be a child process of:
For getting an unique representation of the test session, we are creating the following hash:
As a consequence, the reaper (Ryuk) has to be started just once across the entire test session, so we have implemented a way to look up the Ryuk container, and if it exists, create our own reaper instance from the raw representation of the underlying container. With this, Ryuk should be started just once, killing all containers in the test session. The diagram for getting Ryuk:
graph TD classDef red fill:#ff7e6b, color: #fff; classDef green fill:#6fd890; classDef yellow fill:#fff68f; classDef blue fill:#668aff, color: #fff; G0[Goroutine 0] --> Mutex((sync.Mutex)) G1[Goroutine 1] --> Mutex G2[Goroutine 2] --> Mutex G3[Goroutine 3] --> Mutex Gn[Goroutine n] --> Mutex Mutex --> InstanceNil{"reaperInstance == nil"} InstanceNil -- yes --> LookupDocker{"reaper in Docker"} InstanceNil -- no --> ReaperInstance(Reaper Instance) LookupDocker -- no --> Once((sync.Once)) LookupDocker -- yes --> ReadFromDocker["Read instance from Docker"] ReadFromDocker --> ReaperInstance Once --> CreateReaper["Create Reaper Container"] CreateReaper --> ReaperInstance class Mutex,Once red class CreateReaper green class InstanceNil,LookupDocker yellow class ReadFromDocker blueThen, Ryuk has to label all the containers in the session properly, including the sessionID, which will be the label used to kill them, among others that we use, such as the testcontainers language. For that reason, we have simplified how the container labels are spread across the codebase, keeping an internal
DefaultLabelsfunction, which will set the default labels for a given sessionID (helpful for tests too).Why is it important?
A perfect use case to consume this unique identifier for a test execution would be Ryuk: avoid multiple instances per Go package. Therefore, Ryuk will label all containers in a test session to kill them. As a consequence, Ryuk has to be started just once across the entire test session.