Skip to content

fix flaky TestRunContainerWithRmFlag tests (take 2)#43733

Merged
thaJeztah merged 1 commit intomoby:masterfrom
thaJeztah:fix_flaky_TestRunContainerWithRmFlag
Jun 22, 2022
Merged

fix flaky TestRunContainerWithRmFlag tests (take 2)#43733
thaJeztah merged 1 commit intomoby:masterfrom
thaJeztah:fix_flaky_TestRunContainerWithRmFlag

Conversation

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Jun 21, 2022

This is a new attempt on making these tests less flaky. The previous attempt in
commit 585c147 assumed that the test was failing
if the test-daemon still had unrelated containers present from other tests, but
it appears that the actual reason for the tests to be flaky may be that the --rm
option was moved to the daemon side and an asynchronous operation. As a result,
the container may not yet be removed once the docker run completes, which happens
frequently on Windows (likely be- cause removing containers is somewhat slower
on Windows).

This patch adds a retry-loop (using poll.WaitOn()) to wait for the container
to be removed.

make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestRunContainerWithRmFlag' test-integration

INFO: Testing against a local daemon
=== RUN   TestDockerSuite
=== RUN   TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer
=== RUN   TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero
--- PASS: TestDockerSuite (1.00s)
--- PASS: TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer (0.50s)
--- PASS: TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero (0.49s)

- A picture of a cute animal (not mandatory but encouraged)

This is a new attempt on making these tests less flaky. The previous attempt in
commit 585c147 assumed that the test was failing
if the test-daemon still had unrelated containers present from other tests, but
it appears that the actual reason for the tests to be flaky may be that the `--rm`
option was moved to the daemon side and an asynchronous operation. As a result,
the container may not yet be removed once the `docker run` completes, which happens
frequently on Windows (likely be- cause removing containers is somewhat slower
on Windows).

This patch adds a retry-loop (using `poll.WaitOn()`) to wait for the container
to be removed.

    make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestRunContainerWithRmFlag' test-integration

    INFO: Testing against a local daemon
    === RUN   TestDockerSuite
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero
    --- PASS: TestDockerSuite (1.00s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer (0.50s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero (0.49s)

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah force-pushed the fix_flaky_TestRunContainerWithRmFlag branch from d457415 to f1c111b Compare June 21, 2022 14:25
@thaJeztah thaJeztah added this to the 22.06.0 milestone Jun 22, 2022
@thaJeztah
Copy link
Member Author

Thanks for review; let me get this one in to (hopefully) get CI less flaky again

@thaJeztah thaJeztah merged commit c17a566 into moby:master Jun 22, 2022
@thaJeztah thaJeztah deleted the fix_flaky_TestRunContainerWithRmFlag branch June 22, 2022 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: TestRunContainerWithRmFlagCannotStartContainer (on windows)

3 participants