Skip to content

pkg/mount/TestMount: fix wrt selinux#36750

Merged
vdemeester merged 1 commit intomoby:masterfrom
kolyshkin:t-mount-seclabel
May 23, 2018
Merged

pkg/mount/TestMount: fix wrt selinux#36750
vdemeester merged 1 commit intomoby:masterfrom
kolyshkin:t-mount-seclabel

Conversation

@kolyshkin
Copy link
Copy Markdown
Contributor

Sometimes docker-master CI fails on rhel4+selinux configuration, like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42 (PR #34965) fixed this failure, but not entirely (i.e. the test is now flaky). It looks like selinux detection code is not always working.

As the subject of this test case is definitely not selinux, it can just ignore the option added by it.

While at it, fix error messages -- add missing commas, fix a typo, and allow for clear distinction between mount and vfs (per-superblock) options.

@thaJeztah
Copy link
Copy Markdown
Member

Do we know what makes selinux detection flaky? I agree we can ignore that option, but curious why we fail to properly detect selinux

@tonistiigi
Copy link
Copy Markdown
Member

alternative #36740

@kolyshkin
Copy link
Copy Markdown
Contributor Author

Do we know what makes selinux detection flaky?

Most probably dind (selinux is detected by checking /sys/fs/selinux mount point which is not there by default). That does not explain flakiness though.

@kolyshkin
Copy link
Copy Markdown
Contributor Author

closing this one in favor of #36740; will create a new one to fix error messages on top of #36740

@kolyshkin kolyshkin closed this Mar 30, 2018
@tonistiigi
Copy link
Copy Markdown
Member

The flakiness is likely from new rhel nodes in CI. I never saw it fail on old nodes and never seen it pass on new ones.

The difference between this and #36740 is that #36740 still requires seclabel when selinuxfs is mounted.

@kolyshkin
Copy link
Copy Markdown
Contributor Author

Reopening; I think this test should not check for seclabel at all

@kolyshkin kolyshkin reopened this Mar 30, 2018
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 30, 2018

Codecov Report

Merging #36750 into master will decrease coverage by 0.27%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #36750      +/-   ##
==========================================
- Coverage   35.34%   35.06%   -0.28%     
==========================================
  Files         615      615              
  Lines       45813    45813              
==========================================
- Hits        16194    16066     -128     
- Misses      27473    27638     +165     
+ Partials     2146     2109      -37

@thaJeztah
Copy link
Copy Markdown
Member

experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/40569/console failing on
#33041

18:33:38 FAIL: docker_cli_swarm_test.go:1372: DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
18:33:38 
18:33:38 [d5499325018b2] waiting for daemon to start
18:33:38 [d5499325018b2] daemon started
18:33:38 
18:33:38 [d4f51fccb9f64] waiting for daemon to start
18:33:38 [d4f51fccb9f64] daemon started
18:33:38 
18:33:38 [dc956d769cffd] waiting for daemon to start
18:33:38 [dc956d769cffd] daemon started
18:33:38 
18:33:38 [d4f51fccb9f64] exiting daemon
18:33:38 [d4f51fccb9f64] waiting for daemon to start
18:33:38 [d4f51fccb9f64] daemon started
18:33:38 
18:33:38 [dc956d769cffd] exiting daemon
18:33:38 [dc956d769cffd] waiting for daemon to start
18:33:38 [dc956d769cffd] daemon started
18:33:38 
18:33:38 [d4f51fccb9f64] exiting daemon
18:33:38 [d4f51fccb9f64] waiting for daemon to start
18:33:38 [d4f51fccb9f64] daemon started
18:33:38 
18:33:38 [dc956d769cffd] exiting daemon
18:33:38 [dc956d769cffd] waiting for daemon to start
18:33:38 [dc956d769cffd] daemon started
18:33:38 
18:33:38 docker_cli_swarm_test.go:1449:
18:33:38     c.Assert(err, checker.IsNil)
18:33:38 ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc4201e8000), Stderr:[]uint8(nil)} ("exit status 1")
18:33:38 
18:33:38 [d5499325018b2] exiting daemon
18:33:38 [d4f51fccb9f64] exiting daemon
18:33:38 [dc956d769cffd] exiting daemon

Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42 (PR moby#34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).

As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.

While at it, fix error messages:
 - add missing commas;
 - fix a typo;
 - allow for clear distinction between mount
   and vfs (per-superblock) options.

Signed-off-by: Kir Kolyshkin <[email protected]>

Signed-off-by: Kir Kolyshkin <[email protected]>
@kolyshkin
Copy link
Copy Markdown
Contributor Author

A different failure on experimental now:

00:49:31 === RUN   TestServicePlugin
00:50:30 --- FAIL: TestServicePlugin (59.53s)
00:50:30 	daemon.go:285: [d1e7868f8b0ee] waiting for daemon to start
00:50:30 	daemon.go:317: [d1e7868f8b0ee] daemon started
00:50:30 	daemon.go:275: [d1e7868f8b0ee] exiting daemon
00:50:30 	daemon.go:285: [d828dcf4073c4] waiting for daemon to start
00:50:30 	daemon.go:317: [d828dcf4073c4] daemon started
00:50:30 	daemon.go:285: [d409f87ad25cc] waiting for daemon to start
00:50:30 	daemon.go:317: [d409f87ad25cc] daemon started
00:50:30 	daemon.go:285: [d3b90e73c9588] waiting for daemon to start
00:50:30 	daemon.go:317: [d3b90e73c9588] daemon started
00:50:30 	plugin_test.go:92: timeout hit after 30s: plugin "test" exists
00:50:30 	daemon.go:275: [d3b90e73c9588] exiting daemon
00:50:30 	daemon.go:275: [d409f87ad25cc] exiting daemon
00:50:30 	daemon.go:275: [d828dcf4073c4] exiting daemon
00:50:30 FAIL

haven't seen it before.

Rebased.

@thaJeztah
Copy link
Copy Markdown
Member

I think I've seen TestServicePlugin fail a couple of times; https://github.com/moby/moby/search?q=TestServicePlugin&type=Issues, but looks like we didn't open an issue for it. t.b.h. all those failures seem to relate to "timeouts"; could be a common cause (actual issue) somewhere.

#37086 was actually increasing the timeout for swarm tests to address the flakiness, but that was removed in favor of changing the timeout on a test-by-test base (#37086 (comment))

Opened #37132 for the flaky test

@thaJeztah
Copy link
Copy Markdown
Member

New failures (also flaky) in Janky https://jenkins.dockerproject.org/job/Docker-PRs/49536/console tracked through #36877

Looks like the fix in #37086 didn't resolve the problem; this PR has that fix, but is still failing (https://github.com/kolyshkin/moby/blob/d78e885326213e8ef89919c3cc6d16e712e852a8/integration-cli/docker_cli_exec_unix_test.go#L18-L44)

07:31:00 FAIL: docker_cli_exec_unix_test.go:18: DockerSuite.TestExecInteractiveStdinClose
07:31:00 
07:31:00 docker_cli_exec_unix_test.go:40:
07:31:00     c.Assert(strings.TrimSpace(output), checker.Equals, "hello")
07:31:00 ... obtained string = "hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
07:31:00 ... expected string = "hello"

Other failure is tracked through #34051

08:14:17 FAIL: docker_cli_swarm_test.go:1142: DockerSwarmSuite.TestSwarmLockUnlockCluster
08:14:17 
08:14:17 [dbe219fbfd416] waiting for daemon to start
08:14:17 [dbe219fbfd416] daemon started
08:14:17 
08:14:17 [de90c3ddc5830] waiting for daemon to start
08:14:17 [de90c3ddc5830] daemon started
08:14:17 
08:14:17 [d88fcd3d05a08] waiting for daemon to start
08:14:17 [d88fcd3d05a08] daemon started
08:14:17 
08:14:17 [de90c3ddc5830] exiting daemon
08:14:17 [de90c3ddc5830] waiting for daemon to start
08:14:17 [de90c3ddc5830] daemon started
08:14:17 
08:14:17 [de90c3ddc5830] exiting daemon
08:14:17 [dbe219fbfd416] exiting daemon
08:14:17 [dbe219fbfd416] waiting for daemon to start
08:14:17 [dbe219fbfd416] daemon started
08:14:17 
08:14:17 [d88fcd3d05a08] exiting daemon
08:14:17 [d88fcd3d05a08] waiting for daemon to start
08:14:17 [d88fcd3d05a08] daemon started
08:14:17 
08:14:17 [de90c3ddc5830] waiting for daemon to start
08:14:17 [de90c3ddc5830] daemon started
08:14:17 
08:14:17 [de90c3ddc5830] exiting daemon
08:14:17 [de90c3ddc5830] waiting for daemon to start
08:14:17 [de90c3ddc5830] daemon started
08:14:17 
08:14:17 [dbe219fbfd416] exiting daemon
08:14:17 [dbe219fbfd416] waiting for daemon to start
08:14:17 [dbe219fbfd416] daemon started
08:14:17 
08:14:17 docker_cli_swarm_test.go:1196:
08:14:17     checkSwarmLockedToUnlocked(c, d, unlockKey)
08:14:17 docker_cli_swarm_test.go:1017:
08:14:17     c.Assert(getNodeStatus(c, d), checker.Equals, swarm.LocalNodeStateActive)
08:14:17 ... obtained swarm.LocalNodeState = "pending"
08:14:17 ... expected swarm.LocalNodeState = "active"
08:14:17 
08:14:17 [dbe219fbfd416] exiting daemon
08:14:17 [de90c3ddc5830] exiting daemon
08:14:17 [d88fcd3d05a08] exiting daemon

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🐯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants