[epic] flaky tests #37306
Description
This is a list of tests that have shown to be flakey on master. This list is collected from this gist; https://gist.github.com/tonistiigi/9e91cbb968dea83113d09ce5f9dbff79 (generated from logs since January; some of these may already have been resolved)
| Status | Count | Test | Tracking-issue |
|---|---|---|---|
| ✅ #37315 | |||
| 130 | DockerSwarmSuite.TestAPIServiceUpdatePort | #36501 / #30427 / #36743 | |
| 129 | DockerSwarmSuite.TestSwarmPublishDuplicatePorts | #36501 / #30427 / #36743 | |
| ✅ #36750 | rhel-74-selinux-overlay2-stable |
||
| 119 | DockerSuite.TestRunSeccompUnconfinedCloneUserns | note: 100% of failures is on centos-7-noselinux-devicemapper |
|
| 119 | DockerSuite.TestRunSeccompAllowPrivCloneUserns | note: 100% of failures is on centos-7-noselinux-devicemapper |
|
| ✅ #35336 | 119 | DockerDaemonSuite.TestDaemonUserNamespaceRootSetting | #35336 note: 100% of failures is on centos-7-noselinux-devicemapper |
| 116 | DockerDaemonSuite.TestRemoveContainerAfterLiveRestore | note: 100% of failures is on rhel72 && overlay |
|
| ✅ #35998 | |||
| ✅ #36502 | |||
| 55 | DockerSwarmSuite.TearDownTest | #33041 / #33805 | |
| 44 | TestServicePlugin | #37132 | |
| 41 | DockerSwarmSuite.TestAPISwarmLeaderElection | #32673 (related: #36788) | |
| 39 | TestCreateServiceSecretFileMode | #37132 | |
| 35 | DockerSwarmSuite.TestSwarmClusterRotateUnlockKey | #38885 (related: #33041) | |
| 33 | TestCreateServiceConfigFileMode | #36907 | |
| 26 | DockerSwarmSuite.TestAPISwarmServicesCreate | #36903 | |
| 21 | DockerSwarmSuite.TestSwarmLockUnlockCluster | #34051 / #35689 | |
| 20 | DockerSwarmSuite.TestAPISwarmLeaderProxy | #32562 | |
| 20 | DockerSuite.TestRunOOMExitCode | 100% of failures on CentOS and RHEL. Related: #36540 | |
| ✅ #37438 | |||
| 16 | DockerSwarmSuite.TestSwarmNetworkPluginV2 | - | |
| 16 | DockerSuite.TestEventsOOMDisableTrue | related: #37220 | |
| 15 | DockerSwarmSuite.TestAPISwarmRaftQuorum | #34988 / #35039 | |
| 14 | DockerNetworkSuite.TestDockerPluginV2NetworkDriver | 100% of failures on RHEL 7.2 + overlay (related: #28305) | |
| 9 | TestServiceRemoveKeepsIngressNetwork | #39426 |
|
| 9 | DockerSwarmSuite.TestAPISwarmServicesMultipleAgents | #23626 | |
| ✅ #36551 | |||
| 7 | DockerSwarmSuite.TestAPISwarmNodeDrainPause | #23516 | |
| 7 | DockerSuite.TestExecInteractiveStdinClose | #36877 | |
| 6 | DockerSwarmSuite.TestAPISwarmManagerRestore | #34993 | |
| 6 | DockerSuite.TestLogsTail | - | |
| 6 | DockerSuite.TestLogsContainerSmallerThanPage | - | |
| 5 | TestIsCopyFileRangeSyscallAvailable | 100% of failures on rhel-74-selinux-overlay2-stable |
|
| 5 | DockerSuite.TestLogsContainerBiggerThanPage | - | |
| 4 | TestUpdateMemory | - | |
| 4 | DockerSwarmSuite.TestSwarmJoinPromoteLocked | #35263 | |
| ✅ | |||
| 4 | DockerSuite.TestRestartContainerwithRestartPolicy | #32219 100% on rhel72 && overlay |
|
| 4 | DockerSuite.TestLogsTimestamps | - | |
| ✅ | |||
| ✅ #35982 | |||
| #27466 | 3 | DockerSwarmSuite.TestSwarmContainerAutoStart | |
| ✅ #25077 |
Other failures:
| Count | Failure | Notes |
|---|---|---|
| 25 | failed to get layer | 100% of failures on Ubuntu 16.04, either with BTRFS or ZFS |
| 10 | access denied | 100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS |
| 6 | GIT_SHA1: unbound variable | 100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS |