CI: add oraclelinux-8 for running tests with cgroup v1 by AkihiroSuda · Pull Request #49819 · moby/moby

AkihiroSuda · 2025-04-16T10:57:20Z

- What I did
Fix #49576

- How I did it

Used Lima for running a VM of ~~template://almalinux-8~~ template://oraclelinux-8
(some tests do not pass with template://almalinux-8: #49819 (comment))

https://lima-vm.io/docs/examples/gha/

- How to verify it

Observe the CI result

- Human readable description for the release notes

- A picture of a cute animal (not mandatory but encouraged)

AkihiroSuda · 2025-04-18T15:44:53Z

Failures on almalinux-8

Rootful

https://github.com/moby/moby/actions/runs/14537211827/job/40787925346?pr=49819

== Failed
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromHost/userland-proxy=true/IPv6=false (1.09s)
    port_mapping_linux_test.go:412: assertion failed: error is not nil: Get "http://127.0.0.1:1234": read tcp 127.0.0.1:55268->127.0.0.1:1234: read: connection reset by peer
    --- FAIL: TestAccessPublishedPortFromHost/userland-proxy=true/IPv6=false (1.09s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromHost/userland-proxy=false/IPv6=false (1.09s)
    port_mapping_linux_test.go:412: assertion failed: error is not nil: Get "http://127.0.0.1:1235": dial tcp 127.0.0.1:1235: connect: connection refused
    --- FAIL: TestAccessPublishedPortFromHost/userland-proxy=false/IPv6=false (1.09s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromHost/userland-proxy=true/IPv6=true (1.13s)
    port_mapping_linux_test.go:412: assertion failed: error is not nil: Get "http://[::1]:1236": read tcp [::1]:50406->[::1]:1236: read: connection reset by peer
    --- FAIL: TestAccessPublishedPortFromHost/userland-proxy=true/IPv6=true (1.13s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromHost (4.51s)
=== FAIL: amd64.integration.networking TestRestartUserlandProxyUnder2MSL (1.10s)
    port_mapping_linux_test.go:621: assertion failed: error is not nil: Get "http://127.0.0.1:1780": read tcp 127.0.0.1:47610->127.0.0.1:1780: read: connection reset by peer
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromAnotherNetwork/IPv4/Gateway (7.10s)
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:903: assertion failed: string "" does not contain "foobar"
    --- FAIL: TestAccessPublishedPortFromAnotherNetwork/IPv4/Gateway (7.10s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromAnotherNetwork/IPv6/Gateway (7.07s)
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:903: assertion failed: string "" does not contain "foobar"
    --- FAIL: TestAccessPublishedPortFromAnotherNetwork/IPv6/Gateway (7.07s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromAnotherNetwork/IPv4/Host (7.05s)
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:903: assertion failed: string "" does not contain "foobar"
    --- FAIL: TestAccessPublishedPortFromAnotherNetwork/IPv4/Host (7.05s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromAnotherNetwork/IPv6/Host (7.05s)
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:899: Retrying...
    port_mapping_linux_test.go:903: assertion failed: string "" does not contain "foobar"
    --- FAIL: TestAccessPublishedPortFromAnotherNetwork/IPv6/Host (7.05s)
=== FAIL: amd64.integration.networking TestAccessPublishedPortFromAnotherNetwork (29.23s)

Rootless

https://github.com/moby/moby/actions/runs/14537211827/job/40787925685?pr=49819

=== Failed
=== FAIL: amd64.integration.container TestNetworkLocalhostTCPNat (0.38s)
    nat_test.go:78: assertion failed: hi yall (msg string) !=  (string)

AkihiroSuda · 2025-04-18T21:36:13Z

+          # On almalinux-8, port forwarding tests are failing:
+          # https://github.com/moby/moby/pull/49819#issuecomment-2815676000
+          - template://oraclelinux-8 # Oracle's kernel 5.15
+          # - template://almalinux-8 # kernel 4.18


Let me just use oraclelinux-8 (kernel 5.15) and call it a day.

The failures on almalinux-8 (kernel 4.18) have to be analyzed though.

kolyshkin · 2025-04-18T23:50:05Z

Are we against using cirrus-ci here?

AkihiroSuda · 2025-04-19T00:22:25Z

Are we against using cirrus-ci here?

No strong objection, but it is somewhat cumbersome to maintain two different CIs, and the GCP cost might not be negligible

thaJeztah · 2025-05-15T14:58:20Z

I looked at this before, and SGTM, but could use some eyes from @crazy-max @vvoland if they have time

vvoland · 2025-05-16T15:37:03Z

The only concern I have is that it takes around 30 minutes while for these tests to execute:

Wondering if we should only limit this workflow to only run cgroup v1 specific tests?

AkihiroSuda · 2025-05-17T09:44:18Z

Wondering if we should only limit this workflow to only run cgroup v1 specific tests?

Doesn't seem easy to identify such tests

vvoland · 2025-05-19T12:28:06Z

Isn't it mostly for tests like

moby/integration/container/kill_test.go

Line 152 in 7937f08

    
           skip.If(t, testEnv.DaemonInfo.CgroupVersion == "2", "FIXME: flaky on cgroup v2 (https://github.com/moby/moby/issues/41929)")

(which are skipped on non v1)?

AkihiroSuda · 2025-05-20T18:41:33Z

There could be other tests that implicitly depend on cgroups

AkihiroSuda · 2025-08-27T08:44:44Z

Rebased

AkihiroSuda · 2025-08-28T11:44:09Z

CI is now green again

@thaJeztah @vvoland PTAL 🙏

thaJeztah

I'm OK with adding this; or at least, it makes sure we have some coverage for v1; we'd discover soon enough if adding the extra checks becomes problematic to run on every PR.

LGTM

but @vvoland @robmry @crazy-max for a second review ❤️

Signed-off-by: Akihiro Suda <[email protected]>

Used for rootless on kernel < 5.11 Signed-off-by: Akihiro Suda <[email protected]>

Lima is used for running a VM of `template://oraclelinux-8`. My initial attempt was to use almalinux-8, but some port forwarding tests do not seem to work on almalinux-8. https://lima-vm.io/docs/examples/gha/ Fix issue 49576 Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the cgroup1 branch from 1125f98 to e23fe6c Compare April 16, 2025 11:00

AkihiroSuda added the area/testing label Apr 16, 2025

AkihiroSuda force-pushed the cgroup1 branch from e23fe6c to 1876f17 Compare April 16, 2025 11:28

thaJeztah reviewed Apr 16, 2025

View reviewed changes

Comment thread .github/workflows/.vm.yml Outdated

AkihiroSuda force-pushed the cgroup1 branch 3 times, most recently from 3faee07 to 6910276 Compare April 16, 2025 12:50

This comment was marked as resolved.

Sign in to view

AkihiroSuda force-pushed the cgroup1 branch 3 times, most recently from a12bd51 to 140c4fe Compare April 17, 2025 00:25

AkihiroSuda marked this pull request as draft April 17, 2025 00:25

AkihiroSuda force-pushed the cgroup1 branch 7 times, most recently from 885f404 to 8f9f099 Compare April 18, 2025 15:07

AkihiroSuda force-pushed the cgroup1 branch 4 times, most recently from 5653897 to 03da18f Compare April 18, 2025 21:34

AkihiroSuda changed the title ~~CI: add almalinux-8 for running tests with cgroup v1~~ CI: add oraclelinux-8 for running tests with cgroup v1 Apr 18, 2025

AkihiroSuda commented Apr 18, 2025

View reviewed changes

AkihiroSuda marked this pull request as ready for review April 18, 2025 21:36

AkihiroSuda requested a review from tianon as a code owner April 18, 2025 21:36

thaJeztah reviewed May 15, 2025

View reviewed changes

Comment thread .github/workflows/.vm.yml Outdated

AkihiroSuda force-pushed the cgroup1 branch from 03da18f to 023bfa1 Compare May 15, 2025 15:01

AkihiroSuda requested a review from thaJeztah May 27, 2025 13:55

AkihiroSuda requested a review from tonistiigi June 20, 2025 03:43

AkihiroSuda requested a review from dmcgowan July 3, 2025 03:44

AkihiroSuda force-pushed the cgroup1 branch from 023bfa1 to e28070f Compare August 27, 2025 08:44

This comment was marked as resolved.

Sign in to view

AkihiroSuda force-pushed the cgroup1 branch 2 times, most recently from c5365db to 5cb3304 Compare August 28, 2025 10:21

AkihiroSuda added this to the 29.0.0 milestone Aug 28, 2025

thaJeztah approved these changes Aug 28, 2025

View reviewed changes

AkihiroSuda added the process/cherry-pick label Aug 29, 2025

vvoland reviewed Aug 29, 2025

View reviewed changes

Comment thread .github/workflows/.vm.yml

Comment thread .github/workflows/.vm.yml Outdated

AkihiroSuda added 3 commits August 29, 2025 19:02

Makefile: propagate DOCKER_IGNORE_BR_NETFILTER_ERROR

e15c51a

Signed-off-by: Akihiro Suda <[email protected]>

Dockerfile: install fuse-overlayfs

1570ca9

Used for rootless on kernel < 5.11 Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the cgroup1 branch from 5cb3304 to 09ecd74 Compare August 29, 2025 10:04

vvoland approved these changes Aug 29, 2025

View reviewed changes

vvoland added the area/ci label Aug 29, 2025

thaJeztah added the status/4-merge label Aug 29, 2025

AkihiroSuda merged commit b98046f into moby:master Aug 29, 2025
275 of 277 checks passed

AkihiroSuda mentioned this pull request Aug 29, 2025

TestAccessPublishedPort* (rootful) and TestNetworkLocalhostTCPNat (rootless) are failing on almalinux-8 #50849

Open

thaJeztah mentioned this pull request Sep 4, 2025

gha: skip "vm" checks if ci/validate-only label is set #50900

Merged

Conversation

AkihiroSuda commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

AkihiroSuda commented Apr 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Failures on almalinux-8

Rootful

Rootless

Uh oh!

AkihiroSuda Apr 18, 2025

Choose a reason for hiding this comment

Uh oh!

kolyshkin commented Apr 18, 2025

Uh oh!

AkihiroSuda commented Apr 19, 2025

Uh oh!

Uh oh!

thaJeztah commented May 15, 2025

Uh oh!

vvoland commented May 16, 2025

Uh oh!

AkihiroSuda commented May 17, 2025

Uh oh!

vvoland commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented May 20, 2025

Uh oh!

AkihiroSuda commented Aug 27, 2025

Uh oh!

This comment was marked as resolved.

AkihiroSuda commented Aug 28, 2025

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AkihiroSuda commented Apr 16, 2025 •

edited

Loading

AkihiroSuda commented Apr 18, 2025 •

edited

Loading

vvoland commented May 19, 2025 •

edited

Loading