[WIP] testing buildkit 0.12 fixes by thaJeztah · Pull Request #46035 · moby/moby

thaJeztah · 2023-07-20T09:18:36Z

quick attempt to fix Update to BuildKit 0.12 #45966 (comment)

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Bjorn Neergaard <[email protected]>

This is a bit ugly, but it works given the new codegen output. Signed-off-by: Bjorn Neergaard <[email protected]>

Introduced in containerd/containerd@dd3eedf Signed-off-by: Bjorn Neergaard <[email protected]>

Signed-off-by: Bjorn Neergaard <[email protected]>

ref was made a return value in moby/buildkit@330cf7a Signed-off-by: Bjorn Neergaard <[email protected]>

We still use interface{} for now to reduce code churn. flightcontrol.Group was made generic in moby/buildkit@8ffc03b Signed-off-by: Bjorn Neergaard <[email protected]>

This is a horrible kludge, but not /the worst/ as a reference to the underlying `osl.Sandbox` is part of the `Sandbox`; so keeping a reference to the underlying `osl.Interface` in the `Endpoint` is about the same level of impure. The lifecycle and assignment here is bad, but that can be fixed during future refactors, and keeps this diff minimal. Signed-off-by: Bjorn Neergaard <[email protected]>

This method was added to the interface in moby/buildkit@963f161 Signed-off-by: Bjorn Neergaard <[email protected]>

Introduced in moby/buildkit@6e87e4b Signed-off-by: Bjorn Neergaard <[email protected]>

The controller moved to CacheManager in moby/buildkit@2b30693 Signed-off-by: Bjorn Neergaard <[email protected]>

Duration is a deserializing wrapper introduced in moby/buildkit@e06c962 Signed-off-by: Bjorn Neergaard <[email protected]>

…tore The concrete types are used instead of the interface after moby/buildkit@f044e0a Signed-off-by: Bjorn Neergaard <[email protected]>

Introduced years ago in moby/buildkit@6644e1b Signed-off-by: Bjorn Neergaard <[email protected]>

Introduced in moby/buildkit@4fc2d7b Signed-off-by: Bjorn Neergaard <[email protected]>

thaJeztah · 2023-07-20T09:19:22Z

vendor validation will fail on this one, as I manually modified the vendored files

thaJeztah · 2023-07-20T10:12:49Z

Linter found some issues;

libcontainerd/remote/client.go:212:13: copylocks: assignment copies lock value to opts: github.com/docker/docker/vendor/github.com/containerd/containerd/runtime/v2/runc/options.Options contains github.com/docker/docker/vendor/google.golang.org/protobuf/internal/impl.MessageState contains sync.Mutex (govet)
				opts := *c.v2runcoptions
				        ^
daemon/runtime_unix_test.go:246:17: copylocks: assignment copies lock value to configdOpts: github.com/docker/docker/vendor/github.com/containerd/containerd/runtime/v2/runc/options.Options contains github.com/docker/docker/vendor/google.golang.org/protobuf/internal/impl.MessageState contains sync.Mutex (govet)
	configdOpts := *stockRuntime.Opts.(*v2runcoptions.Options)
	               ^
builder/builder-next/adapters/containerimage/pull.go:22:2: SA1019: "github.com/containerd/containerd/remotes/docker/schema1" is deprecated: use images formatted in Docker Image Manifest v2, Schema 2, or OCI Image Spec v1. (staticcheck)
	"github.com/containerd/containerd/remotes/docker/schema1"
	^

thaJeztah · 2023-07-20T10:54:46Z

Two flaky test failures;

=== Failed
=== FAIL: github.com/docker/docker/integration/container TestExecWithCloseStdin (6.03s)
    exec_test.go:77: failed to read the content in time

=== FAIL: github.com/docker/docker/integration/container TestRenameStoppedContainer (13.11s)
    rename_test.go:58: timeout hit after 10s: waiting for container to be one of (exited), currently running

Some failures on Windows that I THINK are unrelated, but I'll kick those again after

=== Failed
=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/0_config:_{volume__c:\foo_false__<nil>_<nil>_<nil>_<nil>} (12.30s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container d8899eaa18c06679eb8f5d836861b00ce887613d7e29b38a5c79a354f45c4c09 is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/1_config:_{volume__c:\foo\_false__<nil>_<nil>_<nil>_<nil>} (12.18s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container 1f98febef0ed3bba3f2c8cc964dfd56e7af160e3b307c4f90716aaf36b634dd1 is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/2_config:_{volume_test1_c:\foo_false__<nil>_<nil>_<nil>_<nil>} (12.13s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container 2e22f8b3d0229272ca4a581872afaa1f895e88ab681cdf9d2c947db5971dbbd7 is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/3_config:_{volume_test2_c:\foo_true__<nil>_<nil>_<nil>_<nil>} (12.29s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container f7bbe4731a3b4b9e886c8bf256492305c71e91152b228e6fe5fed212dd2fba89 is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/4_config:_{volume_test3_c:\foo_false__<nil>_0xc000cb4018_<nil>_<nil>} (12.15s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container 39cd582a0cbeeaba949d0b92c42bea47660a74a6d4d85286abbe06f40b2fdb2c is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/5_config:_{bind_C:\Users\RUNNER~1\AppData\Local\Temp\test-mounts-api-13699587998_c:\foo_false__<nil>_<nil>_<nil>_<nil>} (12.21s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container d599fca52be8409f25433e5d64957e39a97cc61cd528586e3dd476aade4fd57b is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate/6_config:_{bind_C:\Users\RUNNER~1\AppData\Local\Temp\test-mounts-api-13699587998_c:\foo_true__<nil>_<nil>_<nil>_<nil>} (13.08s)
    docker_api_containers_test.go:2157: timeout hit after 10s: container 67863418730b3320bb297b492d79b5d66a3d00109be881811bf3f97dc0fb571d is running, waiting for exit

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite/TestContainersAPICreateMountsCreate (89.19s)

=== FAIL: github.com/docker/docker/integration-cli TestDockerAPISuite (857.05s)

Not sure what's happening with docker-py would have to check;

---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary /usr/local/cli-integration/docker
# DOCKER_EXPERIMENTAL is set: starting daemon with experimental features enabled! 
Not: starting.
Starting dockerd
INFO: Waiting for daemon to start...
.
---> Making bundle: .build-empty-images (in bundles/test-docker-py)
Loaded image: emptyfs:latest
Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
INFO: Building docker-sdk-python3:5.0.3...
ERROR: failed to solve: Unavailable: error reading from server: EOF
make: *** [Makefile:198: test-docker-py] Error 1
Error: Process completed with exit code 2.

thaJeztah · 2023-07-20T10:56:15Z

Some buildkit CI failures;

=== FAIL: frontend TestFrontendIntegration/TestRefEvaluate/worker=dockerd (1.27s)
    frontend_test.go:341: 
        	Error Trace:	/src/frontend/frontend_test.go:341
        	            				/src/util/testutil/integration/run.go:90
        	            				/src/util/testutil/integration/run.go:204
        	Error:      	Received unexpected error:
        	            	lease does not exist: not found

Signed-off-by: Sebastiaan van Stijn <[email protected]>

neersighted · 2023-07-24T18:05:54Z

Closing as the relevant changes are in BK/the original PR now.

neersighted added 15 commits July 20, 2023 11:01

vendor: github.com/containerd/containerd v1.7.2

4e42694

Signed-off-by: Bjorn Neergaard <[email protected]>

libcontainerd/remote: adjust for API type changes in containerd v1.7

1dcfab6

Signed-off-by: Bjorn Neergaard <[email protected]>

daemon: adapt tests to containerd v1.7 non-gogo protobufs

71630ba

This is a bit ugly, but it works given the new codegen output. Signed-off-by: Bjorn Neergaard <[email protected]>

daemon/c8d: use new containerd LabelDistributionSource constant

d304197

Introduced in containerd/containerd@dd3eedf Signed-off-by: Bjorn Neergaard <[email protected]>

vendor: github.com/moby/buildkit v0.12.0

432c7f3

Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: plumb through ref during resolve

81dc58a

ref was made a return value in moby/buildkit@330cf7a Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: adjust for flightcontrol.Group becoming a generic type

c6d26f4

We still use interface{} for now to reduce code churn. flightcontrol.Group was made generic in moby/buildkit@8ffc03b Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: implement util/network/Namespace.Sample()

1264c07

This method was added to the interface in moby/buildkit@963f161 Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: update Windows stub executor for resourcetypes.Recorder

cf58128

Introduced in moby/buildkit@6e87e4b Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: use CacheManager instead of CacheKeyStorage

6d9482c

The controller moved to CacheManager in moby/buildkit@2b30693 Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: implement MaxAge as buildkit config.Duration

e9e8ee6

Duration is a deserializing wrapper introduced in moby/buildkit@e06c962 Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next: adapt to enforced wrapping of LeaseManager and ContentS…

f4b205b

…tore The concrete types are used instead of the interface after moby/buildkit@f044e0a Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next/pull: use ResolveMode.String() helper

a5c1819

Introduced years ago in moby/buildkit@6644e1b Signed-off-by: Bjorn Neergaard <[email protected]>

builder-next/wrapper: use new exptypes constants

3aaf063

Introduced in moby/buildkit@4fc2d7b Signed-off-by: Bjorn Neergaard <[email protected]>

thaJeztah force-pushed the buildkit_0.12_carry branch from 5b3988a to ef8e58d Compare July 20, 2023 09:33

thaJeztah mentioned this pull request Jul 20, 2023

executor/resource: stub out NewSysSampler on Windows moby/buildkit#4040

Merged

thaJeztah mentioned this pull request Jul 20, 2023

[0.12 backport] executor/resource: stub out NewSysSampler on Windows moby/buildkit#4042

Merged

WIP: fix buildkit

460e100

Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah force-pushed the buildkit_0.12_carry branch from ef8e58d to 460e100 Compare July 20, 2023 12:35

neersighted closed this Jul 24, 2023

thaJeztah deleted the buildkit_0.12_carry branch July 24, 2023 18:07

neersighted mentioned this pull request Sep 13, 2023

Update to BuildKit 0.12 #45966

Merged

thaJeztah mentioned this pull request Feb 3, 2025

Flaky test (on Windows): TestContainersAPICreateMountsCreate #49382

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] testing buildkit 0.12 fixes#46035

[WIP] testing buildkit 0.12 fixes#46035
thaJeztah wants to merge 16 commits intomoby:masterfrom
thaJeztah:buildkit_0.12_carry

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

neersighted commented Jul 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

thaJeztah commented Jul 20, 2023

Uh oh!

neersighted commented Jul 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants