revendor BuildKit (master branch) by thaJeztah · Pull Request #42473 · moby/moby

thaJeztah · 2021-06-05T20:00:48Z

relates to #42471

depends on:

Updates:

vendor: golang.org/x/crypto 0c34fe9e7dc2486962ef9867e3edb3503537209f
full diff: golang/crypto@c1f2f97...0c34fe9
vendor: golang.org/x/time 3af7569d3a1e776fc2a3c1cec133b43105ea9c2e
full diff: golang/time@555d28b...3af7569
vendor: golang.org/x/net e18ecbb051101a46fc263334b127c89bc7bff7ea
full diff: golang/net@6772e93...e18ecbb
vendor: golang.org/x/sync 036812b2e83c0ddf193dd5a34e034151da389d09
full diff: golang/sync@6e8e738...036812b
vendor: google.golang.org/genproto
full diff: googleapis/go-genproto@3f1135a...8816d57
vendor: github.com/tonistiigi/fsutil 5dfbf5db66b9c8fb7befcab1722381de99ec0430
full diff: tonistiigi/fsutil@0834f99...d72af97

thaJeztah · 2021-06-07T07:01:41Z

Missing a vendor

[2021-06-05T20:01:38.922Z] vendor/google.golang.org/genproto/googleapis/rpc/status/status.pb.go:28:2: cannot find package "google.golang.org/protobuf/reflect/protoreflect" in any of:
[2021-06-05T20:01:38.922Z] 	/go/src/github.com/docker/docker/vendor/google.golang.org/protobuf/reflect/protoreflect (vendor tree)
[2021-06-05T20:01:38.922Z] 	/usr/local/go/src/google.golang.org/protobuf/reflect/protoreflect (from $GOROOT)
[2021-06-05T20:01:38.922Z] 	/go/src/google.golang.org/protobuf/reflect/protoreflect (from $GOPATH)

thaJeztah · 2021-06-07T07:30:45Z

dropping the genproto bump for now (see if it works without that bump) and moved that to #42475

thaJeztah · 2021-06-07T07:34:59Z

Local changes needed;


[2021-06-07T07:31:47.776Z] # github.com/docker/docker/builder/builder-next/worker
[2021-06-07T07:31:47.776Z] builder/builder-next/worker/worker.go:194:26: not enough arguments in call to ops.NewSourceOp
[2021-06-07T07:31:47.776Z] 	have (solver.Vertex, *pb.Op_Source, *pb.Platform, *source.Manager, *session.Manager, *Worker)
[2021-06-07T07:31:47.776Z] 	want (solver.Vertex, *pb.Op_Source, *pb.Platform, *source.Manager, *semaphore.Weighted, *session.Manager, worker.Worker)
[2021-06-07T07:31:47.776Z] builder/builder-next/worker/worker.go:196:24: not enough arguments in call to ops.NewExecOp
[2021-06-07T07:31:47.776Z] 	have (solver.Vertex, *pb.Op_Exec, *pb.Platform, cache.Manager, *session.Manager, *"github.com/docker/docker/vendor/github.com/moby/buildkit/cache/metadata".Store, executor.Executor, *Worker)
[2021-06-07T07:31:47.776Z] 	want (solver.Vertex, *pb.Op_Exec, *pb.Platform, cache.Manager, *semaphore.Weighted, *session.Manager, *"github.com/docker/docker/vendor/github.com/moby/buildkit/cache/metadata".Store, executor.Executor, worker.Worker)
[2021-06-07T07:31:47.776Z] builder/builder-next/worker/worker.go:198:24: not enough arguments in call to ops.NewFileOp
[2021-06-07T07:31:47.776Z] 	have (solver.Vertex, *pb.Op_File, cache.Manager, *"github.com/docker/docker/vendor/github.com/moby/buildkit/cache/metadata".Store, *Worker)
[2021-06-07T07:31:47.776Z] 	want (solver.Vertex, *pb.Op_File, cache.Manager, *semaphore.Weighted, *"github.com/docker/docker/vendor/github.com/moby/buildkit/cache/metadata".Store, worker.Worker)
script returned exit code 2

thaJeztah · 2021-06-07T10:57:43Z

builder/builder-next/worker/worker.go

@tonistiigi @AkihiroSuda I could use some help here; this relates to moby/buildkit#2049. Is there any value I should pass here? If so; where do I get this from?

Should be safe to ignore and pass nil. Later you may connect this to dockerd configuration option.

You can just pass nil though, don't need a var.

thaJeztah · 2021-06-07T16:38:56Z

These failures look related. Sounds like the test is not setting all options that the code expects;

=== Failed
=== FAIL: builder/dockerfile TestRunIgnoresHealthcheck (0.00s)
    dispatchers_test.go:571: assertion failed: error is not nil: no mount state

=== FAIL: builder/dockerfile TestDispatchUnsupportedOptions/RUN_with_unsupported_options (0.00s)
    --- FAIL: TestDispatchUnsupportedOptions/RUN_with_unsupported_options (0.00s)
        dispatchers_test.go:607: assertion failed: expected error "the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled", got "no mount state"
            no mount state

=== FAIL: builder/dockerfile TestDispatchUnsupportedOptions (0.00s)

=== FAIL: libnetwork/iptables TestExistsRaw (0.06s)
    iptables_test.go:288: Failed to detect rule. i=3

thaJeztah · 2021-06-07T19:22:57Z

Looks related to moby/buildkit#2089

Looks like this is impossible to test currently, or at least I don't know how;

failure is in runMountPostHook(), which calls getMountState(); if that returns nil, it produces an error
getMountState() calls cmd.getExternalValue(mountsKey), which returns cmd.m["dockerfile/run/mounts"]
cmd.m is set by cmd.setExternalValue() as part of runMountPreHook()

All off these; both types, accessors, and properties are non-exported, so these tests are no longer able to mock things

thaJeztah · 2021-06-07T19:24:46Z

@tonistiigi I could use some help on this one; perhaps you have an idea how to fix this

full diff: golang/crypto@c1f2f97...0c34fe9 Signed-off-by: Sebastiaan van Stijn <[email protected]>

full diff: golang/time@555d28b...3af7569 Signed-off-by: Sebastiaan van Stijn <[email protected]>

full diff: golang/net@6772e93...e18ecbb Signed-off-by: Sebastiaan van Stijn <[email protected]>

full diff: golang/sync@6e8e738...036812b Signed-off-by: Sebastiaan van Stijn <[email protected]>

…23675b83 full diff: tonistiigi/fsutil@0834f99...d72af97 Signed-off-by: Sebastiaan van Stijn <[email protected]>

builder/dockerfile/copy.go

Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah · 2021-06-16T06:29:39Z

@tonistiigi @AkihiroSuda PTAL - this is ready for review now (and green)

thaJeztah added area/builder Build status/2-code-review area/builder/buildkit Build labels Jun 5, 2021

thaJeztah added this to the 21.xx milestone Jun 5, 2021

thaJeztah mentioned this pull request Jun 5, 2021

Post-libnetwork integration tasks #42471

Open

17 tasks

thaJeztah force-pushed the unfork_buildkit branch 2 times, most recently from 42d8063 to 701e18b Compare June 7, 2021 07:30

thaJeztah mentioned this pull request Jun 7, 2021

[WIP] vendor: google.golang.org/genproto, add google.golang.org/protobuf #42475

Closed

thaJeztah force-pushed the unfork_buildkit branch from 701e18b to f370426 Compare June 7, 2021 10:56

thaJeztah commented Jun 7, 2021

View reviewed changes

thaJeztah mentioned this pull request Jun 8, 2021

docs: remove note about buildkit not supporting git subdirectories docker/cli#3125

Merged

3 tasks

thaJeztah force-pushed the unfork_buildkit branch 6 times, most recently from 07e8364 to 9d2ac9c Compare June 10, 2021 19:31

thaJeztah added 5 commits June 15, 2021 16:20

vendor: golang.org/x/crypto 0c34fe9e7dc2486962ef9867e3edb3503537209f

28156f4

full diff: golang/crypto@c1f2f97...0c34fe9 Signed-off-by: Sebastiaan van Stijn <[email protected]>

vendor: golang.org/x/time 3af7569d3a1e776fc2a3c1cec133b43105ea9c2e

e1ae2d2

full diff: golang/time@555d28b...3af7569 Signed-off-by: Sebastiaan van Stijn <[email protected]>

vendor: golang.org/x/net e18ecbb051101a46fc263334b127c89bc7bff7ea

037c26d

full diff: golang/net@6772e93...e18ecbb Signed-off-by: Sebastiaan van Stijn <[email protected]>

vendor: golang.org/x/sync 036812b2e83c0ddf193dd5a34e034151da389d09

225f764

full diff: golang/sync@6e8e738...036812b Signed-off-by: Sebastiaan van Stijn <[email protected]>

vendor: github.com/tonistiigi/fsutil d72af97c0eaf93c1d20360e3cb9c63c2…

8733144

…23675b83 full diff: tonistiigi/fsutil@0834f99...d72af97 Signed-off-by: Sebastiaan van Stijn <[email protected]>

This comment has been minimized.

Sign in to view

thaJeztah force-pushed the unfork_buildkit branch from 9d2ac9c to 2e8aa59 Compare June 15, 2021 20:34

This comment has been minimized.

Sign in to view

thaJeztah commented Jun 15, 2021

View reviewed changes

builder/dockerfile/copy.go Outdated Show resolved Hide resolved

thaJeztah force-pushed the unfork_buildkit branch from 2e8aa59 to b100687 Compare June 15, 2021 21:14

thaJeztah mentioned this pull request Jun 15, 2021

Testing: rewrite TestRunIgnoresHealthcheck and TestDispatchUnsupportedOptions for latest buildkit code #42526

Closed

thaJeztah force-pushed the unfork_buildkit branch from b100687 to cdc3774 Compare June 15, 2021 23:14

thaJeztah changed the title ~~[WIP] revendor BuildKit (master branch)~~ revendor BuildKit (master branch) Jun 15, 2021

revendor BuildKit (master branch)

3eb1257

Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah force-pushed the unfork_buildkit branch from cdc3774 to 3eb1257 Compare June 15, 2021 23:18

thaJeztah marked this pull request as ready for review June 16, 2021 06:28

thaJeztah requested a review from tonistiigi as a code owner June 16, 2021 06:28

thaJeztah added the impact/changelog label Jun 16, 2021

tonistiigi approved these changes Jun 17, 2021

View reviewed changes

AkihiroSuda approved these changes Jun 17, 2021

View reviewed changes

AkihiroSuda merged commit 9e8cf10 into moby:master Jun 17, 2021

thaJeztah deleted the unfork_buildkit branch June 17, 2021 08:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

revendor BuildKit (master branch)#42473

revendor BuildKit (master branch)#42473
AkihiroSuda merged 6 commits intomoby:masterfrom
thaJeztah:unfork_buildkit

thaJeztah commented Jun 5, 2021 •

edited

Loading

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021 •

edited

Loading

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah Jun 7, 2021

Uh oh!

tonistiigi Jun 17, 2021

Uh oh!

tonistiigi Jun 17, 2021

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

thaJeztah commented Jun 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

thaJeztah commented Jun 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah Jun 7, 2021

Choose a reason for hiding this comment

Uh oh!

tonistiigi Jun 17, 2021

Choose a reason for hiding this comment

Uh oh!

tonistiigi Jun 17, 2021

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

thaJeztah commented Jun 7, 2021

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

thaJeztah commented Jun 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

thaJeztah commented Jun 5, 2021 •

edited

Loading

thaJeztah commented Jun 7, 2021 •

edited

Loading