Skip to content

imageservice: Add context to various methods#44365

Merged
thaJeztah merged 4 commits intomoby:masterfrom
vvoland:c8d-contexts
Nov 3, 2022
Merged

imageservice: Add context to various methods#44365
thaJeztah merged 4 commits intomoby:masterfrom
vvoland:c8d-contexts

Conversation

@vvoland
Copy link
Copy Markdown
Contributor

@vvoland vvoland commented Oct 27, 2022

Replaces:

- What I did
I tracked down the integration test hang and reverted the changes which caused it.
I will be doing some further investigations to find out exactly why passing the given context causes this - but for now I propose to keep the existing behaviour of passing context.TODO to container.Run as this will unblock us in upstreaming the containerd integration changes.

- How I did it
Bisecting changes from the original commit

- How to verify it
Check if integration tests don't timeout

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@vvoland vvoland marked this pull request as ready for review October 27, 2022 10:44
@vvoland vvoland requested a review from tonistiigi as a code owner October 27, 2022 10:44
@vvoland vvoland requested review from rumpl and thaJeztah October 27, 2022 10:45
@vvoland vvoland added status/2-code-review area/images Image Service containerd-integration Issues and PRs related to containerd integration labels Oct 27, 2022
Copy link
Copy Markdown
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being a context plumber ain't easy, that's for sure!

Comment thread builder/dockerfile/evaluator_test.go Outdated
Comment thread daemon/images/cache.go Outdated
Comment thread builder/dockerfile/imageprobe.go
Comment thread builder/dockerfile/builder.go
Comment thread builder/dockerfile/dispatchers.go
Comment thread daemon/disk_usage.go
Comment on lines 15 to 20
ch := daemon.usage.DoChan("ContainerDiskUsage", func() (interface{}, error) {
// Retrieve container list
containers, err := daemon.Containers(&types.ContainerListOptions{
containers, err := daemon.Containers(context.TODO(), &types.ContainerListOptions{
Size: true,
All: true,
})
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package resenje.org/singleflight might be of help here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, look good. Do you mind addressing this in a follow up? The behaviour here doesn't change, and I would like to avoid blowing up this PR with a new package being vendored.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM! 👍

Comment thread daemon/oci_linux.go
Comment thread daemon/start.go Outdated
Comment thread builder/dockerfile/builder.go
Comment thread builder/dockerfile/builder.go
@vvoland
Copy link
Copy Markdown
Contributor Author

vvoland commented Oct 28, 2022

Thanks for a great analysis @corhere, I addressed your comments.
I ripped out the builder.clientCtx completely as it didn't seem to be used much and there's no need to hold it in struct.

Copy link
Copy Markdown
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great! There's just one place where error-matching needs to be fixed up to handle wrapped errors.

Comment thread builder/dockerfile/dispatchers_test.go Outdated
Comment thread builder/dockerfile/dispatchers_test.go Outdated
Comment thread builder/dockerfile/dispatchers_test.go Outdated
Comment thread daemon/images/cache.go Outdated
Comment thread daemon/start.go Outdated
Comment thread builder/dockerfile/builder.go Outdated
Copy link
Copy Markdown
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@vvoland
Copy link
Copy Markdown
Contributor Author

vvoland commented Oct 31, 2022

Thanks! I just squashed the fix-up commits.

Comment thread daemon/start.go
// TODO(mlaventure): we need to specify checkpoint options here
tsk, err := ctr.Start(ctx, checkpointDir,
container.StreamConfig.Stdin() != nil || container.Config.Tty,
tsk, err := ctr.Start(context.TODO(), // Passing ctx to ctr.Start caused integration tests to be stuck in the cleanup phase
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😞

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this one was added in 4bafaa0#diff-034d55db8b341bd127a5394a84226b04c91d54669dd856051ceb0bcbbb50b380 (#43564)

Perhaps we should create a tracking ticket for this one to see if we can find out what's causing the actual issue.

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes look good; left some comments about commits that should ideally be squashed (let me know if you need help on that)

Comment thread daemon/start.go
// TODO(mlaventure): we need to specify checkpoint options here
tsk, err := ctr.Start(ctx, checkpointDir,
container.StreamConfig.Stdin() != nil || container.Config.Tty,
tsk, err := ctr.Start(context.TODO(), // Passing ctx to ctr.Start caused integration tests to be stuck in the cleanup phase
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this one was added in 4bafaa0#diff-034d55db8b341bd127a5394a84226b04c91d54669dd856051ceb0bcbbb50b380 (#43564)

Perhaps we should create a tracking ticket for this one to see if we can find out what's causing the actual issue.

Comment thread builder/dockerfile/dispatchers_test.go Outdated
Comment thread daemon/disk_usage.go
Comment thread builder/dockerfile/evaluator_test.go Outdated
Comment thread builder/builder.go
Comment thread daemon/daemon.go
Comment thread daemon/daemon.go
Comment thread daemon/oci_linux.go
Comment thread builder/dockerfile/builder.go
Comment thread builder/dockerfile/builder.go Outdated
ndeloof and others added 3 commits November 3, 2022 12:22
Co-authored-by: Paweł Gronowski <[email protected]>
Signed-off-by: Paweł Gronowski <[email protected]>
This caused integration tests to timeout in the CI

Signed-off-by: Paweł Gronowski <[email protected]>
@vvoland
Copy link
Copy Markdown
Contributor Author

vvoland commented Nov 3, 2022

Thanks @thaJeztah. Squashed most of the commits 😄

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/images Image Service containerd-integration Issues and PRs related to containerd integration status/4-merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants