Skip to content

driver: only initialize the client once in the docker and remote drivers#2187

Closed
jsternberg wants to merge 1 commit intodocker:masterfrom
jsternberg:single-driver-client
Closed

driver: only initialize the client once in the docker and remote drivers#2187
jsternberg wants to merge 1 commit intodocker:masterfrom
jsternberg:single-driver-client

Conversation

@jsternberg
Copy link
Copy Markdown
Collaborator

The docker and remote drivers both use the client internally and
therefore re-initialize the client multiple times within themselves. We
do some work to ensure the client is only initialized once through the
driver handle, but these drivers end up initializing the client multiple
times erroneously.

This updates these drivers to have their own caching code for the client
to prevent reinitialization which also has the side effect of fixing a
bug where traces were duplicated and sent multiple times.

The docker and remote drivers both use the client internally and
therefore re-initialize the client multiple times within themselves. We
do some work to ensure the client is only initialized once through the
driver handle, but these drivers end up initializing the client multiple
times erroneously.

This updates these drivers to have their own caching code for the client
to prevent reinitialization which also has the side effect of fixing a
bug where traces were duplicated and sent multiple times.

Signed-off-by: Jonathan A. Sternberg <[email protected]>
@jsternberg jsternberg requested a review from tonistiigi January 11, 2024 21:18
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failures seem to be related. We would need some reference counting and possibly protection for context cancellation.

@jsternberg
Copy link
Copy Markdown
Collaborator Author

I'm going to try to fix this on the client side, but I suspect that this will probably be easier to fix in two different ways:

  1. Fix the underlying tracing issue where each time a client is initialized it adds it to the global list of tracers.
  2. Perform the dedup on the buildkit side. Since it's typically the same client sending the traces and they're likely very close to each other, it would be pretty easy to perform the dedup in a memory-conscious way.

Although it might be as simple as just add reference counting to the close call. I guess I'll see after I try that.

@jsternberg
Copy link
Copy Markdown
Collaborator Author

Going to close this because I don't think this PR works properly and it's no longer the way that I think this should be done. #2362 is a better solution that should be easier to refactor and maintain.

@jsternberg jsternberg closed this Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants