Skip to content

supervisor: fix large opaque data handle#613

Merged
changweige merged 1 commit intocontainerd:mainfrom
imeoer:fix-supervisor
Sep 2, 2024
Merged

supervisor: fix large opaque data handle#613
changweige merged 1 commit intocontainerd:mainfrom
imeoer:fix-supervisor

Conversation

@imeoer
Copy link
Copy Markdown
Collaborator

@imeoer imeoer commented Aug 16, 2024

When the supervisor needs to process a large amount of opaque data (for example,
a nydusd instance has a large number of sub mounts, and the opaque is usually
larger than 32kb), a recvmsg / sendmsg system call cannot process all data.

The commit fixes the problem with multiple recvmsg/sendmsg calls and updates
the test cases.

@imeoer imeoer requested a review from changweige August 16, 2024 03:41
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 16, 2024

Codecov Report

Attention: Patch coverage is 78.08219% with 16 lines in your changes missing coverage. Please review.

Project coverage is 22.07%. Comparing base (e22e77d) to head (4f4dbcd).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
pkg/supervisor/supervisor.go 78.08% 9 Missing and 7 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #613      +/-   ##
==========================================
+ Coverage   21.82%   22.07%   +0.24%     
==========================================
  Files         122      122              
  Lines       10730    10746      +16     
==========================================
+ Hits         2342     2372      +30     
+ Misses       8070     8054      -16     
- Partials      318      320       +2     
Files with missing lines Coverage Δ
pkg/supervisor/supervisor.go 68.91% <78.08%> (+10.71%) ⬆️


func (mss *MemStatesStorage) Write(data []byte) {
l := copy(mss.data[mss.head:], data)
mss.head += l
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without head, States storage can't support append-write

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, we haven't needed append yet. :)

@imeoer imeoer force-pushed the fix-supervisor branch 4 times, most recently from 306cf98 to 2101000 Compare August 19, 2024 07:24
When the supervisor needs to process a large amount of opaque data (for example,
a nydusd instance has a large number of sub mounts, and the opaque is usually
larger than 32kb), a recvmsg / sendmsg system call cannot process all data.

The commit fixes the problem with multiple recvmsg/sendmsg calls and updates
the test cases.

Signed-off-by: Yan Song <[email protected]>
return nil, 0, err
}

return data, su.fd, nil
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check if fd is positive?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be ok, but maybe we don't have to limit it.

})

eg.Go(func() error {
err := receiver()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why trigger and receiver have to run in parallel in separted goroutines?

@changweige changweige merged commit 30f3d65 into containerd:main Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants