Skip to content

Comments

pin: make WalkDir harder to misuse and add Windows support#1736

Merged
lmb merged 1 commit intocilium:mainfrom
lmb:windows-pin
Apr 16, 2025
Merged

pin: make WalkDir harder to misuse and add Windows support#1736
lmb merged 1 commit intocilium:mainfrom
lmb:windows-pin

Conversation

@lmb
Copy link
Contributor

@lmb lmb commented Mar 31, 2025

The current WalkDir API requires the user to close each object that is iterated over. This isn't very intuitive, the two uses of the function in the tetragon code base end up leaking objects. It's also not straight forward to implement WalkDir on Windows, since BPFFS isn't really a thing on that platform.

Instead, turn WalkDir into an iterator which returns a new Pin object. By default, objects are closed once they have been iterated over. The user may call Pin.Take if they wish to retain the object. This is similar to the existing link.Iterator.Take.

One downside is that directories are not returned by the iteration anymore. The existing code in tetragon doesn't use them and is therefore not affected. There is also no more fine grained control over whether to descent into a directory. Aborting an iteration is still supported and as efficient as before.

Fixes: #1652

@lmb
Copy link
Contributor Author

lmb commented Mar 31, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

@mtardy
Copy link
Member

mtardy commented Apr 7, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

will take a look shortly 👀 , was busy conferencing...

@mtardy
Copy link
Member

mtardy commented Apr 14, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

will take a look shortly 👀 , was busy conferencing...

Looks good, am I using it correctly cilium/tetragon@38c1109 ?

@lmb
Copy link
Contributor Author

lmb commented Apr 14, 2025

Yeah those changes look good! You can pass nil options to WalkDir I think. I wonder whether I should drop the return value from Take() for now, not sure it's particularly useful.

@mtardy
Copy link
Member

mtardy commented Apr 14, 2025

Yeah those changes look good! You can pass nil options to WalkDir I think.

ah yeah it seems ReadOnly: true doesn't work well with links

I wonder whether I should drop the return value from Take() for now, not sure it's particularly useful.

ah yeah maybe indeed

@lmb lmb marked this pull request as ready for review April 14, 2025 12:00
@lmb lmb requested a review from a team as a code owner April 14, 2025 12:00
The current WalkDir API requires the user to close each object that
is iterated over. This isn't very intuitive, the two uses of the
function in the tetragon code base end up leaking objects.
It's also not straight forward to implement WalkDir on Windows, since
BPFFS isn't really a thing on that platform.

Instead, turn WalkDir into an iterator which returns a new Pin object.
By default, objects are closed once they have been iterated over. The
user may call Pin.Take if they wish to retain the object. This is
similar to the existing link.Iterator.Take.

One downside is that directories are not returned by the iteration
anymore. The existing code in tetragon doesn't use them and is
therefore not affected. There is also no more fine grained control
over whether to descent into a directory. Aborting an iteration is
still supported and as efficient as before.

Fixes: cilium#1652
Signed-off-by: Lorenz Bauer <[email protected]>
Copy link
Member

@mtardy mtardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this new version when using the iter with existing for := range.

@lmb lmb merged commit ae10986 into cilium:main Apr 16, 2025
17 checks passed
@lmb lmb deleted the windows-pin branch April 16, 2025 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pin: WalkDir makes it too easy to leak objects

2 participants