Skip to content

Commit 0e303ee

Browse files
brandonkeungclaude
andcommitted
refactor: give the Python walk classifier the shape Go already has
Go decides how to treat each entry of a recursive grep in a named function with three named outcomes -- classifyWalkEntry returning walkEntryGrep, walkEntrySkip or walkEntryUnreadable. Python made the same decision inline, held together by comments that referenced the Go symbols by name: # matching the split Go makes in classifyWalkEntry: # ... as Go's walkEntryUnreadable does. # ... as Go's walkEntrySkip does. That is a cross-runtime contract enforced by English. Renaming the Go function would silently falsify it -- no test fails, nothing rebuilds. It matters here specifically because runtime drift is this feature's recurring defect: six of the ten findings in the last review were the two implementations having quietly diverged. Extract _classify_walk_entry with the same three outcomes, the same argument order and the same return shape, so the two can be diffed by reading them side by side rather than by trusting prose. The entry loop drops from 38 lines (25 of them comment, 65%) to 14 (3 comment, 21%), and the reasoning moves onto the function it actually describes. The docstring is explicit that the two are not branch-for-branch identical and should not be forced to be -- Go tests IsDir, EvalSymlinks, Stat and IsRegular separately because WalkDir hands it directories and unresolvable links, while os.walk yields only filenames and Path.is_file() collapses those cases into one call. It also records the one divergence we know of and could not construct: Go counts a failed Stat on a non-symlink as UNREADABLE, where Path.is_file() swallows the OSError and reaches SKIP. Behavior is unchanged. Each outcome is independently pinned, verified by inverting it and confirming the suite catches it: UNREADABLE -> SKIP (broken links uncounted) caught SKIP -> GREP (escaping symlinks searched) caught GREP -> SKIP (regular files never read) caught SKIP -> UNREADABLE (FIFOs counted) caught Adds a direct table test for the classifier, which the inline form could not have: the three-way split is now assertable without going through grep_content, including the symlink-loop case. Co-Authored-By: Claude Opus 5 <[email protected]> Signed-off-by: brandonkeung <[email protected]>
1 parent 97a23d8 commit 0e303ee

2 files changed

Lines changed: 120 additions & 30 deletions

File tree

‎python/packages/kagent-skills/src/kagent/skills/shell.py‎

Lines changed: 68 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
from __future__ import annotations
44

55
import asyncio
6+
import enum
67
import logging
78
import os
89
import re
@@ -27,6 +28,68 @@ def _truncate_line(line: str) -> str:
2728
return line
2829

2930

31+
class _WalkEntryAction(enum.Enum):
32+
"""How a recursive grep should treat one entry it walked."""
33+
34+
#: A regular, in-bounds file that should be grepped.
35+
GREP = enum.auto()
36+
#: Silently excluded by policy, not a read failure -- a non-regular file
37+
#: (FIFO/socket/device), or a symlink whose target escapes the search root.
38+
SKIP = enum.auto()
39+
#: A genuine read/stat failure on this entry.
40+
UNREADABLE = enum.auto()
41+
42+
43+
def _classify_walk_entry(root: Path, entry: Path) -> tuple[_WalkEntryAction, Path | None]:
44+
"""Decide how grep_content should treat entry, given the resolved search root.
45+
46+
This is the Python half of a contract the Go runtime implements as
47+
classifyWalkEntry in go/adk/pkg/tools/grep.go. Both must sort a tree into
48+
the same three outcomes, and reading them side by side is the only
49+
practical way to confirm they still do -- so they share these names, this
50+
argument order, and this return shape. Prefer changing both, or neither.
51+
52+
They are not branch-for-branch identical, and should not be forced to be.
53+
Go tests IsDir, EvalSymlinks, Stat and IsRegular separately because
54+
filepath.WalkDir hands it directories and unresolvable links; here
55+
os.walk yields only filenames, and Path.is_file() already collapses
56+
"exists, resolves, and is a regular file" into one call. The outcomes
57+
agree; the number of branches reaching them does not.
58+
59+
One known divergence, not reachable through any case we could construct:
60+
Go counts a failed Stat on a non-symlink as UNREADABLE, while
61+
Path.is_file() swallows the OSError and lands here on SKIP. Reaching it
62+
needs a stat that fails on an entry os.walk just listed -- EIO, a stale
63+
NFS handle, a misbehaving FUSE mount. Symlink loops and broken links are
64+
unaffected; both are UNREADABLE in either runtime.
65+
66+
Returns the resolved path alongside GREP so the caller reads what was
67+
just checked rather than re-resolving the symlink separately.
68+
"""
69+
# is_file() follows symlinks and checks S_ISREG, so a False covers two
70+
# cases that deserve different treatment.
71+
if not entry.is_file():
72+
# A broken symlink (or a symlink loop) is a genuine read failure.
73+
# Counting it is what stops a tree of dangling links from reporting a
74+
# confidently empty "no matches found".
75+
if entry.is_symlink() and not entry.exists():
76+
return _WalkEntryAction.UNREADABLE, None
77+
# A FIFO/socket/device is excluded by policy, not failure: opening one
78+
# can block indefinitely, and grep has no business reading it.
79+
return _WalkEntryAction.SKIP, None
80+
81+
resolved = entry.resolve()
82+
# Bound each entry by the directory actually being searched. The caller's
83+
# allowed_root is the whole session plus the skills dir, so it alone would
84+
# let a symlink here pull in a file from a sibling directory nobody asked
85+
# to search. Containment against root subsumes it: root was itself
86+
# validated against allowed_root, so anything under root is inside a root.
87+
if not resolved.is_relative_to(root):
88+
return _WalkEntryAction.SKIP, None
89+
90+
return _WalkEntryAction.GREP, resolved
91+
92+
3093
def _validate_path(
3194
file_path: Path,
3295
allowed_roots: Path | list[Path] | None,
@@ -236,43 +299,19 @@ def grep_file(file_path: Path) -> list[str]:
236299
skipped += len(walk_errors)
237300

238301
for entry in sorted(entries):
239-
# is_file() follows symlinks and checks S_ISREG, so a False here
240-
# covers two cases that deserve different treatment -- matching
241-
# the split Go makes in classifyWalkEntry:
242-
# - a broken symlink (or a symlink loop) is a genuine read
243-
# failure, so count it, as Go's walkEntryUnreadable does.
244-
# Otherwise a tree of dangling links reports a confidently
245-
# empty "no matches found".
246-
# - a FIFO/socket/device is excluded by policy, not failure
247-
# (opening one can block indefinitely), so stay silent, as
248-
# Go's walkEntrySkip does.
249-
if not entry.is_file():
250-
if entry.is_symlink() and not entry.exists():
251-
skipped += 1
302+
action, safe_entry = _classify_walk_entry(file_or_dir_path, entry)
303+
if action is _WalkEntryAction.SKIP:
252304
continue
253-
# Bound each entry by the directory actually being searched, not
254-
# by allowed_root: allowed_root is the whole session plus the
255-
# skills dir, so on its own it would let a symlink here pull in a
256-
# file from a sibling directory the caller never asked to search.
257-
# Go bounds each entry the same way (WithinRoot in
258-
# classifyWalkEntry), and both the tool description and the README
259-
# promise that behavior. Silent, matching Go's walkEntrySkip for an
260-
# out-of-root symlink.
261-
#
262-
# This subsumes a _validate_path(entry, allowed_root) check:
263-
# file_or_dir_path was itself validated against allowed_root above,
264-
# so anything relative to it is transitively inside a root.
265-
safe_entry = entry.resolve()
266-
if not safe_entry.is_relative_to(file_or_dir_path):
305+
if action is _WalkEntryAction.UNREADABLE:
306+
skipped += 1
267307
continue
268308
try:
269309
results.extend(grep_file(safe_entry))
270310
except OSError:
271311
# A read error on one file shouldn't abort matches already
272312
# found elsewhere in the tree, but it also shouldn't look
273-
# identical to a genuinely empty search -- see the note below.
313+
# identical to a genuinely empty search -- hence the count.
274314
skipped += 1
275-
continue
276315
else:
277316
if not file_or_dir_path.is_file():
278317
raise OSError(f"{file_or_dir_path} is not a regular file")

‎python/packages/kagent-skills/src/kagent/tests/unittests/test_skill_execution.py‎

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
write_file_content,
2222
)
2323
from kagent.skills.prompts import get_bash_description
24-
from kagent.skills.shell import _sanitize_env
24+
from kagent.skills.shell import _classify_walk_entry, _sanitize_env, _WalkEntryAction
2525

2626

2727
@pytest.fixture
@@ -463,6 +463,57 @@ def test_grep_content_counts_broken_symlink_as_unreadable(tmp_path):
463463
assert "could not be read" in result
464464

465465

466+
def test_classify_walk_entry_covers_each_outcome(tmp_path):
467+
"""Pin the three-way split directly, not just through grep_content.
468+
469+
This is the Python half of the contract go/adk/pkg/tools/grep.go states as
470+
classifyWalkEntry. Asserting it at the unit level is what makes the two
471+
runtimes comparable side by side: same three outcomes, same inputs, same
472+
answers. A divergence should fail here rather than surface later as a
473+
grep that quietly returns different results in one runtime.
474+
"""
475+
root = tmp_path / "root"
476+
root.mkdir()
477+
478+
plain = root / "plain.txt"
479+
plain.write_text("hello\n")
480+
481+
inside_link = root / "inside-link"
482+
inside_link.symlink_to(plain)
483+
484+
outside = tmp_path / "outside.txt"
485+
outside.write_text("secret\n")
486+
escaping_link = root / "escaping-link"
487+
escaping_link.symlink_to(outside)
488+
489+
broken_link = root / "broken-link"
490+
broken_link.symlink_to(tmp_path / "does-not-exist")
491+
492+
assert _classify_walk_entry(root, plain) == (_WalkEntryAction.GREP, plain.resolve())
493+
# A symlink that stays inside the root is greppable, and resolves through
494+
# to its target so the caller reads what was just checked.
495+
assert _classify_walk_entry(root, inside_link) == (_WalkEntryAction.GREP, plain.resolve())
496+
# Escaping the searched directory is a policy exclusion, so it is silent.
497+
assert _classify_walk_entry(root, escaping_link) == (_WalkEntryAction.SKIP, None)
498+
# A dangling link is a real read failure and must be counted.
499+
assert _classify_walk_entry(root, broken_link) == (_WalkEntryAction.UNREADABLE, None)
500+
# A symlink loop is the same class of failure as a dangling link.
501+
loop = root / "loop-link"
502+
loop.symlink_to(loop)
503+
assert _classify_walk_entry(root, loop) == (_WalkEntryAction.UNREADABLE, None)
504+
505+
506+
def test_classify_walk_entry_treats_a_fifo_as_a_silent_skip(tmp_path):
507+
"""A FIFO is excluded by policy, not failure, so it must not be counted."""
508+
fifo_path = tmp_path / "pipe"
509+
try:
510+
os.mkfifo(fifo_path)
511+
except (AttributeError, NotImplementedError, OSError):
512+
pytest.skip("FIFOs not supported")
513+
514+
assert _classify_walk_entry(tmp_path, fifo_path) == (_WalkEntryAction.SKIP, None)
515+
516+
466517
def test_grep_content_does_not_count_fifo_as_unreadable(tmp_path):
467518
"""A FIFO is excluded by policy, not failure -- it must stay silent.
468519

0 commit comments

Comments
 (0)