Skip to content

feat: add structured project search to native environments - #5574

Merged
markstuart-oai merged 9 commits into
feat/codex-harnessfrom
mark/req739-project-search-python
Nov 7, 2026
Merged

markstuart-oai merged 9 commits into
feat/codex-harnessfrom
mark/req739-project-search-python

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This pull request adds opt-in project_glob and project_grep tools to native Environment sessions. Agents can find files and matching lines as structured results without writing shell commands. Search stays within the current Environment.cwd and returns relative paths, line context, errors, and whether limits cut the search short.

Search uses the existing approval, argument, stop, and result policies. It does not follow symlinks and caps directory scans, file reads, and output size. Remote executors must advertise support for these bounds. Search rejects control characters and dot components in requested paths, except for the . root. It also checks that joining a path to the root preserves the exact spelling approved by policy. Traversal preserves literal filenames. Restored sessions use the current root and tool grants after executor-policy checks pass. Application tools may use either search name when the corresponding native tool is disabled. The pinned native revision stays unchanged.

Test plan

  • Built a fresh native wheel and passed the clean installed-wheel runtime check.
  • Full local verification passed: formatting, lint, both type checkers, 5,262 parallel tests, and 53 serial tests. The parallel run skipped 228 tests.
  • Focused local search checks passed 5 tests and skipped 46 cases that require native sandbox support. The macOS native CI run passed 5,367 parallel tests and 44 serial tests; 233 parallel and 9 serial cases were skipped.
  • Shared native search tests passed all 7 cases, including path normalization and literal-name regressions.
  • Native filesystem regression checks passed on Linux, including all 10 current focused cases. The invalid-byte fixtures ran without skipping. Windows-specific native unit tests have not run.
  • CI is green on ae9a9c01ddded54fde57a325860558e22d19317b: 24 checks passed and 15 were skipped. Current code and security reviews are complete, with all review threads resolved. The earlier changes-requested maintainer review still needs to be updated.

Issue number

None linked.

Checks

  • I've added new tests, if relevant
  • I've run the applicable checks in AGENTS.md, including .agents/skills/code-change-verification/scripts/run.sh when required
  • I've confirmed all required checks pass
  • I've completed the review required by AGENTS.md

@markstuart-oai
markstuart-oai requested review from a team, rm-openai and seratch as code owners November 7, 2026 05:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-07T08:04:13.239772Z ae9a9c0 Manual request
🔒 Security Review ✅ Completed 2026-11-07T08:05:51.688344Z ae9a9c0 Manual request

Security findings

Advisory findings (3)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment thread packages/codex/rust/python_tools.rs Resolved

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing reserved-name finding is valid: application FunctionTools named project_glob/project_grep are rejected even without opting into native search. Please condition collision checks on enabled native tools and retain the existing no-Environment path. Referencing the existing discussion rather than duplicating it: #5574 (comment)

Coordinator and independent static review covered rooted filesystem access, bounded traversal/read/output, executor capabilities, result policies, restore, cancellation, and relevant regression tests. No additional actionable findings. Hosted native-platform CI and Rust CodeQL remain pending. No candidate code or tests executed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb9f9553a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The application tool-name compatibility issue is fixed: names remain available unless the matching Environment tool is enabled. Static re-review confirms one remaining P2 already reported at #5574 (comment): a single non-UTF-8 directory entry aborts read_directory_bounded and causes sync_walk_no_follow to discard every valid sibling and subdirectory. Retain valid entries while reporting/skipping the unsupported name, keeping skipped entries within the enumeration budget. No duplicate inline finding added.

Coordinator and independent static review only; no candidate code or local tests run. Current-head hosted platform CI remains pending. Merge/release hold remains in place.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining filename finding is fixed at d73996b. Unsupported Unix/Windows names are individual errors, so valid siblings and descendants survive. Errors consume the scan budget before being skipped; enumeration failures preserve prior entries, while setup/cancellation failures retain their prior behavior. Native and public regression coverage addresses these cases; patch SHA-256 matches the manifest.

Coordinator and independent static re-review found no additional actionable issue. No candidate code or tests executed locally. Current-head platform CI and CodeQL remain pending; Windows-specific native cases are author-reported unexecuted. Keep the stated merge/release hold until validation is complete.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: d73996bf12

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: d73996bf12

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread packages/codex/rust/project_search.rs Resolved

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the earlier approval to request changes for the newly identified P2 path-authorization mismatch, independently validated against the exact pinned source: #5574 (comment)

The pre-dispatch Environment callback receives the original arguments. relative_path accepts TAB/LF/CR, but root.join uses pinned PathUri::join -> url 2.5.8 PathSegmentsMut -> parser Input, which drops those characters. For example, a callback denying the secrets subtree can approve the distinct spelling sec\nrets, after which project_grep walks and reads secrets. Root containment and no-follow traversal do not prevent this within-root policy bypass.

Reject these characters before filesystem dispatch, or preserve path spelling losslessly and ensure authorization applies to the actual I/O path. Add regression coverage for TAB/LF/CR with a subtree-denying callback and verify denied content is never read or returned. Referencing the existing discussion rather than duplicating an inline finding.

Coordinator and independent static source assessment; no candidate code/tests executed. The previous filename fix remains valid. Hosted CI and merge/release follow-up remain open.

Comment thread packages/codex/rust/project_search.rs Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: 81f5f267ac

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread packages/codex/rust/project_search.rs Resolved

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The control-character path fix, native filename identity fix, and EILSEQ-only fixture gating address the prior findings. One authorization-normalization gap remains: ./secrets passes relative_path and the existing regression callback (which denies the first component secrets), then root.join normalizes it to secrets before filesystem access. I independently verified the existing finding: #5574 (comment) . Please reject dot components in non-root paths (preserving the complete default path .), or authorize the same canonical path used for I/O; cover both tools with a denying callback and no protected filesystem access. Referencing the existing thread instead of duplicating its inline finding.

Coordinator and independent static re-review of this exact commit; no candidate code or tests executed. Hosted Linux/macOS/Windows native snapshot jobs and Rust CodeQL remain pending.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining path-authorization blocker is fixed: non-root paths reject dot components, and the joined path must round-trip to the exact approved relative spelling before any filesystem access. The whole-path root sentinel remains supported. Reviewed both-tool denial regressions, literal percent/Unicode/space round trips, and the drive-like URI edge case against the pinned path implementation. No new actionable findings.

Coordinator and independent static re-review; no candidate code or tests executed. Hosted Linux/macOS/Windows native snapshots and Rust CodeQL remain pending. This supersedes my prior request-changes decision; CI follow-up remains open.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: ae9a9c01dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: ae9a9c01dd

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread packages/codex/rust/project_search.rs Resolved
@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: ae9a9c01dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: ae9a9c01dd

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Copy link
Copy Markdown
Contributor Author

@dpiet-oai Please re-review ae9a9c01ddded54fde57a325860558e22d19317b when you can; your changes-requested review is on the earlier d73996bf commit. The reported issues are addressed, all review threads are resolved, and the current code and security review cycle has finished. The fresh wheel, installed-package check, and full local suite passed. One platform CI job is still running with no failures so far.

@markstuart-oai
markstuart-oai merged commit e283d9a into feat/codex-harness Nov 7, 2026
39 checks passed
@markstuart-oai
markstuart-oai deleted the mark/req739-project-search-python branch November 7, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants