Skip to content

Bound the outline file queue#2787

Merged
HerringtonDarkholme merged 1 commit into
ast-grep:mainfrom
morgan-coded:fix/2741-bound-outline-queue
Jul 2, 2026
Merged

Bound the outline file queue#2787
HerringtonDarkholme merged 1 commit into
ast-grep:mainfrom
morgan-coded:fix/2741-bound-outline-queue

Conversation

@morgan-coded

@morgan-coded morgan-coded commented Jul 2, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Large ast-grep outline runs could queue extracted file payloads faster than the emitter wrote output, because the parallel walker handed files through an unbounded channel. The handoff now uses a bounded sync_channel with a 256-file queue, applying backpressure without changing the walker, extraction, or emit flow.

The extractor backpressure case and outline CLI coverage passed, followed by cargo fmt --all -- --check, cargo test -p ast-grep outline::extract, cargo test -p ast-grep test_outline, full cargo test, and git diff --check. Fixes #2741

Summary by CodeRabbit

  • Bug Fixes
    • Improved outline processing reliability by limiting queued work, helping prevent runaway memory growth during heavy or slow operations.
    • Added safeguards so outline extraction now handles backpressure more gracefully under load.

Copilot AI review requested due to automatic review settings July 2, 2026 21:07
Copilot AI reviewed Jul 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c9c51cbc-bedb-4952-b596-1e3ee26e1d12

📥 Commits

Reviewing files that changed from the base of the PR and between ba4047f and 6d9ee50.

📒 Files selected for processing (1)
  • crates/cli/src/outline/extract.rs
📝 Walkthrough

Walkthrough

The outline extraction pipeline in crates/cli/src/outline/extract.rs now uses a bounded mpsc::sync_channel (capped at a new OUTLINE_QUEUE_BOUND constant) instead of an unbounded channel, via a new outline_channel() helper. A unit test verifies the queue rejects sends once full.

Changes

Bounded Outline Queue

Layer / File(s) Summary
Bounded channel constant and helper
crates/cli/src/outline/extract.rs		 Adds OUTLINE_QUEUE_BOUND constant and outline_channel() helper wrapping mpsc::sync_channel, and updates stream_paths to use it instead of an unbounded channel.
Backpressure test
crates/cli/src/outline/extract.rs		 Adds a test filling the channel to its bound via try_send and asserting the next try_send returns TrySendError::Full.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Poem

A queue once vast without a bound,
Now capped at two-five-six, sound!
Producers pause, no memory spike,
Backpressure hops just like I like 🐇
Full channel? Full stop, try again!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately describes the main change: bounding the outline file queue.
Linked Issues check ✅ Passed The change replaces the unbounded channel with a bounded sync_channel of 256 in stream_paths, matching #2741.
Out of Scope Changes check ✅ Passed The diff stays focused on the queue-bounding fix and its test, with no unrelated changes visible.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.15385% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.39%. Comparing base (ba4047f) to head (6d9ee50).

Files with missing lines Patch % Lines
crates/cli/src/outline/extract.rs 96.15% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2787      +/-   ##
==========================================
+ Coverage   86.37%   86.39%   +0.01%     
==========================================
  Files         126      126              
  Lines       22766    22791      +25     
==========================================
+ Hits        19665    19691      +26     
+ Misses       3101     3100       -1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HerringtonDarkholme
HerringtonDarkholme added this pull request to the merge queue Jul 2, 2026
Merged via the queue into ast-grep:main with commit fe3607e Jul 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@HerringtonDarkholme HerringtonDarkholme HerringtonDarkholme approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bound the outline file queue to prevent memory spikes under parallel walk

3 participants

@morgan-coded @HerringtonDarkholme