Skip to content

fix(community): run Leiden in a worker so its timeout can fire - #3478

Merged
magyargergo merged 6 commits into
abhigyanpatwari:mainfrom
drakeo338:claude/3476-fix
Oct 6, 2026
Merged

magyargergo merged 6 commits into
abhigyanpatwari:mainfrom
drakeo338:claude/3476-fix

Conversation

@drakeo338

@drakeo338 drakeo338 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Refs #3476.

Summary

Covers the Leiden community-detection timeout item from the list in #3476.

Motivation / context

The Leiden timeout never fired: graphology's Leiden runs synchronously on the main thread, so the timer could not run until it finished. The call now runs in a worker thread so the timeout can fire and fall back to one community.

Areas touched

  • gitnexus/ (CLI / core / MCP server)
  • gitnexus-web/ (Vite / React UI)
  • .github/ (workflows, actions)
  • eval/ or other tooling
  • Docs / agent config only (AGENTS.md, CLAUDE.md, .cursor/, llms.txt, etc.)

Scope & constraints

In scope

  • Leiden timeout in community-processor.ts and its unit test

Explicitly out of scope / not done here

Implementation notes

Testing & verification

Ran only test/unit/community-processor.test.ts on HEAD (23 passed, 0 failed); nothing else was run, so the boxes stay unticked.

  • cd gitnexus && npm test
  • cd gitnexus && npm run test:integration (if core/indexing/MCP paths changed)
  • cd gitnexus && npx tsc --noEmit
  • cd gitnexus-web && npm test (if web changed)
  • cd gitnexus-web && npx tsc -b --noEmit (if web changed)
  • Manual / Playwright E2E (note environment — see gitnexus-web/e2e/)

Risk & rollout

Checklist

  • PR body meets repo minimum length (workflow may label short descriptions)
  • If AGENTS.md / overlays changed: headers, scope block, and changelog updated per project conventions
  • No secrets, tokens, or machine-specific paths committed

📝 Summary by GitNexus

Summary

This appears to be a focused community-processing and worker-execution change. Its blast classification is critical, although the file-level risk is low.

🔴 CRITICAL blast radius. A worker-thread change to community processing in community-processor.ts, reaching three direct and transitive dependents.

The core changes are in runCommunityEngine, runIcebugLeiden, and runIcebugWorker in gitnexus/src/core/ingestion/community-processor.ts. The related tests are in gitnexus/test/unit/community-processor.test.ts and gitnexus/test/helpers/leiden-timeout-child.ts, with worker-thread coverage also in gitnexus/scripts/cross-platform-tests.ts.

Impact lands in the Ingestion, Unit, Passes, and Pipeline-phases modules, and the changed code appears in 5 affected flows. Review the worker execution and timeout behavior first, then check the related test coverage.

Added by GitNexus for PR #3478. Edit freely — this block is replaced on the next review, everything above it is left untouched.

…fire

leiden.detailed is synchronous and never returns on some graphs
(graphology/graphology#557), so the Promise.race timeout could not fire
and the one-community fallback was unreachable. Run it in a worker thread
and terminate the worker on timeout.

Fixes abhigyanpatwari#3476
@vercel

vercel Bot commented Oct 5, 2026

Copy link
Copy Markdown

@drakeo338 is attempting to deploy a commit to the NexusCore Team on Vercel.

A member of the Team first needs to authorize it.

@nexus-check

nexus-check Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Akon Labs

GitNexus Review · PR #3478

No issues found in 4 changed files.

Summary

This appears to be a focused community-processing and worker-execution change. Its blast classification is critical, although the file-level risk is low.

🔴 CRITICAL blast radius. A worker-thread change to community processing in community-processor.ts, reaching three direct and transitive dependents.

The core changes are in runCommunityEngine, runIcebugLeiden, and runIcebugWorker in gitnexus/src/core/ingestion/community-processor.ts. The related tests are in gitnexus/test/unit/community-processor.test.ts and gitnexus/test/helpers/leiden-timeout-child.ts, with worker-thread coverage also in gitnexus/scripts/cross-platform-tests.ts.

Impact lands in the Ingestion, Unit, Passes, and Pipeline-phases modules, and the changed code appears in 5 affected flows. Review the worker execution and timeout behavior first, then check the related test coverage.

🔀 Structural changes · claude/3476-fix → main

Both branches are separately indexed, so this compares their code graphs directly — what the diff cannot show.

Symbols removed (2)

  • gitnexus/src/core/ingestion/community-processor.ts::LeidenModule
  • gitnexus/src/core/ingestion/community-processor.ts::createSeededRng

Full detail lives in the GitNexus check run for this commit.

@nexus-check

nexus-check Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

🤖 Agent context for GitNexus Review · PR #3478

This comment carries deterministic graph detail for coding agents and reviewers who want the receipts — the main review comment carries the human summary.

🔴 CRITICAL blast radius — this change reaches 3 downstream symbols across 4 modules; this lands on a critical surface, so review the dependents carefully before merging. (driven by dependent/module count, not file risk)

Blast Level Dependents Modules Files
🔴 CRITICAL 3 4 4

What changed

Symbol Changes (24)
Kind Symbol Location
Const WORKER_THREADS gitnexus/scripts/cross-platform-tests.ts:290
Property fallbackReason gitnexus/src/core/ingestion/community-processor.ts:476
Property engine gitnexus/src/core/ingestion/community-processor.ts:498
Property engineRequested gitnexus/src/core/ingestion/community-processor.ts:498
Property communities gitnexus/src/core/ingestion/community-processor.ts:510
Property count gitnexus/src/core/ingestion/community-processor.ts:511
Property fallbackReason gitnexus/src/core/ingestion/community-processor.ts:515
Property communities gitnexus/src/core/ingestion/community-processor.ts:539
Property count gitnexus/src/core/ingestion/community-processor.ts:540
Property modularity gitnexus/src/core/ingestion/community-processor.ts:541
Property engine gitnexus/src/core/ingestion/community-processor.ts:542
Property engineRequested gitnexus/src/core/ingestion/community-processor.ts:543
Function runCommunityEngine gitnexus/src/core/ingestion/community-processor.ts:447
Function runGraphologyLeiden gitnexus/src/core/ingestion/community-processor.ts:480
Function runIcebugLeiden gitnexus/src/core/ingestion/community-processor.ts:520
Function isIntegerPartition gitnexus/src/core/ingestion/community-processor.ts:547
Function runIcebugWorker gitnexus/src/core/ingestion/community-processor.ts:554
Interface LeidenModule gitnexus/src/core/ingestion/community-processor.ts:34
Property Worker gitnexus/test/unit/community-processor.test.ts:214
Function runWorkerAgainstStub gitnexus/test/unit/community-processor.test.ts:312
Class MockWorker gitnexus/test/unit/community-processor.test.ts:198
Method constructor gitnexus/test/unit/community-processor.test.ts:199
Method terminate gitnexus/test/unit/community-processor.test.ts:206
Method unref gitnexus/test/unit/community-processor.test.ts:211
Changed Files (4)
File Status
gitnexus/scripts/cross-platform-tests.ts 🟡 modified
gitnexus/src/core/ingestion/community-processor.ts 🟡 modified
gitnexus/test/helpers/leiden-timeout-child.ts 🟢 added
gitnexus/test/unit/community-processor.test.ts 🟡 modified

What it affects

Architecture Impact

Module Hits Direct
Ingestion 4 🟢
Unit 1 ⚪
Passes 1 ⚪
Pipeline-phases 1 ⚪

Blast Radius

Depth Count
d1 (direct) 1
d2 (indirect) 1
d3 (transitive) 1
Direct dependents (d1)
  • gitnexus/src/core/ingestion/community-processor.ts:225 · processCommunities
Indirect dependents (d2)
  • gitnexus/src/core/ingestion/pipeline-phases/communities.ts:29 · execute
Transitive dependents (d3)
  • gitnexus/src/core/ingestion/pipeline-phases/communities.ts:22 · communitiesPhase

@koriyoshi2041 koriyoshi2041 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head faf3ab69. I reran npx vitest run test/unit/community-processor.test.ts on macOS: 23/23 passed. The regression exercises the real 105-node non-terminating graph, and the worker-backed path returned the one-community timeout fallback within the 500 ms guard instead of blocking the event loop. I also checked the worker lifecycle paths: message/error clear the timer, timeout terminates the worker, and an unexpected pre-settlement exit rejects. No correctness issue found in this scope.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The Icebug fallback path overwrites the new Graphology timeout reason, concealing degraded single-community results.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Moves synchronous Graphology Leiden execution to a worker so community detection can time out and degrade gracefully.

Changes:

  • Runs vendored Leiden in a terminable worker thread.
  • Adds regression coverage using the non-terminating graph from #3476.
File Description
gitnexus/​src/​core/​ingestion/​community-processor.ts Adds worker-based Leiden execution and timeout handling.
gitnexus/​test/​unit/​community-processor.test.ts Tests timeout fallback and updates worker mocks.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@magyargergo magyargergo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 81448a89c0308224ee52f74c4d1aaf8512be6b86.

The worker change fixes the reported hang: the real fixture returns the timeout fallback and the process exits. I'm requesting changes because the new test fails the test typecheck that CI runs. The second inline comment covers a gap in the termination regression.

I also reproduced the existing fallback-metadata finding. When Icebug fails and Graphology then times out, runCommunityEngine overwrites the timeout reason with the Icebug error. Preserve both reasons and cover that full chain. I used a shortened deadline to exercise this path.

Verification on Linux, Node 24.19.0:

  • 37 focused tests passed, including community detection and runtime-asset checks.
  • Production typechecking passed. Test typechecking failed with TS2351 at the new graph constructor; the same check passed on base df49f908 with the same dependencies.
  • Deleting only worker.terminate() in a disposable copy still left all 23 community tests green, while the standalone process stayed alive beyond five seconds.

Full suites, macOS/Windows runtime checks and large-graph memory benchmarks were not run. GitHub CI is awaiting approval.

Keep the Icebug failure and Graphology timeout in the diagnostic when both engines fall back. Preserve the existing single-error message.
Use the existing typed graph builder and preserve the original 105-node, 128-edge insertion order. The child returns its result and must exit naturally so the parent can detect a leaked worker.
Assert the hanging fixture returns the timeout fallback and exits naturally. Cover Icebug and auto failures followed by a Graphology timeout through processCommunities. Removing worker termination fails the child-process test; reverting diagnostic composition fails both fallback-chain tests.
Include the real worker shutdown regression in the existing Windows/macOS platform-sensitive test inventory.
@nexus-check

nexus-check Bot commented Oct 6, 2026

Copy link
Copy Markdown

🔄 What's new in this push (7b0a929)

The newest push appears to be a test configuration change to WORKER_THREADS in gitnexus/scripts/cross-platform-tests.ts. No modules were newly reached, and the risk transition is none.

The main GitNexus review comment has the full, updated report.

@magyargergo

Copy link
Copy Markdown
Collaborator

Addressed all three review findings in 2cd074b, a7c23c2, 9c952f3, and 7b0a929. Each thread has the fix and verification evidence and is now resolved.

  • Preserve both the Icebug failure and Graphology timeout in the final diagnostic.
  • Use the existing typed graph builder and preserve the original fixture order.
  • Require the hanging fixture to return its fallback and exit naturally in a child process; include the suite in the Windows/macOS test inventory.

Validation on the published tree:

  • Community suite: 25/25 pass.
  • Mutation checks: removing worker termination fails the child-process test; reverting the diagnostic fix fails both fallback-chain tests.
  • Production and test typechecks, final build, Prettier, and git diff --check: pass. ESLint: no errors, two existing warnings.
  • Full npm test run: incomplete. It hit an unrelated native-database assertion failure and repeated unavailable LadybugDB extension downloads, so I stopped the run. The same lbug-delete-nodes-for-files.test.ts:471 assertion fails on the unchanged PR head 81448a89 (1 failed, 9 passed).

The four published files match the validated local files byte for byte. No new dependency or production abstraction was added.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

CI Report

✅ All checks passed

Pipeline Status

Stage Status Details
✅ Typecheck success tsc --noEmit
✅ Tests success unit tests, 3 platforms
✅ E2E success gitnexus-web changes only

Test Results

Tests Passed Failed Unverified Duration
25544 25544 0 0 1821s

✅ All 25544 tests passed

Code Coverage

Tests

Metric Coverage Covered Base Delta Status
Statements 84.56% 91341/108011 N/A% — 🟢 ████████████████░░░░
Branches 74.43% 60945/81872 N/A% — 🟢 ██████████████░░░░░░
Functions 90.54% 11156/12321 N/A% — 🟢 ██████████████████░░
Lines 87.97% 80954/92021 N/A% — 🟢 █████████████████░░░

📋 View full run · Generated by CI

@magyargergo
magyargergo merged commit 047354c into abhigyanpatwari:main Oct 6, 2026
54 of 55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants