Skip to content

UI /api/layout drops connection for large graph when max_nodes is high #726

Open

Description

@sharif-smj

Summary

The graph UI fails for a large indexed private repository. Indexing succeeds and CLI/MCP graph queries work, but opening the project in the browser UI causes /api/layout to hang and then drop the connection without a JSON error response.

This appears isolated to the browser UI layout endpoint, not the core indexer.

Environment

  • codebase-memory-mcp --version: 0.8.1
  • Binary: UI release asset for macOS arm64
  • OS: macOS arm64
  • UI URL: http://127.0.0.1:9749

Project size

The indexed private project is roughly:

  • 23k nodes
  • 96k edges

list_projects and get_architecture work for this project.

Reproduction

After indexing a large project, open it in the graph UI, or call the layout endpoint directly:

curl -v 'http://127.0.0.1:9749/api/layout?project=<large-private-project>&max_nodes=50000'

Observed result:

curl: (52) Empty reply from server

In Chrome, the graph page shows:

Failed to fetch

The request hangs for roughly 20-35 seconds before the connection is dropped. The local UI process then restarts if supervised externally.

Threshold observed locally

These returned successfully:

max_nodes=10000
max_nodes=12000
max_nodes=14000

These failed or caused the UI server connection to be dropped:

max_nodes=16000
max_nodes=20000
max_nodes=50000

The bundled frontend appears to request max_nodes=50000 when loading a graph.

Expected behavior

For large projects, the UI should ideally do one of the following:

  • render a bounded layout that the server can handle,
  • automatically lower max_nodes,
  • stream/progressively load layout data,
  • or return a structured JSON error instead of dropping the connection.

Notes

I intentionally redacted the repository name, local paths, and project-specific identifiers because this was reproduced against a private codebase. Happy to provide more diagnostics if there is a safe way to collect them without exposing source or project details.

Activity

  1. added
    bugSomething isn't working
    stability/performanceServer crashes, OOM, hangs, high CPU/memory
    ux/behaviorDisplay bugs, docs, adoption UX
    priority/highNeeds near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.
    on Jun 30, 2026
  2. DeusData commented on Jun 30, 2026

    @DeusData
    Owner

    Thanks for the careful report and for keeping the private repo details out. This is a valid UI/server resilience bug: the layout endpoint needs a bounded default and a structured failure path instead of letting a large graph drop the connection. I’m tracking it under the large-repo performance epic and marking it high priority for the graph UI path. The thresholds you included are already actionable.

  3. rudi193-cmd commented on Jul 5, 2026

    @rudi193-cmd
    Contributor

    Data point — willow-2.0 graph size (v0.8.1, linux-amd64)

    Public repo: willow-2.0. Indexed project home-sean-campbell-github-willow-2.0:

    Metric Value
    Nodes 17,835
    Edges 57,740
    Status ready

    This is a Python-heavy fleet monorepo (hooks, MCP server, Kart worker, Postgres bridge) — relevant to your question about edge density after the 10k max_nodes cap on current main.

    We have not exercised the UI layout endpoint directly; we use CBM via stdio/cbm_* MCP tools for audits. Flagging the numbers in case they help scope the adjacency-list fix: at full index size we sit just above the 10k cap, so any UI path that clamps to 10k nodes still sees a substantial edge set and would hit compute_call_depth on a filtered subgraph.

    Audit context (graph = discovery, not measurement): docs/audits/CODEBASE_MEMORY_MCP_TOOL_EVAL_ADDENDUM_2026-07-05.md

  4. added
    awaiting-reporterWaiting on the reporter for info/repro; stale bot will warn then close
    on Jul 8, 2026
  5. github-actions commented on Jul 8, 2026

    @github-actions

    Thanks for the report! To move this forward we need a bit more so we can reproduce it ourselves:

    • the codebase-memory-mcp --version you're on
    • the exact steps or command you ran
    • a public repo (or a small dummy snippet) that shows the problem — please don't paste proprietary code

    Once that's here we'll pick it straight back up. Heads-up: issues left awaiting-reporter are automatically closed after a few weeks of silence, but a comment reopens the door anytime.

  6. DeusData commented on Jul 8, 2026

    @DeusData
    Owner

    v0.9.0 includes graph layout/rendering stability work for much larger graphs, so this dropped layout request needs a current retest. Could you try v0.9.0 and report whether /api/layout still disconnects on the same repo? Marking awaiting-reporter.

  7. added this to the 0.9.1-rc milestone on Jul 8, 2026
  8. github-actions commented on Jul 30, 2026

    @github-actions

    This issue has been waiting on more information for 21 days (a version, exact steps, or a public repro), so it's now marked stale. It will be closed in 14 days if there's no update — just add a comment to keep it open. We're happy to pick it back up the moment we can reproduce it.

  9. added
    staleNo activity after an info request; will be closed soon
    on Jul 30, 2026
  10. rudi193-cmd commented on Jul 31, 2026

    @rudi193-cmd
    Contributor

    Retest on a public repo — does not reproduce on current main

    Ran the layout ladder against a public repo so this is reproducible outside a private codebase: willow-2.0, indexed project home-sean-campbell-github-willow-2.0.

    Graph: 20,489 nodes / 87,597 edges (grown since my earlier data point of 17,835 / 57,740 — closer now to the reporter's 23k / 96k).

    Build: main @ 7d6cdb23 (v0.9.0-237-g7d6cdb23), built with make -f Makefile.cbm cbm-with-ui. Not the v0.9.0 release asset — see caveats.
    Platform: linux x86_64, 15 GB RAM, server reports mem.init budget_mb=3795.

    max_nodes=5000   http=200 time=0.869s size=4,419,203
    max_nodes=10000  http=200 time=1.379s size=8,741,072
    max_nodes=12000  http=200 time=1.687s size=11,096,710
    max_nodes=14000  http=200 time=2.031s size=13,520,018
    max_nodes=16000  http=200 time=2.442s size=16,343,967
    max_nodes=20000  http=200 time=3.324s size=21,635,042
    max_nodes=50000  http=200 time=3.466s size=22,072,098
    

    No dropped connections, no empty replies, no restarts. Server log shows status=200 for all seven, max duration_ms=3467.

    Two things worth noting beyond the pass/fail:

    • Latency scales smoothly with returned size (0.87s → 3.47s), with no cliff anywhere near the reported 20–35s hang.
    • The clamp behaves. max_nodes=20000 and 50000 return nearly identical payloads (21.6 MB vs 22.1 MB) because the graph has 20,489 nodes — the top rung is "whole graph", and it completed in 3.5s.

    That last point is the interesting contrast with the original report: the failures there started at max_nodes=16000 on a 23k-node graph, i.e. below the node count, whereas here every request at or above node count succeeded.

    Caveats — what this does and doesn't establish

    1. Not the v0.9.0 release binary. This is main 237 commits past the tag, so it can't tell you whether v0.9.0 itself fixed it — only that it's not present on current main.
    2. Stub frontend. I built with a placeholder graph-ui/dist (no npm available in my sandbox), so the embedded UI is a stub and all requests were via curl. /api/layout is server-side so this shouldn't matter for the endpoint — but the frontend's own max_nodes=50000 load path wasn't exercised end to end.
    3. Different platform. linux x86_64 here vs macOS arm64 in the report.
    4. Slightly smaller graph. 20,489 / 87,597 vs 23k / 96k.

    Suggestion

    If the maintainers want to narrow it before closing: the discriminator may be memory headroom rather than graph size. The server logs mem.init budget_mb=… at startup (3795 here, on a 15 GB machine). Asking @sharif-smj for that line from their run — plus whether a v0.9.0+ retest still fails on macOS arm64 — would separate "fixed" from "platform/RAM-dependent" cleanly.

    I can re-run this ladder against a proper release UI asset for linux if that's useful.

  11. removed
    staleNo activity after an info request; will be closed soon
    on Aug 1, 2026
  12. github-actions commented on Aug 23, 2026

    @github-actions

    This issue has been waiting on more information for 21 days (a version, exact steps, or a public repro), so it's now marked stale. It will not be closed automatically — a maintainer reviews stale issues by hand. Adding a comment with the details clears the label and puts it straight back in the queue. We're happy to pick it back up the moment we can reproduce it.

  13. added
    staleNo activity after an info request; will be closed soon
    on Aug 23, 2026
  14. removed
    awaiting-reporterWaiting on the reporter for info/repro; stale bot will warn then close
    staleNo activity after an info request; will be closed soon
    on Aug 24, 2026
  15. DeusData commented on Aug 24, 2026

    @DeusData
    Owner

    Thank you for the public current-main layout ladder and for stating its release, frontend, platform, and graph-size limitations clearly. It provides the requested shareable control and shows that the disconnect is not reproduced on Linux current main, while still leaving the original macOS release and memory-headroom cases unresolved.

    I have removed awaiting-reporter and stale because substantive follow-up was already provided. The issue remains open and high priority while we review the remaining platform and release distinction. Thank you for supplying measured timings and payload sizes rather than treating one passing environment as proof that the original failure is fixed.

  16. DeusData commented on Sep 26, 2026

    @DeusData
    Owner

    Thank you, @sharif-smj, for the careful thresholds, and @rudi193-cmd for the public Linux ladder!

    To close the remaining gap, the macOS arm64 release, we ran the same ladder against the v0.11.0 release binary on macOS arm64, with a synthetic graph larger than yours (45,654 nodes / 103,241 edges). max_nodes 10k, 16k, 20k and 50k all returned HTTP 200. The full 50k request took 3.2 s for a 19 MB payload, with no dropped connection. Layout now uses a Barnes-Hut octree and a node budget.

    Could you confirm on v0.11.0 with your project? If /api/layout still drops, the mem.init budget_mb=… line and the failing max_nodes value would tell us whether memory headroom is the factor.

  17. added
    awaiting-reporterWaiting on the reporter for info/repro; stale bot will warn then close
    on Sep 26, 2026
  18. github-actions commented on Sep 26, 2026

    @github-actions

    Thanks for the report! To move this forward we need a bit more so we can reproduce it ourselves:

    • the codebase-memory-mcp --version you're on
    • the exact steps or command you ran
    • a public repo (or a small dummy snippet) that shows the problem — please don't paste proprietary code

    Once that's here we'll pick it straight back up. Heads-up: issues left awaiting-reporter are automatically closed after a few weeks of silence, but a comment reopens the door anytime.

  19. github-actions commented on Oct 18, 2026

    @github-actions

    This issue has been waiting on more information for 21 days (a version, exact steps, or a public repro), so it's now marked stale. It will not be closed automatically — a maintainer reviews stale issues by hand. Adding a comment with the details clears the label and puts it straight back in the queue. We're happy to pick it back up the moment we can reproduce it.

  20. added
    staleNo activity after an info request; will be closed soon
    on Oct 18, 2026
  21. DeusData commented on Oct 19, 2026

    @DeusData
    Owner

    Thank you again for the measured layout thresholds and Linux control. Checking back on the September macOS retest request: does the layout still disconnect on your current version and project? If so, the failing max_nodes value and memory-budget line are enough to start; private source is not needed.

  22. removed
    staleNo activity after an info request; will be closed soon
    on Oct 20, 2026
  23. DeusData commented on Oct 25, 2026

    @DeusData
    Owner

    Thank you again for the report and your patience. If you have already retested the layout disconnect, the current version, failing max_nodes value and memory-budget line would help; private source is not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    awaiting-reporterWaiting on the reporter for info/repro; stale bot will warn then closebugSomething isn't workingpriority/highNeeds near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.stability/performanceServer crashes, OOM, hangs, high CPU/memoryux/behaviorDisplay bugs, docs, adoption UX

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions