Skip to content

memory: add a scale-correctness round-trip test (5,000 entities, exact read-back) #5540

Open

Description

@reservoir-cmyk

Which server?

memory (@modelcontextprotocol/server-memory)

Protocol era

Modern (2026-07-28)

MCP client

No response

The problem

This proposes one test for the memory server: not a benchmark, a correctness pin at scale. The existing scale coverage is small-scale checks around correlated search; nothing verifies that what is written at thousands-of-records scale reads back exactly — no dropped records, no observations attached to the wrong entity, no relations losing endpoints — when read through a fresh manager instance (the restart equivalent).

Case

Create 5,000 entities in batches of 500 (two observations each) plus 999 chained relations, then read the graph back through a new KnowledgeGraphManager and assert: entity and relation counts, the last entity's observations verbatim, a unique-observation search hit, and the on-disk line count matching the record count. Given the JSONL one-record-per-line format and the atomic full-rewrite save, the round trip should be the identity; if some scale breaks it, that is a correctness problem before it is a performance one. (Throughput observations are deliberately not part of this proposal.)

Solution you have in mind (optional)

Proposed test

For src/memory/__tests__/knowledge-graph.test.ts (a scale describe) or a new file:

it(
  "reads back exactly what was written at 5,000-entity scale",
  async () => {
    const writer = new KnowledgeGraphManager(testFilePath);
    const TOTAL = 5000;
    const BATCH = 500;
    for (let start = 0; start < TOTAL; start += BATCH) {
      const batch: Entity[] = Array.from({ length: BATCH }, (_, i) => {
        const n = start + i;
        return {
          name: `entity-${n}`,
          entityType: "bulk",
          observations: [`obs-${n}`, `marker-${n % 7}`],
        };
      });
      await writer.createEntities(batch);
    }
    const relations: Relation[] = Array.from({ length: 999 }, (_, i) => ({
      from: `entity-${i}`,
      to: `entity-${i + 1}`,
      relationType: "next",
    }));
    await writer.createRelations(relations);

    const reader = new KnowledgeGraphManager(testFilePath);
    const graph = await reader.readGraph();
    expect(graph.entities).toHaveLength(TOTAL);
    expect(graph.relations).toHaveLength(999);
    const last = graph.entities.find((e) => e.name === "entity-4999");
    expect(last?.observations).toEqual(["obs-4999", `marker-${4999 % 7}`]);

    const found = await reader.searchNodes("obs-4242");
    expect(found.entities.map((e) => e.name)).toEqual(["entity-4242"]);

    const lines = (await fs.readFile(testFilePath, "utf-8"))
      .split("\n")
      .filter((l) => l.trim() !== "");
    expect(lines).toHaveLength(TOTAL + 999);
  },
  120_000,
);

Which MCP feature does it demonstrate? (optional)

No response

Alternatives or workarounds you have tried

Closest hit checked: #3137 concerns an open_nodes bug, not a duplicate of this test proposal. No workaround covers this — no existing test verifies exact read-back at 5,000-entity scale.

Already built it locally?

Verification

Ran against a byte-identical copy of v2/main @ 6cfff62: the case passed (the round trip itself took ~0.6 s on the test machine; the generous timeout is there to keep it non-flaky on slow CI, not as a performance claim). Full memory suite: 241 passed + 1 skipped.

Prompt used

"In src/memory, add a scale-correctness test (not a benchmark): create 5,000 entities in batches of 500 with two observations each, plus 999 chained relations, then read the graph back through a FRESH KnowledgeGraphManager instance and assert exact equality of counts, the last entity's observations, a unique-observation search hit, and the on-disk line count. It should complete in about a second; give it a generous timeout so it stays non-flaky on slow CI."

Before you submit

  • I searched existing issues and this is not a duplicate.
  • This is a request for this repository or one of its servers, not a new server, the MCP specification or an SDK.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions