Description

Hi.

I would like to propose discussing a pluggable module system for agent context management.

Short version:

A context module should own the context rewrite strategy.
It should be able to modify context through deterministic rules.
It should be able to ask the runtime to run an LLM summary.
It should be able to combine both approaches.
Existing hardcoded context passes could gradually become the built-in implementation of this module API.

Why

Context handling is one of the most important parts of the runtime.

LibreFang already has several hardcoded mechanisms that are effectively context rewrite policies:

• tool result spill / truncation;
• per-turn tool-result budget enforcement;
• context guard compaction for older tool results;
• history fold for stale tool results;
• LLM-based context compression;
• developer-tool loop aggregation;
• reasoning / thinking pruning;
• image and media stripping;
• safe history trimming;
• tool_use / tool_result pairing protection.

These mechanisms are useful, but they are currently spread across the runtime and embedded directly into the agent-loop pipeline.

I think it would be useful to gradually move this into a dedicated context policy / context module layer.

Motivation

Different agent types likely need different context strategies.

For example:

• a web-search agent needs to clean HTML/search noise, deduplicate sources, and preserve URLs, dates, titles, and source names;
• a coding agent needs to preserve commands, exit codes, test/compiler errors, file paths, recent changes, and unfinished task state;
• a workflow agent may need to pin final outputs and aggressively compress intermediate steps;
• a chat agent may be able to compress old small-talk turns more aggressively.

A universal compression/trimming strategy is unlikely to work equally well for all agent types.

Proposed Shape

This should not be only an "LLM summarization plugin".

The broader shape is:

The context module manages the context rewrite strategy.

It should be able to:

• edit incoming user/system messages before they are written into context;
• edit tool results before they are added to history;
• compress or rewrite old tool results without an LLM;
• aggregate sequences of developer-tool calls;
• remove or collapse noisy/repeated output;
• prune reasoning / thinking blocks;
• decide which messages should stay verbatim;
• decide when context should be compacted;
• receive an explicit runtime request to compact now;
• ask the runtime to perform LLM summarization;
• combine deterministic rewrite and LLM summarization.

Two Compaction Directions

It is important to support both directions.

1. Module-driven compaction

The module can inspect context pressure and decide what to do:

• do nothing yet;
• run a cheap deterministic rewrite;
• request an LLM summary;
• run a mixed strategy.

Example:

Context is above 50%.
First collapse stale shell/test outputs.
Then summarize the cleaned middle section.
Keep the last 8 messages verbatim.

2. Runtime-requested compaction

The runtime should also be able to ask the module to compact context.

Possible reasons:

• context window pressure;
• manual /compact;
• cron/session compaction;
• provider overflow recovery;
• before an expensive LLM call;
• background maintenance;
• model switch to a smaller context window.

So the module API should support not only should_compact, but also an explicit request:

runtime -> module:
  "Please compact this context now.
   Reason: token_pressure/manual/cron/provider_overflow.
   Budget: X tokens.
   Context window: Y.
   Preserve safety invariants."

The module could respond with:

• NoChange;
• a deterministic ContextPatch;
• a full replacement Vec<Message>;
• RequestLlmSummary;
• a mixed plan.

LLM Boundary

I do not think an external module should receive Arc<dyn LlmDriver> directly.

A serializable contract seems safer.

The module could return an LLM summary request:

{
  "action": "request_llm_summary",
  "messages": [],
  "prompt": "Summarize for a coding agent. Preserve file paths, commands, errors, and current task state.",
  "max_tokens": 1200
}

The runtime remains the owner of:

• the LLM driver;
• credentials;
• provider routing;
• fallback providers;
• budget accounting;
• telemetry;
• rate limits;
• safety checks.

After the LLM call, the runtime could either apply the summary plan itself or return the summary to the module for a final rewrite.

Possible API Sketch

Conceptually:

ContextSnapshot -> ContextDecision

ContextSnapshot could contain:

• agent id / session id;
• agent metadata;
• messages;
• tools metadata;
• context window;
• estimated tokens;
• invocation reason;
• current phase.

Possible invocation reasons:

incoming_message
tool_result
pre_llm_call
runtime_requested_compaction
manual_compact
cron_compaction
provider_overflow
after_turn

Possible decisions:

NoChange
PatchMessages(Vec<ContextPatch>)
ReplaceMessages(Vec<Message>)
RequestLlmSummary(CompactionPlan)
MixedPlan(Vec<ContextAction>)

Possible actions:

truncate_tool_result
spill_tool_result
fold_tool_results
aggregate_tool_loop
prune_reasoning
strip_media
summarize_messages
preserve_recent
pin_message
drop_message

The runtime should validate the result before sending it to a provider:

• tool_use / tool_result pairing must remain valid;
• history must start with a valid role for strict providers;
• pinned messages must be respected;
• session repair should still run as a final safety net;
• unsafe module output should fall back to built-in behavior.

Debugging Benefit

This would also help with debugging context loss.

Right now, when an agent loses context, it is hard to know why:

• was it safe trim?
• history fold?
• context guard?
• artifact spill?
• LLM compression?
• developer-loop aggregation?
• memory recall?
• prompt assembly?

A module-based system could emit trace output like:

tool_result abc shortened from 80k chars to 3k
messages 12..24 collapsed as stale shell loop
reasoning blocks removed from 6 old assistant turns
LLM summary requested for cleaned middle section
kept last 8 messages verbatim

That would make context behavior much easier to inspect, test, and tune.

Hot Reload

Another benefit is development ergonomics.

If context modules can be hot-reloaded, it becomes much easier to debug and iterate on context strategies:

• change module rules;
• replay the same session;
• compare output messages;
• inspect trace;
• adjust thresholds;
• test agent-specific behavior without rebuilding the whole daemon.

This seems especially useful while current context-loss issues are being investigated.

Migration Path

This does not need to be a big-bang rewrite.

A reasonable path could be:

1. Define a ContextPolicy / ContextRewriteModule contract.
2. Move existing hardcoded passes behind a built-in implementation.
3. Add trace output for every rewrite decision.
4. Add per-agent context policy selection.
5. Add deterministic module actions first.
6. Add RequestLlmSummary as a host-executed action.
7. Later add WASM/sidecar/plugin backends.

The first useful version could keep all current behavior, but route it through the new module interface.

Question

Does this direction fit LibreFang's architecture?

If yes, I would like to discuss what the right boundary should be:

• extend the existing ContextEngine;
• add a new ContextPolicy layer above or beside it;
• make WASM/plugin modules a backend for that layer;
• or start by moving the current built-in passes behind a common interface.

Alternatives Considered

No response

Additional Context

No response