Bump socket2 from 0.5.10 to 0.6.3#8
Closed
dependabot[bot] wants to merge 1 commit into
Closed
Conversation
Bumps [socket2](https://github.com/rust-lang/socket2) from 0.5.10 to 0.6.3. - [Release notes](https://github.com/rust-lang/socket2/releases) - [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/socket2/commits/v0.6.3) --- updated-dependencies: - dependency-name: socket2 dependency-version: 0.6.3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
Contributor
Author
LabelsThe following labels could not be found: Please fix the above issues or remove invalid values from |
Contributor
Author
|
OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting If you change your mind, just re-open this PR and I'll resolve any conflicts on it. |
houko
added a commit
that referenced
this pull request
Mar 18, 2026
Knowledge graph cleanup (#3): - Migration v10: add agent_id column to entities and relations tables - KnowledgeStore: add_entity/add_relation now track agent_id - KnowledgeStore: add delete_by_agent() for per-agent cleanup - reset() now cleans up knowledge graph data for the agent Chat memory indicators (#8): - Store memories_saved/memories_used in chat message objects - Add hasMemoryActivity() to detect proactive memory ops - Add memoryIndicatorForMsg() and memoryDetailItems() for display - Update HTML template to use proactive memory fields Inline edit (#5): - Add startEdit/cancelEdit/saveEdit methods to memory page - Double-click or edit button to inline-edit memory content - Enter to save, Escape to cancel
houko
added a commit
that referenced
this pull request
Mar 18, 2026
* Memory: implement mem0-style proactive memory system Implements issue #1106 with: - ProactiveMemory trait: search(), add(), add_with_level(), get(), list() - ProactiveMemoryHooks: auto_memorize(), auto_retrieve() - Multi-level memory: User, Session, Agent - Hook integration: BeforePromptBuild, AgentLoopEnd - Kernel integration for agent loop memory injection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: overhaul mem0 proactive memory core with LLM extraction, REST API, and bulk ops - Fix critical issues from PR review: dead hook handlers, double-store, empty return, streaming gap, cross-agent memory leakage - Add LLM-powered MemoryExtractor with robust JSON parser (code blocks, invalid input handling), falls back to rule-based DefaultMemoryExtractor - Add full mem0-style REST API (13 endpoints): search, list, add, update, delete, stats, per-agent operations, level-based cleanup, duplicate detection - Add bulk operations: reset (agent), clear_level (scope), cleanup_expired_sessions - Add semantic store primitives: forget_by_agent, forget_by_scope, forget_older_than, count - Integrate session memory auto-cleanup with configurable TTL (session_ttl_hours) - ProactiveMemoryConfig: add #[serde(default)], session_ttl_hours, config.toml docs - Kernel init: smart extractor selection based on extraction_model config - MemoryLevel: derive Default (clippy), add scope_str() centralizing scope mapping - ProactiveMemory::add() returns Vec<MemoryItem>, uses extractor pipeline - auto_memorize: substring-based dedup replacing exact-match only - stats(): efficient SQL COUNT + config status reporting - OnceLock replaces RwLock for set-once proactive_memory on kernel - Remove ~200 lines dead code (fire-and-forget hooks, unused session capture) - Add 12 new tests across semantic store, proactive store, and LLM parser * feat: add mem0-style memory conflict resolution (ADD/UPDATE/NOOP decision flow) - MemoryAction enum: Add, Update(existing_id), Noop for conflict decisions - MemoryExtractor::decide_action() trait method with default heuristic - LLM-powered decide_action() with dedicated conflict resolution prompt - Rewrite add() and auto_memorize() to use unified decision flow - SemanticStore::update_content() for in-place memory updates - Jaccard text similarity for same-category conflict detection - Fix kernel pm_config borrow error * feat: integrate knowledge graph with proactive memory + relation extraction - RelationTriple type for LLM-extracted entity relationships - ExtractionResult now includes relations alongside memories - LLM prompt extracts both memories and knowledge graph triples - store_relations() writes extracted triples to KnowledgeStore (upsert) - graph_context() queries knowledge graph for entity-relevant relations - format_context_with_query() merges semantic memories + graph context - build_prompt_context_with_memory() uses enriched context - Entity/relation type parsers (person, org, works_at, uses, etc.) - Expose MemorySubstrate::knowledge() for graph access * feat: semantic dedup, version history, knowledge graph tests, history API - find_duplicates() uses Jaccard similarity (>0.5) alongside substring matching - Memory version history: UPDATE preserves previous_content + version_history chain - history() method retrieves version chain for any memory - GET /api/memory/{id}/history endpoint for version history - SemanticStore::get_by_id() for single memory lookup - extract_search_keywords() for broader LIKE matching in decision flow - 10 new tests: dedup NOOP, conflicting UPDATE, version history, knowledge graph relations, semantic find_duplicates, text_similarity, entity/relation type parsing, new extraction format with relations, decision response parsing (ADD/UPDATE/NOOP) * feat: update() preserves history, rule-based relation extraction, session TTL, configurable dedup - update() now uses update_content() with version_history chain instead of delete+re-add - DefaultMemoryExtractor extracts basic relations: prefers, works_at, located_in, is_named - auto_retrieve() runs session TTL cleanup automatically before each retrieval - ProactiveMemoryConfig.duplicate_threshold: configurable similarity threshold (default 0.5) - search() enriches results with knowledge graph context when entities match - extract_after_pattern() / capitalize_first() helpers for rule-based extraction * feat: memory consolidation, graph query optimization, relation dedup, more tests - consolidate(): merge duplicate memory groups, soft-delete older entries - POST /api/memory/agents/{id}/consolidate endpoint - graph_context(): targeted entity lookups instead of loading all relations - extract_entity_candidates(): find proper nouns + "User" for graph queries - store_relations() deduplicates: skip existing (source, type, target) triples - KnowledgeStore::has_relation() for relation existence check - find_duplicates() falls back to semantic store when KV store is empty - DefaultMemoryExtractor: "I use X" / "I'm using X" pattern + uses relation - 7 new tests: update version history, relation extraction, relation dedup, consolidation, entity candidate extraction, "I use" pattern * fix: agent_loop auto_memorize now logs extraction results instead of discarding - Non-streaming and streaming paths both log stored memories + relations count - Relations were already being stored inside auto_memorize() via store_relations() - This was the last integration gap identified in the audit * feat: enhanced rule-based extractor, dashboard Memory tab, auto-consolidation - DefaultMemoryExtractor: 30+ patterns covering preferences, facts, tools, task context; extracts precise phrases instead of whole messages - push_memory() deduplicates within same extraction pass - New patterns: "I'm a/I am a", "we're building", "we're migrating to", "I code in", "our stack includes", "the problem is", "I'm debugging", etc. - Task context extraction (session-level) for debugging/fixing patterns - Dashboard: Memory tab under Agents nav section - memory.js: search, list, delete, version history viewer - webchat.rs: include memory.js in script bundle - Auto-consolidation: runs every 10 auto_memorize calls to merge duplicates - AtomicU32 counter with manual Clone impl for ProactiveMemoryStore * feat: integrate embedding driver for vector similarity search in proactive memory - EmbeddingFn trait in proactive.rs (bridge to avoid circular deps) - EmbeddingBridge wraps runtime EmbeddingDriver → EmbeddingFn - ProactiveMemoryStore.with_embedding() builder method - search() / auto_retrieve(): use cosine similarity when embedding available - add_with_decision(): store memories with embeddings for future vector search - init_proactive_memory_full(): unified init with optional LLM + embedding - init_proactive_memory_with_embedding(): convenience constructor - Kernel passes embedding_driver to proactive memory on init - Automatic fallback to LIKE text matching when no embedding driver * fix: resolve critical issues in proactive memory system - Fix route path conflict: /memory/{memory_id} → /memory/items/{memory_id} - Fix UTF-8 panic in truncate_for_log using char_indices - Replace nightly-only is_multiple_of with modulo operator - Write to KV store on add so get()/list() return results - Use nil UUID for default_user_id instead of unparseable "default" - Remove duplicate cleanup_expired_sessions calls in agent_loop - Send only new turn messages to auto_memorize, not full history - Remove duplicate system prompt in LlmMemoryExtractor requests * fix: resolve remaining issues in proactive memory system - Sync KV store on delete/reset/clear/update/consolidate/cleanup - Preserve embedding on update_content instead of nullifying - Per-agent consolidation counters instead of global atomic - extract_search_keywords returns all significant words joined by % - delete() returns false for non-existent memories - Cap find_duplicates to 100 most recent items - Update tests to verify KV store writes and delete behavior - Register all memory routes in OpenAPI spec with proactive-memory tag - Fix utoipa path param mismatch {agent_id} → {id} - Cap search limit to 100 - Add i18n support to Memory dashboard page (en + zh-CN) * fix: address third-round review issues in proactive memory - Validate level param in memory_clear_level, return 400 for invalid - Add stats_all/list_all/search_all for cross-agent dashboard queries - Resolve real agent_id for delete/update KV cleanup - Add KV store write in add_with_level for stats consistency - Combine double get_by_id into single call to fix TOCTOU race - Add badge-user/badge-session/badge-agent CSS classes - Implement after/before filter in semantic recall SQL - Escape LIKE wildcards in user search queries * fix: use single keyword in extract_search_keywords to avoid escape_like conflict The %‐joined multi-keyword output conflicted with escape_like which escapes % characters. Use the longest keyword instead; decide_action handles semantic dedup regardless. * fix: use semantic store MemoryId as KV key for ID consistency The semantic store generates its own UUID in remember(), while MemoryItem has a separate UUID. Using item.id as the KV key meant delete/update could not find the KV entry by the semantic ID. Now all KV keys use the MemoryId returned by remember(). * feat: enhance proactive memory UX and natural conversation integration - Add memories_saved/memories_used fields to AgentLoopResult and API response - Wire memory metadata through REST, WebSocket, and streaming paths - Improve memory injection prompt: agent uses recalled knowledge naturally without announcing "I remember..." — behaves like someone who knows you - Improve extraction prompt: prioritize communication style, frustrations, work preferences; write memories as natural observations with nuance - Add agent filter dropdown on Memory dashboard page - Add manual memory creation form (content + level) - Add memory hit indicator on chat page (brain emoji badge) - Add i18n keys for all new UI elements (en + zh-CN) * feat: add memory confidence decay, import/export, and conflict detection - Confidence decay: exponential decay based on days since last access, boosted by access frequency. Runs at most once per hour during auto_retrieve. - Import/export: GET /api/memory/agents/{id}/export and POST .../import for memory data migration (JSON format) - Conflict detection: when UPDATE replaces contradictory content, surface MemoryConflict in agent response with old vs new content - Dashboard: yellow conflict banner on chat responses, i18n (en + zh-CN) - Config: confidence_decay_rate (default 0.01/day) in ProactiveMemoryConfig * fix: always enable LLM extraction, fall back to default model Previously, LLM-powered memory extraction only activated when extraction_model was explicitly set in config. Now it defaults to the agent's main model, so extraction works out of the box. Users can still set extraction_model to a cheaper model (e.g., llama-3.3-70b) to reduce cost while keeping an expensive model for agent responses. * feat: add proactive memory settings to dashboard and config template - Add Proactive Memory tab to Settings page with toggles and inputs for auto_memorize, auto_retrieve, extraction_model, session_ttl, confidence_decay_rate, and duplicate_threshold - Expose proactive_memory config in GET /api/config and schema endpoint - Add [proactive_memory] section to config.toml generated by `librefang init` - Add i18n translations for all new settings (en + zh-CN) * fix: expose missing proactive memory config fields (max_retrieve, extraction_threshold, extract_categories) - Add max_retrieve, extraction_threshold, extract_categories to GET /api/config - Add same 3 fields to config/schema endpoint - Add UI inputs for all 3 fields in Settings > Proactive Memory tab - Add i18n translations (en + zh-CN) for the new fields * fix: repair smart quotes in zh-CN.json that broke JSON parsing Lines 756-765 used Unicode smart quotes (U+201C/U+201D) as JSON structural delimiters instead of ASCII double quotes. This caused JSON.parse() to fail, breaking the entire Chinese locale. * feat: expose all KernelConfig fields in GET /api/config and config/schema - get_config now returns all ~50 config sections with proper secret redaction (api_key, shared_secret, credentials_path, oauth client IDs) - config_schema now covers all sections for the dashboard dynamic form - Split giant json!() macros into per-section insert calls to avoid Rust recursion limit on macro expansion * fix: harden proactive memory system — bug fixes, TTL cleanup, pagination, memory cap Core fixes (proactive.rs): - Arc<Mutex> for shared Clone state (consolidation_counters, last_decay_run) - Ownership checks on delete/update operations - KV errors logged via tracing::warn instead of silently swallowed - Consolidation counter cleanup after threshold to prevent unbounded growth - TOCTOU race fix in confidence decay scheduling - text_similarity("","") returns 0.0 instead of 1.0 - Import deduplication (similarity > 0.9 skipped) Runtime fixes (agent_loop.rs, proactive_memory.rs): - Deduplicate proactive vs regular recall results - Filter system messages from LLM extraction input - Malformed JSON defaults to NOOP instead of ADD - UPDATE with invalid ID falls back to ADD instead of blindly updating first match - Robust code block stripping (case-insensitive, handles leading text) - stable_prefix_mode skips proactive recall for cache stability API fixes (memory.rs, openapi.rs): - POST /api/memory returns 201 Created - PUT /api/memory/items/{id} validates non-empty content - Memory list endpoint supports pagination (offset/limit) - Missing export/import endpoints added to OpenAPI spec - Restore ProactiveMemory trait import (was incorrectly removed) New features: - Session TTL cleanup: auto-expires session memories older than session_ttl_hours, rate-limited to once per hour, also exposed as POST /api/memory/cleanup - Per-agent memory cap (default 1000): evicts lowest-confidence memories when exceeded - Dashboard: paginated memory list, numeric input validation, removed non-functional level dropdown from add form, max_memories_per_agent setting exposed * fix: address logic review findings — pagination, eviction, cleanup, validation Bug 1: Agent-filtered pagination was broken — added GET /api/memory/agents/{id} list endpoint with offset/limit/total, frontend now uses it instead of abusing the search endpoint with empty query. Bug 2: Over-eviction on batch add — moved evict_if_over_cap to after the ADD/NOOP decision loop so NOOPs don't cause unnecessary eviction. Bug 3: UPDATE fallback creates duplicates — parse_decision_response now tries interpreting existing_id as a 1-based list index when UUID validation fails, matching the numbered format shown to the LLM. Bug 4: cleanup_expired KV orphans — collect agent_ids BEFORE soft-delete so agents whose only memories were expired sessions still get KV cleanup. Bug 5: Empty string → 0 for threshold fields — added empty-string validation before Number() conversions in settings.js. Bug 6: No integer enforcement — added Number.isInteger() checks for max_retrieve, session_ttl_hours, max_memories_per_agent. Also: evict_if_over_cap caps eviction at min(to_evict, current_count) and warns when new_count alone exceeds the cap. * fix: critical logic fixes — ADD action parsing, memory cap enforcement, locale dedup Critical: - parse_decision_response now case-insensitive (ADD/add/Add all work); previously ADD fell through to NOOP, silently dropping new memories - existing_id handles JSON number values (as_u64 fallback), not just strings - Conversation text capped at 8000 chars for LLM extraction to prevent context overflow and runaway costs - auto_memorize now enforces per-agent memory cap (was bypassed entirely) Logic fixes: - find_duplicates marks seed items as used to prevent multi-group assignment - extract_categories always sent on save (empty array clears; was silently preserving old value when cleared) - Removed duplicate i18n key blocks in en.json and zh-CN.json that caused extractionModelDesc to show wrong description - HTML max_retrieve input max=50 aligned with JS validation max=100 * fix: UTF-8 safe truncation and memory cap on all add paths - String::truncate at 8000 bytes could panic on multi-byte chars (Chinese/emoji); now finds nearest char boundary before truncating - add_with_level() and import_memories() were missing evict_if_over_cap() calls, allowing memory cap to be bypassed via these code paths * fix: improve proactive memory robustness and performance - Replace unwrap() with let-else in consolidation to prevent panic - Add count_by_category() SQL method to avoid loading all items for stats - Add migration v9 with composite indexes for recall/decay/eviction queries - Extract maybe_run_maintenance() called from search/consolidate/auto_retrieve - Implement metadata filtering via json_extract in semantic recall - Sanitize error responses to prevent internal info leakage (20 endpoints) - Cap consolidation_counters HashMap to prevent unbounded growth - Log JSON serialization failures during export instead of silent swallow - Fix frontend pagination: reset offset on error, clamp after load * fix: correct SQL alias in WHERE clause and sanitize metadata keys - Replace column alias 'cat' in WHERE with full json_extract() expression (SQLite does not support aliases in WHERE clauses) - Restrict metadata filter keys to alphanumeric/underscore to prevent SQL injection via crafted JSON paths * fix: clarify consolidation winner selection comment * fix: critical bugs found in full PR review - Add 'memory' to validPages and remove redirect to 'sessions' in app.js (memory page was completely unreachable from UI navigation) - Add Array arm to json_to_toml_value so extract_categories saves correctly (was serialized as string "[...]" instead of TOML array) - Filter empty extraction_model string as None to fall back to default model (empty string from settings UI caused LLM API failures) * fix: align CLI template defaults and settings.js with actual config defaults - CLI template: duplicate_threshold 0.85→0.5, confidence_decay_rate 0.05→0.01 - settings.js: max_retrieve default 5→10 to match ProactiveMemoryConfig::default() * fix: address proactive memory logic gaps - Clean up proactive memories when agent is killed (kernel.rs) - Fix KV store ID mismatch by syncing with semantic store ID (proactive.rs) - Re-embed content on memory update so vector search stays accurate (proactive.rs) * feat: hot-reload proactive memory config at runtime - Add UpdateProactiveMemory hot action to config reload system - Change ProactiveMemoryStore.config to Arc<RwLock<>> for runtime updates - Add update_config() method for hot-swapping config without restart - Wire reload through kernel apply_hot_actions * feat: add master enabled toggle for proactive memory - Add enabled field to ProactiveMemoryConfig (defaults to true) - Skip proactive memory init when disabled in kernel boot - Check enabled flag in auto_memorize and auto_retrieve hooks - Expose toggle in config API, schema, settings UI, and CLI template - Fix max_retrieve default from 5 to 10 in CLI templates * fix: include enabled field in MemoryStats API response * feat: add consolidate and cleanup buttons to memory page - Add consolidateAgent() and cleanupExpired() actions to memory.js - Add Consolidate and Cleanup Expired buttons in memory page toolbar - Consolidate button only shows when an agent is selected * feat: add bulk delete for memories - Add POST /api/memory/bulk-delete endpoint accepting { ids: [...] } - Register route in server.rs - Add selection checkboxes to memory table with select-all header - Add bulk delete button (shown when items selected) * fix: clear selection when reloading memory list * feat: knowledge graph cleanup, chat memory indicators, inline edit Knowledge graph cleanup (#3): - Migration v10: add agent_id column to entities and relations tables - KnowledgeStore: add_entity/add_relation now track agent_id - KnowledgeStore: add delete_by_agent() for per-agent cleanup - reset() now cleans up knowledge graph data for the agent Chat memory indicators (#8): - Store memories_saved/memories_used in chat message objects - Add hasMemoryActivity() to detect proactive memory ops - Add memoryIndicatorForMsg() and memoryDetailItems() for display - Update HTML template to use proactive memory fields Inline edit (#5): - Add startEdit/cancelEdit/saveEdit methods to memory page - Double-click or edit button to inline-edit memory content - Enter to save, Escape to cancel * feat: vector cosine similarity dedup + VectorStore trait for backend extensibility - Upgrade find_duplicates() to use vector cosine similarity when stored embeddings are available (mem0-quality dedup), falling back to Jaccard word overlap when no embeddings exist - Enhance default decide_action() heuristic with tiered similarity: vector cosine > substring containment > Jaccard, including high-similarity NOOP detection (≥0.95) and best-candidate UPDATE selection - Add VectorStore trait in librefang-types — backend-agnostic abstraction enabling pluggable vector databases (Qdrant, Pinecone, Chroma, PgVector) - Implement SqliteVectorStore as the default backend with BLOB embeddings and in-process cosine similarity re-ranking - Add cosine_similarity() to types crate as shared utility - Add SemanticStore::get_embeddings_batch() for efficient bulk embedding load - Stash query embedding in _embedding metadata key during decision flow so default heuristic can leverage it without breaking storage paths * feat: add missing API endpoints for full memory management coverage - POST /api/memory/decay — manual confidence decay trigger - GET /api/memory/agents/{id}/count?level= — per-agent memory count - POST /api/memory/agents/{id}/relations — store knowledge graph triples - GET /api/memory/agents/{id}/relations — query knowledge graph - Add ProactiveMemoryStore::query_relations() wrapper for graph queries - Add Knowledge Graph UI panel in memory page (toggle Relations button) - Add Decay Confidence button to memory toolbar - Add loadRelations() and decayConfidence() JS functions --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
houko
added a commit
that referenced
this pull request
Mar 18, 2026
* Memory: implement mem0-style proactive memory system Implements issue #1106 with: - ProactiveMemory trait: search(), add(), add_with_level(), get(), list() - ProactiveMemoryHooks: auto_memorize(), auto_retrieve() - Multi-level memory: User, Session, Agent - Hook integration: BeforePromptBuild, AgentLoopEnd - Kernel integration for agent loop memory injection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: overhaul mem0 proactive memory core with LLM extraction, REST API, and bulk ops - Fix critical issues from PR review: dead hook handlers, double-store, empty return, streaming gap, cross-agent memory leakage - Add LLM-powered MemoryExtractor with robust JSON parser (code blocks, invalid input handling), falls back to rule-based DefaultMemoryExtractor - Add full mem0-style REST API (13 endpoints): search, list, add, update, delete, stats, per-agent operations, level-based cleanup, duplicate detection - Add bulk operations: reset (agent), clear_level (scope), cleanup_expired_sessions - Add semantic store primitives: forget_by_agent, forget_by_scope, forget_older_than, count - Integrate session memory auto-cleanup with configurable TTL (session_ttl_hours) - ProactiveMemoryConfig: add #[serde(default)], session_ttl_hours, config.toml docs - Kernel init: smart extractor selection based on extraction_model config - MemoryLevel: derive Default (clippy), add scope_str() centralizing scope mapping - ProactiveMemory::add() returns Vec<MemoryItem>, uses extractor pipeline - auto_memorize: substring-based dedup replacing exact-match only - stats(): efficient SQL COUNT + config status reporting - OnceLock replaces RwLock for set-once proactive_memory on kernel - Remove ~200 lines dead code (fire-and-forget hooks, unused session capture) - Add 12 new tests across semantic store, proactive store, and LLM parser * feat: add mem0-style memory conflict resolution (ADD/UPDATE/NOOP decision flow) - MemoryAction enum: Add, Update(existing_id), Noop for conflict decisions - MemoryExtractor::decide_action() trait method with default heuristic - LLM-powered decide_action() with dedicated conflict resolution prompt - Rewrite add() and auto_memorize() to use unified decision flow - SemanticStore::update_content() for in-place memory updates - Jaccard text similarity for same-category conflict detection - Fix kernel pm_config borrow error * feat: integrate knowledge graph with proactive memory + relation extraction - RelationTriple type for LLM-extracted entity relationships - ExtractionResult now includes relations alongside memories - LLM prompt extracts both memories and knowledge graph triples - store_relations() writes extracted triples to KnowledgeStore (upsert) - graph_context() queries knowledge graph for entity-relevant relations - format_context_with_query() merges semantic memories + graph context - build_prompt_context_with_memory() uses enriched context - Entity/relation type parsers (person, org, works_at, uses, etc.) - Expose MemorySubstrate::knowledge() for graph access * feat: semantic dedup, version history, knowledge graph tests, history API - find_duplicates() uses Jaccard similarity (>0.5) alongside substring matching - Memory version history: UPDATE preserves previous_content + version_history chain - history() method retrieves version chain for any memory - GET /api/memory/{id}/history endpoint for version history - SemanticStore::get_by_id() for single memory lookup - extract_search_keywords() for broader LIKE matching in decision flow - 10 new tests: dedup NOOP, conflicting UPDATE, version history, knowledge graph relations, semantic find_duplicates, text_similarity, entity/relation type parsing, new extraction format with relations, decision response parsing (ADD/UPDATE/NOOP) * feat: update() preserves history, rule-based relation extraction, session TTL, configurable dedup - update() now uses update_content() with version_history chain instead of delete+re-add - DefaultMemoryExtractor extracts basic relations: prefers, works_at, located_in, is_named - auto_retrieve() runs session TTL cleanup automatically before each retrieval - ProactiveMemoryConfig.duplicate_threshold: configurable similarity threshold (default 0.5) - search() enriches results with knowledge graph context when entities match - extract_after_pattern() / capitalize_first() helpers for rule-based extraction * feat: memory consolidation, graph query optimization, relation dedup, more tests - consolidate(): merge duplicate memory groups, soft-delete older entries - POST /api/memory/agents/{id}/consolidate endpoint - graph_context(): targeted entity lookups instead of loading all relations - extract_entity_candidates(): find proper nouns + "User" for graph queries - store_relations() deduplicates: skip existing (source, type, target) triples - KnowledgeStore::has_relation() for relation existence check - find_duplicates() falls back to semantic store when KV store is empty - DefaultMemoryExtractor: "I use X" / "I'm using X" pattern + uses relation - 7 new tests: update version history, relation extraction, relation dedup, consolidation, entity candidate extraction, "I use" pattern * fix: agent_loop auto_memorize now logs extraction results instead of discarding - Non-streaming and streaming paths both log stored memories + relations count - Relations were already being stored inside auto_memorize() via store_relations() - This was the last integration gap identified in the audit * feat: enhanced rule-based extractor, dashboard Memory tab, auto-consolidation - DefaultMemoryExtractor: 30+ patterns covering preferences, facts, tools, task context; extracts precise phrases instead of whole messages - push_memory() deduplicates within same extraction pass - New patterns: "I'm a/I am a", "we're building", "we're migrating to", "I code in", "our stack includes", "the problem is", "I'm debugging", etc. - Task context extraction (session-level) for debugging/fixing patterns - Dashboard: Memory tab under Agents nav section - memory.js: search, list, delete, version history viewer - webchat.rs: include memory.js in script bundle - Auto-consolidation: runs every 10 auto_memorize calls to merge duplicates - AtomicU32 counter with manual Clone impl for ProactiveMemoryStore * feat: integrate embedding driver for vector similarity search in proactive memory - EmbeddingFn trait in proactive.rs (bridge to avoid circular deps) - EmbeddingBridge wraps runtime EmbeddingDriver → EmbeddingFn - ProactiveMemoryStore.with_embedding() builder method - search() / auto_retrieve(): use cosine similarity when embedding available - add_with_decision(): store memories with embeddings for future vector search - init_proactive_memory_full(): unified init with optional LLM + embedding - init_proactive_memory_with_embedding(): convenience constructor - Kernel passes embedding_driver to proactive memory on init - Automatic fallback to LIKE text matching when no embedding driver * fix: resolve critical issues in proactive memory system - Fix route path conflict: /memory/{memory_id} → /memory/items/{memory_id} - Fix UTF-8 panic in truncate_for_log using char_indices - Replace nightly-only is_multiple_of with modulo operator - Write to KV store on add so get()/list() return results - Use nil UUID for default_user_id instead of unparseable "default" - Remove duplicate cleanup_expired_sessions calls in agent_loop - Send only new turn messages to auto_memorize, not full history - Remove duplicate system prompt in LlmMemoryExtractor requests * fix: resolve remaining issues in proactive memory system - Sync KV store on delete/reset/clear/update/consolidate/cleanup - Preserve embedding on update_content instead of nullifying - Per-agent consolidation counters instead of global atomic - extract_search_keywords returns all significant words joined by % - delete() returns false for non-existent memories - Cap find_duplicates to 100 most recent items - Update tests to verify KV store writes and delete behavior - Register all memory routes in OpenAPI spec with proactive-memory tag - Fix utoipa path param mismatch {agent_id} → {id} - Cap search limit to 100 - Add i18n support to Memory dashboard page (en + zh-CN) * fix: address third-round review issues in proactive memory - Validate level param in memory_clear_level, return 400 for invalid - Add stats_all/list_all/search_all for cross-agent dashboard queries - Resolve real agent_id for delete/update KV cleanup - Add KV store write in add_with_level for stats consistency - Combine double get_by_id into single call to fix TOCTOU race - Add badge-user/badge-session/badge-agent CSS classes - Implement after/before filter in semantic recall SQL - Escape LIKE wildcards in user search queries * fix: use single keyword in extract_search_keywords to avoid escape_like conflict The %‐joined multi-keyword output conflicted with escape_like which escapes % characters. Use the longest keyword instead; decide_action handles semantic dedup regardless. * fix: use semantic store MemoryId as KV key for ID consistency The semantic store generates its own UUID in remember(), while MemoryItem has a separate UUID. Using item.id as the KV key meant delete/update could not find the KV entry by the semantic ID. Now all KV keys use the MemoryId returned by remember(). * feat: enhance proactive memory UX and natural conversation integration - Add memories_saved/memories_used fields to AgentLoopResult and API response - Wire memory metadata through REST, WebSocket, and streaming paths - Improve memory injection prompt: agent uses recalled knowledge naturally without announcing "I remember..." — behaves like someone who knows you - Improve extraction prompt: prioritize communication style, frustrations, work preferences; write memories as natural observations with nuance - Add agent filter dropdown on Memory dashboard page - Add manual memory creation form (content + level) - Add memory hit indicator on chat page (brain emoji badge) - Add i18n keys for all new UI elements (en + zh-CN) * feat: add memory confidence decay, import/export, and conflict detection - Confidence decay: exponential decay based on days since last access, boosted by access frequency. Runs at most once per hour during auto_retrieve. - Import/export: GET /api/memory/agents/{id}/export and POST .../import for memory data migration (JSON format) - Conflict detection: when UPDATE replaces contradictory content, surface MemoryConflict in agent response with old vs new content - Dashboard: yellow conflict banner on chat responses, i18n (en + zh-CN) - Config: confidence_decay_rate (default 0.01/day) in ProactiveMemoryConfig * fix: always enable LLM extraction, fall back to default model Previously, LLM-powered memory extraction only activated when extraction_model was explicitly set in config. Now it defaults to the agent's main model, so extraction works out of the box. Users can still set extraction_model to a cheaper model (e.g., llama-3.3-70b) to reduce cost while keeping an expensive model for agent responses. * feat: add proactive memory settings to dashboard and config template - Add Proactive Memory tab to Settings page with toggles and inputs for auto_memorize, auto_retrieve, extraction_model, session_ttl, confidence_decay_rate, and duplicate_threshold - Expose proactive_memory config in GET /api/config and schema endpoint - Add [proactive_memory] section to config.toml generated by `librefang init` - Add i18n translations for all new settings (en + zh-CN) * fix: expose missing proactive memory config fields (max_retrieve, extraction_threshold, extract_categories) - Add max_retrieve, extraction_threshold, extract_categories to GET /api/config - Add same 3 fields to config/schema endpoint - Add UI inputs for all 3 fields in Settings > Proactive Memory tab - Add i18n translations (en + zh-CN) for the new fields * fix: repair smart quotes in zh-CN.json that broke JSON parsing Lines 756-765 used Unicode smart quotes (U+201C/U+201D) as JSON structural delimiters instead of ASCII double quotes. This caused JSON.parse() to fail, breaking the entire Chinese locale. * feat: expose all KernelConfig fields in GET /api/config and config/schema - get_config now returns all ~50 config sections with proper secret redaction (api_key, shared_secret, credentials_path, oauth client IDs) - config_schema now covers all sections for the dashboard dynamic form - Split giant json!() macros into per-section insert calls to avoid Rust recursion limit on macro expansion * fix: harden proactive memory system — bug fixes, TTL cleanup, pagination, memory cap Core fixes (proactive.rs): - Arc<Mutex> for shared Clone state (consolidation_counters, last_decay_run) - Ownership checks on delete/update operations - KV errors logged via tracing::warn instead of silently swallowed - Consolidation counter cleanup after threshold to prevent unbounded growth - TOCTOU race fix in confidence decay scheduling - text_similarity("","") returns 0.0 instead of 1.0 - Import deduplication (similarity > 0.9 skipped) Runtime fixes (agent_loop.rs, proactive_memory.rs): - Deduplicate proactive vs regular recall results - Filter system messages from LLM extraction input - Malformed JSON defaults to NOOP instead of ADD - UPDATE with invalid ID falls back to ADD instead of blindly updating first match - Robust code block stripping (case-insensitive, handles leading text) - stable_prefix_mode skips proactive recall for cache stability API fixes (memory.rs, openapi.rs): - POST /api/memory returns 201 Created - PUT /api/memory/items/{id} validates non-empty content - Memory list endpoint supports pagination (offset/limit) - Missing export/import endpoints added to OpenAPI spec - Restore ProactiveMemory trait import (was incorrectly removed) New features: - Session TTL cleanup: auto-expires session memories older than session_ttl_hours, rate-limited to once per hour, also exposed as POST /api/memory/cleanup - Per-agent memory cap (default 1000): evicts lowest-confidence memories when exceeded - Dashboard: paginated memory list, numeric input validation, removed non-functional level dropdown from add form, max_memories_per_agent setting exposed * fix: address logic review findings — pagination, eviction, cleanup, validation Bug 1: Agent-filtered pagination was broken — added GET /api/memory/agents/{id} list endpoint with offset/limit/total, frontend now uses it instead of abusing the search endpoint with empty query. Bug 2: Over-eviction on batch add — moved evict_if_over_cap to after the ADD/NOOP decision loop so NOOPs don't cause unnecessary eviction. Bug 3: UPDATE fallback creates duplicates — parse_decision_response now tries interpreting existing_id as a 1-based list index when UUID validation fails, matching the numbered format shown to the LLM. Bug 4: cleanup_expired KV orphans — collect agent_ids BEFORE soft-delete so agents whose only memories were expired sessions still get KV cleanup. Bug 5: Empty string → 0 for threshold fields — added empty-string validation before Number() conversions in settings.js. Bug 6: No integer enforcement — added Number.isInteger() checks for max_retrieve, session_ttl_hours, max_memories_per_agent. Also: evict_if_over_cap caps eviction at min(to_evict, current_count) and warns when new_count alone exceeds the cap. * fix: critical logic fixes — ADD action parsing, memory cap enforcement, locale dedup Critical: - parse_decision_response now case-insensitive (ADD/add/Add all work); previously ADD fell through to NOOP, silently dropping new memories - existing_id handles JSON number values (as_u64 fallback), not just strings - Conversation text capped at 8000 chars for LLM extraction to prevent context overflow and runaway costs - auto_memorize now enforces per-agent memory cap (was bypassed entirely) Logic fixes: - find_duplicates marks seed items as used to prevent multi-group assignment - extract_categories always sent on save (empty array clears; was silently preserving old value when cleared) - Removed duplicate i18n key blocks in en.json and zh-CN.json that caused extractionModelDesc to show wrong description - HTML max_retrieve input max=50 aligned with JS validation max=100 * fix: UTF-8 safe truncation and memory cap on all add paths - String::truncate at 8000 bytes could panic on multi-byte chars (Chinese/emoji); now finds nearest char boundary before truncating - add_with_level() and import_memories() were missing evict_if_over_cap() calls, allowing memory cap to be bypassed via these code paths * fix: improve proactive memory robustness and performance - Replace unwrap() with let-else in consolidation to prevent panic - Add count_by_category() SQL method to avoid loading all items for stats - Add migration v9 with composite indexes for recall/decay/eviction queries - Extract maybe_run_maintenance() called from search/consolidate/auto_retrieve - Implement metadata filtering via json_extract in semantic recall - Sanitize error responses to prevent internal info leakage (20 endpoints) - Cap consolidation_counters HashMap to prevent unbounded growth - Log JSON serialization failures during export instead of silent swallow - Fix frontend pagination: reset offset on error, clamp after load * fix: correct SQL alias in WHERE clause and sanitize metadata keys - Replace column alias 'cat' in WHERE with full json_extract() expression (SQLite does not support aliases in WHERE clauses) - Restrict metadata filter keys to alphanumeric/underscore to prevent SQL injection via crafted JSON paths * fix: clarify consolidation winner selection comment * fix: critical bugs found in full PR review - Add 'memory' to validPages and remove redirect to 'sessions' in app.js (memory page was completely unreachable from UI navigation) - Add Array arm to json_to_toml_value so extract_categories saves correctly (was serialized as string "[...]" instead of TOML array) - Filter empty extraction_model string as None to fall back to default model (empty string from settings UI caused LLM API failures) * fix: align CLI template defaults and settings.js with actual config defaults - CLI template: duplicate_threshold 0.85→0.5, confidence_decay_rate 0.05→0.01 - settings.js: max_retrieve default 5→10 to match ProactiveMemoryConfig::default() * fix: address proactive memory logic gaps - Clean up proactive memories when agent is killed (kernel.rs) - Fix KV store ID mismatch by syncing with semantic store ID (proactive.rs) - Re-embed content on memory update so vector search stays accurate (proactive.rs) * feat: hot-reload proactive memory config at runtime - Add UpdateProactiveMemory hot action to config reload system - Change ProactiveMemoryStore.config to Arc<RwLock<>> for runtime updates - Add update_config() method for hot-swapping config without restart - Wire reload through kernel apply_hot_actions * feat: add master enabled toggle for proactive memory - Add enabled field to ProactiveMemoryConfig (defaults to true) - Skip proactive memory init when disabled in kernel boot - Check enabled flag in auto_memorize and auto_retrieve hooks - Expose toggle in config API, schema, settings UI, and CLI template - Fix max_retrieve default from 5 to 10 in CLI templates * fix: include enabled field in MemoryStats API response * feat: add consolidate and cleanup buttons to memory page - Add consolidateAgent() and cleanupExpired() actions to memory.js - Add Consolidate and Cleanup Expired buttons in memory page toolbar - Consolidate button only shows when an agent is selected * feat: add bulk delete for memories - Add POST /api/memory/bulk-delete endpoint accepting { ids: [...] } - Register route in server.rs - Add selection checkboxes to memory table with select-all header - Add bulk delete button (shown when items selected) * fix: clear selection when reloading memory list * feat: knowledge graph cleanup, chat memory indicators, inline edit Knowledge graph cleanup (#3): - Migration v10: add agent_id column to entities and relations tables - KnowledgeStore: add_entity/add_relation now track agent_id - KnowledgeStore: add delete_by_agent() for per-agent cleanup - reset() now cleans up knowledge graph data for the agent Chat memory indicators (#8): - Store memories_saved/memories_used in chat message objects - Add hasMemoryActivity() to detect proactive memory ops - Add memoryIndicatorForMsg() and memoryDetailItems() for display - Update HTML template to use proactive memory fields Inline edit (#5): - Add startEdit/cancelEdit/saveEdit methods to memory page - Double-click or edit button to inline-edit memory content - Enter to save, Escape to cancel * feat: vector cosine similarity dedup + VectorStore trait for backend extensibility - Upgrade find_duplicates() to use vector cosine similarity when stored embeddings are available (mem0-quality dedup), falling back to Jaccard word overlap when no embeddings exist - Enhance default decide_action() heuristic with tiered similarity: vector cosine > substring containment > Jaccard, including high-similarity NOOP detection (≥0.95) and best-candidate UPDATE selection - Add VectorStore trait in librefang-types — backend-agnostic abstraction enabling pluggable vector databases (Qdrant, Pinecone, Chroma, PgVector) - Implement SqliteVectorStore as the default backend with BLOB embeddings and in-process cosine similarity re-ranking - Add cosine_similarity() to types crate as shared utility - Add SemanticStore::get_embeddings_batch() for efficient bulk embedding load - Stash query embedding in _embedding metadata key during decision flow so default heuristic can leverage it without breaking storage paths * feat: add missing API endpoints for full memory management coverage - POST /api/memory/decay — manual confidence decay trigger - GET /api/memory/agents/{id}/count?level= — per-agent memory count - POST /api/memory/agents/{id}/relations — store knowledge graph triples - GET /api/memory/agents/{id}/relations — query knowledge graph - Add ProactiveMemoryStore::query_relations() wrapper for graph queries - Add Knowledge Graph UI panel in memory page (toggle Relations button) - Add Decay Confidence button to memory toolbar - Add loadRelations() and decayConfidence() JS functions --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
houko
added a commit
that referenced
this pull request
Apr 18, 2026
…ation Fifth Codex batch on #2704 — covers the skill-evolution subsystem (code the upstream PR #2694 shipped to main but that shows up in this PR's diff), plus the KV race on worker telemetry writes. librefang-skills (evolution.rs): - fuzzy_find_and_replace rejects empty old_string up front (#16). - update_skill / patch_skill re-verify skill_dir.exists() under the lock so concurrent delete_skill can't be resurrected (#3, #5). - delete_skill runs validate_name to reject path traversal (#12). - remove_supporting_file canonicalises + contains-checks the target (#6), matching write_supporting_file. - write_supporting_file scans content BEFORE writing (#7) so a rejected update doesn't destroy the pre-existing valid file. librefang-runtime (tool_runner.rs): - tool_skill_evolve_delete resolves the real installed skill's parent dir via registry.get() (#15) instead of unconditionally targeting the global skills_dir. librefang-types / librefang-kernel (approval policy): - Default require_approval list includes skill_evolve_* (#14). Updated serde `true`-shorthand + tests to match. librefang-kernel (kernel/mod.rs): - build_skill_summary sanitises the category key (#1) — tags are third-party data. - extract_json_from_llm_response tries every '{' instead of giving up after the first (#8). - Background skill review uses the agent's resolved driver and model for cost attribution (#13); falls back to defaults only when manifest resolution fails. worker (github-stats-worker/index.js): - Click counts sharded across 8 blobs, unioned on read (#39). - UI error reports sharded across 4 blobs plus legacy-key fallback for historical data (#41).
houko
added a commit
that referenced
this pull request
Apr 20, 2026
…oard parity, smoke script Six follow-ups bundled per request to keep the channel-progress effort in one PR. Done from #2 onward; #1 (per-OutputFormat backtick adaptation) was investigated and dropped — backtick renders correctly in TelegramHtml (<code>), SlackMrkdwn, Discord/Matrix Markdown (inline code), and the only adapter that strips backticks (PlainText for Mastodon) is already opted out of the progress path via suppress_error_responses. #2 buffered_text fallback test: new integration test test_bridge_streaming_adapter_kernel_and_transport_both_fail covers the V2 4th outcome (send_streaming Err + kernel Err). #3 surface context_warning PhaseChange so the user sees when the agent's context window was trimmed/overflowed. Other phases stay SSE-only. #4 collapse repeated tool calls within an iteration — replace last_progress_tool with iter_tools_seen HashSet cleared at every ContentComplete. Batch agents no longer spam "🔧 web_search" per parallel call; retries in a later iteration still get a fresh line. #5 i18n for the "failed" suffix — supports en/zh-CN/es/ja/de/fr (matches librefang_types::i18n). Language threaded from kernel.config_snapshot().language through start_stream_text_bridge. #6 prettify tool names — web_search → Web Search; MCP_call → MCP Call (preserves internal caps). Backticks dropped from progress lines since the prettified form is now the identity. #8 dashboard ToolCallCard parity — mirror prettifyToolName() in dashboard/src/lib/string.ts and apply it in the card header so chat reply and dashboard show the same rendering. #7 live-integration smoke script — scripts/tests/channel_progress_smoke.sh automates the daemon-side flow once the user supplies an LLM API key and a configured channel adapter. Documents the channel-delivery gap (needs an external webhook receiver). No driver injection hook added to the kernel — that would have been scope creep. Tests added/updated: - test_prettify_tool_name_snake_to_title / _kebab_and_dotted / _preserves_internal_caps - test_tr_progress_failed_languages - test_bridge_streaming_adapter_kernel_and_transport_both_fail - Updated existing assertions to expect prettified names
houko
added a commit
that referenced
this pull request
Apr 20, 2026
…m fix, show_progress, i18n, prettify, dashboard parity (#2793) * feat(channels): channel-progress v2 — Telegram fallback fix, show_progress config, integration test Three follow-ups to #2792 (channel progress markers): 1. Telegram (streaming-adapter) buffered_text fallback now uses the `_status` variant. The same regression class fixed for non-streaming adapters in V1 was still latent on the Telegram path: - send_streaming Ok + kernel Err → Done reaction + success=true - send_streaming Err + buffered → Done reaction + success=true Both now correctly emit Error reaction, record_delivery(false), and journal Failed when the kernel actually errored. The fallback path also honors `suppress_error_responses` when the buffered text is a sanitized error (defense-in-depth — Telegram is not in the opt-in list today, but the path is now uniform across all adapters). 2. New `agent.toml show_progress` field (default true) plumbed through the streaming bridge. When false, the bridge skips both the `🔧 tool_name` and `⚠️ tool_name failed` injections — useful for agents whose output is parsed downstream, or for pristine-output scenarios where status markers would leak into the response. The trait-impl side looks up the agent's manifest from the kernel registry once per dispatch and passes it through to `start_stream_text_bridge[_with_status]`. 3. New end-to-end integration test in `crates/librefang-channels/tests/bridge_integration_test.rs` that wires a real `BridgeManager` + `MockAdapter` (non-streaming) + `MockProgressHandle` (override of `send_message_streaming_with_sender_status` that synthesises a delta stream with progress markers). Verifies the V2 dispatch pipeline actually surfaces progress to non-streaming adapters end-to-end via the consolidated `send_response` call. Tests: - test_stream_bridge_show_progress_false_suppresses_all_markers: show_progress=false produces no 🔧/⚠️ markers but still flows the actual model prose through - test_bridge_non_streaming_adapter_sees_progress_markers: full BridgeManager → MockAdapter pipeline; verifies progress markers end up in the captured `send()` call Notes: - True "live daemon" integration test (per CLAUDE.md) requires LLM API keys + a configured channel adapter (Telegram bot token, etc.) which the daemon environment does not currently have. The in-process integration test exercises the full dispatch wiring using real tokio tasks/channels and a mock kernel handle. * fix(kernel/wizard): add show_progress to AgentManifest struct literal CI compile error: wizard.rs constructed AgentManifest with an explicit struct literal listing every field, so adding show_progress in V2 broke that one site (the wizard agent created via setup intent). All other AgentManifest constructors use ..Default::default() and pick up show_progress=true automatically; only this one was a full-literal. * feat(channels): channel-progress v3 — collapse, i18n, prettify, dashboard parity, smoke script Six follow-ups bundled per request to keep the channel-progress effort in one PR. Done from #2 onward; #1 (per-OutputFormat backtick adaptation) was investigated and dropped — backtick renders correctly in TelegramHtml (<code>), SlackMrkdwn, Discord/Matrix Markdown (inline code), and the only adapter that strips backticks (PlainText for Mastodon) is already opted out of the progress path via suppress_error_responses. #2 buffered_text fallback test: new integration test test_bridge_streaming_adapter_kernel_and_transport_both_fail covers the V2 4th outcome (send_streaming Err + kernel Err). #3 surface context_warning PhaseChange so the user sees when the agent's context window was trimmed/overflowed. Other phases stay SSE-only. #4 collapse repeated tool calls within an iteration — replace last_progress_tool with iter_tools_seen HashSet cleared at every ContentComplete. Batch agents no longer spam "🔧 web_search" per parallel call; retries in a later iteration still get a fresh line. #5 i18n for the "failed" suffix — supports en/zh-CN/es/ja/de/fr (matches librefang_types::i18n). Language threaded from kernel.config_snapshot().language through start_stream_text_bridge. #6 prettify tool names — web_search → Web Search; MCP_call → MCP Call (preserves internal caps). Backticks dropped from progress lines since the prettified form is now the identity. #8 dashboard ToolCallCard parity — mirror prettifyToolName() in dashboard/src/lib/string.ts and apply it in the card header so chat reply and dashboard show the same rendering. #7 live-integration smoke script — scripts/tests/channel_progress_smoke.sh automates the daemon-side flow once the user supplies an LLM API key and a configured channel adapter. Documents the channel-delivery gap (needs an external webhook receiver). No driver injection hook added to the kernel — that would have been scope creep. Tests added/updated: - test_prettify_tool_name_snake_to_title / _kebab_and_dotted / _preserves_internal_caps - test_tr_progress_failed_languages - test_bridge_streaming_adapter_kernel_and_transport_both_fail - Updated existing assertions to expect prettified names * fix(channels): correct success/err pairing + treat timeout as soft success + clippy collapsible_if Three review-driven fixes for the V3 PR: Bug 1 — Telegram path outcome 3 (send_streaming Err + kernel Ok) was recording delivery as success=true with err=Some(stream_error). The fallback send_response had already delivered the buffered text and the kernel succeeded, so the transport-side stream error is irrelevant to delivery accounting — keeping it in the err field produced a contradictory metric (success AND err). Now: err is Some only when kernel actually failed. Bug 2 — TIMEOUT_PARTIAL_OUTPUT_MARKER was being mapped to status=Err, which flipped the lifecycle reaction to Error and record_delivery to success=false. Pre-V2 the bridge had no status channel and treated these turns as Done because the model emitted useful partial output before the inactivity timer fired. Restore that semantics: status=Ok for timeouts. The user still sees the "[Task timed out…]" tail appended to their reply. Clippy collapsible_if — flatten the inner if show_progress && ... into match guards on the ToolExecutionResult and PhaseChange arms (CI clippy -D warnings caught this in run 24651228831). * chore(channels): drop dead 'let _ = e' annotation The variable 'e' (stream transport error) is already used at warn! line 2886 and the empty-buffer fallback at line 2953, so the explicit underscore-binding introduced by the Bug 1 fix is noise — clean it up. * test(channels): regression coverage for Bug 1 (success/err pairing) + Bug 2 (timeout-as-success) Both bugs were caught by review, not tests — add the missing coverage so they cannot silently regress. Bug 2 unit test: test_stream_bridge_timeout_partial_output_reports_ok_status Constructs a kernel handle that fails with a string containing TIMEOUT_PARTIAL_OUTPUT_MARKER. Asserts: - The user-facing text channel still receives the "[Task timed out…]" tail so the user knows the reply may be incomplete. - The status oneshot resolves to Ok(()) — NOT Err — so bridge.rs drives the lifecycle reaction to Done and record_delivery to success=true. Pre-V2 semantics preserved. Bug 1 integration test: test_bridge_streaming_adapter_kernel_ok_transport_fail_records_clean_success Adds MockKernelOkHandle (overrides send_message_streaming_with_sender_status to emit clean text + status=Ok, AND overrides record_delivery to capture every (success, err) pair). Combined with the existing MockFailingStreamingAdapter (always Err on send_streaming), the test exercises Telegram outcome 3: - Fallback send_response delivers the buffered text ✓ - record_delivery is called with success=true AND err=None — proving the transport-side stream error does NOT leak into the err field when the kernel itself succeeded. (Pre-fix this would have been success=true, err=Some(stream_e) — a contradictory metric.) * polish(channels): unify progress-line spacing, codepoint-safe dashboard prettify, smoke script auto-spawn Three minor follow-ups from review: #3 — Symmetric blank lines around progress markers. ToolUseStart used \n\n…\n; ToolExecutionResult and PhaseChange used \n…\n. Adjacent markers (e.g. 🔧 X right before⚠️ X failed) ended up on consecutive lines without a blank separator and many markdown renderers collapsed them into one paragraph. All three now use \n\n…\n\n so every renderer that respects markdown blank-line semantics shows them as separate blocks. #4 — prettifyToolName() in dashboard/src/lib/string.ts now iterates by Unicode codepoint (via spread) instead of UTF-16 unit. A tool name starting with a non-BMP character (e.g. emoji) no longer drops its surrogate half when uppercased. Mirrors the Rust-side prettifier in channel_bridge.rs which uses chars().next() (also codepoint-correct). Tool names are usually ASCII so this is forward-looking, not a fix. #5 — channel_progress_smoke.sh now auto-spawns a temporary agent when the daemon has none. Picks the first available LLM key (GROQ/OPENAI/ANTHROPIC/MINIMAX) for the provider, sends a minimal manifest_toml inline (the SpawnRequest schema requires either manifest_toml or template — name alone isn't valid), and despawns the agent on exit. Pre-existing agents are still reused untouched. * fix(channels/tests): extract DeliveryLog type alias to satisfy clippy::type_complexity CI clippy -D warnings rejected Arc<Mutex<Vec<(bool, Option<String>)>>> as too complex. Hoist into a named alias — same shape, named contract.
houko
added a commit
that referenced
this pull request
May 4, 2026
…esolution (#4534) * feat(api): trusted_proxies + trust_forwarded_for for real-client-IP Adds two top-level KernelConfig fields that let proxied deployments (Cloudflare Tunnel, nginx, Traefik, …) recover the real client IP from forwarding headers without being exploitable when no proxy is present. Closes the long-standing TODO referenced in the now-retired rate_limiter::resolve_client_ip doc comment. trusted_proxies = ["172.19.0.0/16", "127.0.0.1", "::1"] trust_forwarded_for = true When BOTH are set AND the TCP peer matches the allowlist, the daemon resolves the real client IP from forwarding headers (preference: CF-Connecting-IP → X-Real-IP → Forwarded (RFC 7239) → rightmost- untrusted hop in X-Forwarded-For) for: * the GCRA rate limiter (per-IP keying) * the auth-login rate limiter (per-IP keying) * the per-IP WebSocket connection cap + the WS connect log line Fail-closed by default — empty allowlist OR master-switch off OR peer not in allowlist OR malformed headers all collapse back to the TCP peer. A spoofed X-Forwarded-For from an untrusted internet client still hits the limiter on its real source, so the per-IP brute-force properties are preserved. Implementation * New crates/librefang-api/src/client_ip.rs with TrustedProxies (hand-rolled CIDR matcher, no new deps), resolve_real_client_ip, resolve_from_request. * GcraState + new AuthRateLimitState carry Arc<TrustedProxies> + trust_forwarded_for; server.rs compiles the allowlist once at boot and threads it through both middleware layers. * ws.rs::agent_ws uses the resolved IP for the WS slot key and the client_ip log field. * Old fail-closed-only rate_limiter::resolve_client_ip removed. * Auth-bypass loopback gates (no-auth-allow-loopback, LIBREFANG_ALLOW_NO_AUTH) intentionally still key on the TCP peer via addr.ip(), not the resolved IP — auth bypass must never follow a forwarded-header claim. Tests * 19 unit tests in client_ip: CIDR v4/v6, bare IPs, unmasked input, invalid-entry skip, all four header preferences, RFC 7239 bracketed v6 + obfuscated _token, multi-header XFF concatenation, v4:port suffix, malformed-fallback, all-trusted chain. * 3 integration tests in rate_limiter: - browsers behind a trusted proxy get independent buckets - rotating XFF from an untrusted peer does not bypass the limiter - master switch off ignores XFF even from a trusted peer * Existing 611 librefang-api lib + 271 integration tests still pass; workspace cargo clippy --all-targets -D warnings clean. * refactor(api): cache compiled TrustedProxies on AppState Threads `Arc<TrustedProxies>` + `trust_forwarded_for` through `AppState`, compiled once at boot in `server.rs`. The GCRA + auth-login middlewares now read from the cached instance instead of recompiling, and `ws::agent_ws` no longer re-parses the raw config strings (and re-emits the malformed-entry warning) on every WebSocket upgrade. Also adds explicit "Behaviour change" comments at the two `agent_ws` sites that propagate the resolved client IP into `SenderContext.user_id`, so a future reader / operator flipping the new flags on understands that any per-`user_id` kernel state (audit attribution, channel-sender keying, session continuity that keys on it) re-keys from proxy IP to real client IP at that moment. Pure documentation — no behaviour change vs the prior PR commit. Updates the test fixtures (`librefang-testing::test_app::build_state` and the three in-tree `AppState` init sites in `routes/{agents,memory,network}.rs`) to set the new fields to default (header trust off), matching the production default. Addresses review #2 (no per-upgrade recompilation), #5 (migration-notes comments), and #10 (drops the cosmetic `drop(proxy_cfg)` since the allowlist is now read straight off `AppState`). * fix(api): resolve real client IP in terminal_ws WS slot key `terminal_ws` was still keying its per-IP WS slot on `addr.ip()`, which is the same bug `agent_ws` was just fixed for: behind a trusted reverse proxy (cloudflared / nginx / Traefik), the TCP peer is the proxy and every terminal connection from every browser collapses onto a single shared slot — `max_ws_per_ip` then throttles the whole organisation the moment the second tab opens. Mirrors the `agent_ws` fix exactly: pulls the boot-compiled `Arc<TrustedProxies>` and `trust_forwarded_for` flag off `AppState` and calls `client_ip::resolve_real_client_ip` before `try_acquire_ws_slot`. Untrusted peers fall through to `addr.ip()` — a spoofed `X-Forwarded-For` from the open internet still hits the per-IP cap on its real source. `SenderContext.user_id` parity is N/A here: `terminal_ws` does not construct a `SenderContext` (the terminal is a PTY pipe, not a kernel agent message), so the only behavioural surface that needs to swap to the resolved IP is the WS slot key + the rejection log line. Addresses review #1. * fix(api): tighten client_ip parser correctness + safety Five header-parsing fixes lifted from review: - **XFF: bracketed IPv6 with port** — `[2001:db8::1]:1234` fell off the fallback parser because `parse::<IpAddr>` rejects the bracket form and the prior `rsplit_once(':')` left the brackets attached. The walker now strips a surrounding `[...]` (with or without `:port`) before retry, while still bailing on ambiguous unbracketed v6 + port rather than silently truncating. - **RFC 7239 `for=` parameter case-insensitive** — RFC 7230 §3.2.4 makes parameter names case-insensitive; we were only matching `for=` and `For=`. Now compares the key with `eq_ignore_ascii_case("for")`. - **RFC 7239 `unknown` token case-insensitive** — companion fix; now compares with `eq_ignore_ascii_case("unknown")` so `Unknown` / `UNKNOWN` short-circuit the same way as the lowercase form. - **Single-value headers reject the unspecified address** — a misconfigured proxy that injects `0.0.0.0` / `::` into `cf-connecting-ip` / `x-real-ip` no longer poisons the per-IP slot key with an address that can't be a real client. Loopback and RFC1918/ULA addresses are accepted by design — the trusted proxy may legitimately pass an internal-network client. Doc-comment spells out the policy. - **Documentation comments** for the asymmetries reviewers flagged: `single_ip_header` reads only the first instance of the named header (vs the XFF path which concatenates all); the `Forwarded` parser only inspects the first list element and obfuscated tokens short-circuit by design. Adds 8 new unit tests covering: bracketed-v6 with/without port, case-insensitive `for=` (4 variants), case-insensitive `unknown` (3 variants), unspecified-address rejection on both single-value headers (4 variants), loopback + private acceptance (5 variants), plus two WS-slot-key composition tests asserting the resolver collapses spoof attempts from an untrusted peer down to the real TCP source AND separates real clients behind the same trusted proxy into distinct slot keys. Addresses review #3, #4, #6, #7, #8, #9. --------- Co-authored-by: Evan <[email protected]>
This was referenced May 15, 2026
4 tasks
houko
pushed a commit
that referenced
this pull request
May 28, 2026
…eshold, LLM confidence, KV mirror retirement, instrumented spawn, fuzzy categories, configurable prompt cap, CHANGELOG, comment fix 10 follow-ups raised on the code review of the prior two commits in this PR. All within the same memory-system scope. * #1 root user_id is now a constant sentinel UUID (00000000-0000-0000-0000-72006f0074a0, exported as `ROOT_API_KEY_USER_ID`) rather than `UserId::from_name("root")`. The from_name UUIDv5 lives inside `LIBREFANG_USER_NAMESPACE`, so an operator-registered `[users] name = "root"` would have silently inherited the master credential's ACL + per-user budget cap. The sentinel falls outside that namespace; AuthManager returns None for it and the fail-open Owner-default ACL applies. Regression test `root_api_key_user_id_does_not_collide_with_any_named_user` pins the non-collision invariant against {root, admin, owner, system, operator, user}. * #3 `HotAction::UpdateProactiveMemory` now also calls `substrate.set_consolidation_duplicate_threshold(...)`, so when `POST /api/config/reload` swaps in a new `[proactive_memory] duplicate_threshold`, the periodic global consolidation sweep picks it up alongside the per-agent on-demand consolidate. Without this the per-agent path picked up the new value but the global sweep stayed on the old one — exactly the inconsistency H5 set out to remove. `ConsolidationEngine` switched to an `Arc<AtomicU32>` threshold (f32 bits) so the setter takes `&self`, which is required because the hot-reload code path holds only `Arc<MemorySubstrate>`. `docs/operations/config-reload.md` row updated to call out the new behaviour. * #4 `build_extraction_prompt` now asks the LLM to emit a per-memory `confidence` field with a brief calibration guide; the parser reads it, clamps to [0, 1], and stashes the value in `metadata["confidence"]` and `MemoryItem.confidence` so the C3 insert path actually lands a non-default value in the `confidence` column. Missing field still defaults to 1.0 (matches the rule-based extractor's prior behaviour — never silently drops a memory). Tests `parse_extraction_propagates_confidence_to_metadata` and `parse_extraction_clamps_confidence_to_unit_interval` pin the new behaviour. * #5 The KV `memory:*` mirror is gone — fully retired, not just "non-load-bearing". All `structured.set("memory:*", ...)` / `structured.delete("memory:*", ...)` / `list_kv` scans that walked the mirror have been deleted from `import_memories`, `add_with_decision`'s ADD + UPDATE branches, `add_with_level`, `delete`, `update`, `reset`, `clear_level`, `cleanup_expired_sessions`, the eviction loop, and the consolidation merge-loser path. The read path was already on semantic (C1); leaving the writes in place would have grown the mirror without bound and risked future divergence regressions. `test_delete_memory` rewritten to assert behaviour through `search()` (the trait-level contract) instead of probing the underlying KV store. Any legacy `memory:*` entries from older installs are silently ignored. * #6 The detached auto-consolidate `tokio::spawn` is wrapped in a `tracing::info_span!("auto_consolidate", task = "auto_consolidate", agent = ...)` via `.instrument(span)`, so a panic inside the consolidate future surfaces in tracing output (instead of disappearing silently the way bare-spawn panics do) and operators can grep `task = "auto_consolidate"` to find the detached work. * #7 Category allowlist match is now case-insensitive + tolerant of trailing `s` on either side. `"Preferences"` / `"PREFERENCE"` / `"preferences"` all snap to a configured `"preference"` and the canonical configured spelling lands in the column. Test `parse_extraction_fuzzy_matches_category_case_and_plural` pins the four variants. * #8 `format_context_max_chars` is now a field on `ProactiveMemoryConfig` (default 8000 chars / ~2000 tokens); the store's `format_context_with_query` / `format_context` read it from the live config and pass it to `format_memories_with_budget(memories, max_chars)`. The trait fallback (`DefaultMemoryExtractor::format_context`, `LlmMemoryExtractor::format_context`) keeps the const default for callers without config access. Operators on 200k+ context windows can now raise the cap via `config.toml` without recompiling. * #2 + #9 New `Fixed` entry in `[Unreleased]` documents the breaking audit-shape change (root-api_key requests now stamp a `user_id` where they previously stamped `None`) and the `add()` behaviour change (no raw-transcript fallback for extraction misses). Both were noted inline in commits but absent from the CHANGELOG. The entry also lists the full audit-sweep scope so an operator can size-up the upgrade impact from one place. * #10 `import_memories` comment rewritten to explain the 0.95 threshold without the confusing "stricter than extraction-time dedup" framing. Drive-by: collapsed three more `clippy::manual_option_zip` sites in `kernel/tests.rs` that the workspace-clippy gate would have caught on the next CI run. Auto-restamped `.secrets.baseline` line numbers shifted by the CHANGELOG insert. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api -p librefang-kernel --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 269 passed * cargo test -p librefang-runtime --lib proactive_memory — 60 passed (5 new follow-up regression tests included) * cargo test -p librefang-api --lib --test memory_routes_integration --test agent_kv_authz_integration --test auth_public_allowlist — all green
houko
added a commit
that referenced
this pull request
May 29, 2026
…cay, dedup, prompt budget, async consolidate) (#5839) * fix(memory): split-brain reads, raw-transcript fallback, RBAC on writes, forget() leak, immortal decay 5 CRITICAL findings from a memory-system audit, all in the proactive memory layer: * C1 split-brain: list()/get() read from the KV mirror while search()/auto_retrieve read from the semantic store. The KV write was best-effort (warn-and-continue) so a failure left rows visible to search but invisible to list. retrieve_memory_items now reads from semantic (the authoritative source); KV writes stay as a non-load- bearing compatibility mirror. * C2 raw-transcript fallback: when the extractor returned no signal, add() stored the verbatim concatenated message content as a session-level memory with no category. This was the dominant source of `category=null` rows and duplicate transcripts on the dashboard. The fallback is removed; callers that want raw content use add_with_level explicitly. * C3 confidence hardcoded to 1.0 + immortal decay: remember_with_embedding_and_peer now honors metadata["confidence"] (clamped to 0..=1, defaulting to 1.0) so the LLM extractor's signal reaches the column. decay_confidence reworked: boost divides the rate instead of multiplying the result and clamping to 1.0. The old formula made any memory with >=2 accesses freeze at confidence 1.0 forever; the new formula keeps "popular memories decay slower" but strictly monotonic (boost capped at MAX_BOOST=4.0). * C4 RBAC on write endpoints: memory_add, memory_update, memory_delete, memory_bulk_delete, memory_reset_agent, memory_clear_level, memory_consolidate, memory_cleanup, memory_export_agent, memory_import_agent, memory_decay, memory_store_relations now route through the namespace guard. New ProactiveMemoryStore wrappers cover the previously-unguarded ops (reset, clear_level, export_all, import_memories, decay_confidence). The root api_key is attributed as an Owner- equivalent AuthenticatedApiUser in middleware so operators using only the master credential keep their POST/PUT/DELETE access. * C5 forget() never marked deleted_at: SemanticStore::forget* now stamps deleted_at alongside deleted=1 so the prune_soft_deleted_memories sweep (filter `deleted_at IS NOT NULL`) can actually hard-delete user-/API-initiated deletions. Without the stamp every soft-deleted row leaked its embedding BLOB forever. consolidation.rs's merge-loser delete now stamps it too. Drive-by: removed a clippy::manual_option_zip in kernel/background_lifecycle.rs flagged while clippy-gating the change. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 269 passed (5 pre-existing tests adapted to the new add()-no-fallback semantics; new regression tests for C1 list-from-semantic, C2 no-fallback, C3 monotonic-decay + extractor-confidence-roundtrip, C4 viewer-denied on every write wrapper, C5 forget* stamps deleted_at) * cargo test -p librefang-api --lib --test memory_routes_integration --test agent_kv_authz_integration --test auth_public_allowlist — all green * chore(codegen): auto-regenerate openapi.json + sdk + schema baselines [skip ci] * fix(memory): tighten dedup thresholds, validate LLM extraction, cap prompt budget, detach auto-consolidate, unify consolidation knob Follow-up sweep on the same memory-system audit as the prior commit — picks off the HIGH findings that were in scope for the same crate / file cluster. * H1 duplicate_threshold tightened. The configured default jumps from 0.5 → 0.85 (mem0's recommended near-duplicate cut-off); both metrics — cosine and Jaccard — agree that 0.5 means "topically related", which let opposite-meaning sentences sharing keywords silently merge. `DefaultMemoryExtractor::decide_action`'s hardcoded same-category 0.5 / cross-category 0.6 UPDATE thresholds rise to 0.7 / 0.8; the 0.95 NOOP gate stays. `import_memories`' hardcoded 0.9 dedup floor rises to 0.95 with a docstring explaining why bulk-import is stricter than extraction-time dedup. * H2 LLM-extraction validation. `parse_llm_extraction_response` now enforces a 4-char content floor (drops "ok" / "no" / single-letter junk that was trivially unique and survived dedup), validates the emitted `category` against the configured `extract_categories` allowlist (out-of-allowlist values downgrade to "general" instead of polluting the dashboard's facets), and caps a single extraction call at MAX_MEMORIES_PER_EXTRACTION=20 rows so a runaway model can't churn the eviction loop. * H4 prompt-injection budget. `format_context` now goes through a shared `format_memories_with_budget` helper capped at FORMAT_CONTEXT_MAX_CHARS=8000 (~2000 tokens). Pre-fix the formatter concatenated everything with no ceiling — 10 retrieved memories × 2000-char MAX_MEMORY_CONTENT_LENGTH could push 20 KB into every request. Excess rows are reported via a "[+N additional memories omitted to keep the prompt within budget]" footer so the truncation is observable in the rendered prompt rather than silent. * H6 auto-consolidation no longer blocks the agent. The every-10 trigger in `auto_memorize` now `tokio::spawn`s the consolidate call instead of awaiting it inline; the next agent turn doesn't pay for the O(n²) merge pass plus its SQLite transaction. The detached future borrows nothing from `self` thanks to `ProactiveMemoryStore`'s manual Clone over Arc'd inner state. * H5 single source of truth for the consolidation threshold. `ConsolidationEngine` gains a `duplicate_threshold` field (defaulting to 0.85 to match the new config default) with a `set_duplicate_threshold` setter; `MemorySubstrate` exposes a passthrough; kernel boot pushes `config.proactive_memory. duplicate_threshold` down to the engine. The periodic global consolidation sweep and the on-demand `ProactiveMemoryStore::consolidate` now agree on what counts as a near-duplicate. Keeps the existing 142 callers of `MemorySubstrate::open_in_memory(decay_rate)` source-compatible (no signature break). H7 / H8 from the same audit were already addressed by the C3 fix in the previous commit (popular-memory immortality + dead extraction_threshold). H3 (memory_store/recall vs auto_memorize/ retrieve being disconnected tool surfaces) is intentional product shape rather than a bug — left out. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 269 passed * cargo test -p librefang-runtime --lib (proactive_memory) — 57 passed, including the 5 new regressions (parse_extraction_drops_sub_minimum_content, parse_extraction_downgrades_unknown_category, parse_extraction_preserves_category_when_allowlist_empty, parse_extraction_caps_total_memories_per_call, format_context_caps_prompt_budget_with_truncation_marker). * cargo test -p librefang-api --test memory_routes_integration --test agent_kv_authz_integration --test auth_public_allowlist — all green. * fix(memory): review-followups — sentinel root user_id, hot-reload threshold, LLM confidence, KV mirror retirement, instrumented spawn, fuzzy categories, configurable prompt cap, CHANGELOG, comment fix 10 follow-ups raised on the code review of the prior two commits in this PR. All within the same memory-system scope. * #1 root user_id is now a constant sentinel UUID (00000000-0000-0000-0000-72006f0074a0, exported as `ROOT_API_KEY_USER_ID`) rather than `UserId::from_name("root")`. The from_name UUIDv5 lives inside `LIBREFANG_USER_NAMESPACE`, so an operator-registered `[users] name = "root"` would have silently inherited the master credential's ACL + per-user budget cap. The sentinel falls outside that namespace; AuthManager returns None for it and the fail-open Owner-default ACL applies. Regression test `root_api_key_user_id_does_not_collide_with_any_named_user` pins the non-collision invariant against {root, admin, owner, system, operator, user}. * #3 `HotAction::UpdateProactiveMemory` now also calls `substrate.set_consolidation_duplicate_threshold(...)`, so when `POST /api/config/reload` swaps in a new `[proactive_memory] duplicate_threshold`, the periodic global consolidation sweep picks it up alongside the per-agent on-demand consolidate. Without this the per-agent path picked up the new value but the global sweep stayed on the old one — exactly the inconsistency H5 set out to remove. `ConsolidationEngine` switched to an `Arc<AtomicU32>` threshold (f32 bits) so the setter takes `&self`, which is required because the hot-reload code path holds only `Arc<MemorySubstrate>`. `docs/operations/config-reload.md` row updated to call out the new behaviour. * #4 `build_extraction_prompt` now asks the LLM to emit a per-memory `confidence` field with a brief calibration guide; the parser reads it, clamps to [0, 1], and stashes the value in `metadata["confidence"]` and `MemoryItem.confidence` so the C3 insert path actually lands a non-default value in the `confidence` column. Missing field still defaults to 1.0 (matches the rule-based extractor's prior behaviour — never silently drops a memory). Tests `parse_extraction_propagates_confidence_to_metadata` and `parse_extraction_clamps_confidence_to_unit_interval` pin the new behaviour. * #5 The KV `memory:*` mirror is gone — fully retired, not just "non-load-bearing". All `structured.set("memory:*", ...)` / `structured.delete("memory:*", ...)` / `list_kv` scans that walked the mirror have been deleted from `import_memories`, `add_with_decision`'s ADD + UPDATE branches, `add_with_level`, `delete`, `update`, `reset`, `clear_level`, `cleanup_expired_sessions`, the eviction loop, and the consolidation merge-loser path. The read path was already on semantic (C1); leaving the writes in place would have grown the mirror without bound and risked future divergence regressions. `test_delete_memory` rewritten to assert behaviour through `search()` (the trait-level contract) instead of probing the underlying KV store. Any legacy `memory:*` entries from older installs are silently ignored. * #6 The detached auto-consolidate `tokio::spawn` is wrapped in a `tracing::info_span!("auto_consolidate", task = "auto_consolidate", agent = ...)` via `.instrument(span)`, so a panic inside the consolidate future surfaces in tracing output (instead of disappearing silently the way bare-spawn panics do) and operators can grep `task = "auto_consolidate"` to find the detached work. * #7 Category allowlist match is now case-insensitive + tolerant of trailing `s` on either side. `"Preferences"` / `"PREFERENCE"` / `"preferences"` all snap to a configured `"preference"` and the canonical configured spelling lands in the column. Test `parse_extraction_fuzzy_matches_category_case_and_plural` pins the four variants. * #8 `format_context_max_chars` is now a field on `ProactiveMemoryConfig` (default 8000 chars / ~2000 tokens); the store's `format_context_with_query` / `format_context` read it from the live config and pass it to `format_memories_with_budget(memories, max_chars)`. The trait fallback (`DefaultMemoryExtractor::format_context`, `LlmMemoryExtractor::format_context`) keeps the const default for callers without config access. Operators on 200k+ context windows can now raise the cap via `config.toml` without recompiling. * #2 + #9 New `Fixed` entry in `[Unreleased]` documents the breaking audit-shape change (root-api_key requests now stamp a `user_id` where they previously stamped `None`) and the `add()` behaviour change (no raw-transcript fallback for extraction misses). Both were noted inline in commits but absent from the CHANGELOG. The entry also lists the full audit-sweep scope so an operator can size-up the upgrade impact from one place. * #10 `import_memories` comment rewritten to explain the 0.95 threshold without the confusing "stricter than extraction-time dedup" framing. Drive-by: collapsed three more `clippy::manual_option_zip` sites in `kernel/tests.rs` that the workspace-clippy gate would have caught on the next CI run. Auto-restamped `.secrets.baseline` line numbers shifted by the CHANGELOG insert. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api -p librefang-kernel --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 269 passed * cargo test -p librefang-runtime --lib proactive_memory — 60 passed (5 new follow-up regression tests included) * cargo test -p librefang-api --lib --test memory_routes_integration --test agent_kv_authz_integration --test auth_public_allowlist — all green * fix(api): attribute Owner on no-auth loopback so memory writes aren't 403 The RBAC gating added 12 memory-write ACL checks, but the default `librefang start` (no api_key, loopback bind) takes the no-auth bypass which returned next.run() WITHOUT attaching an AuthenticatedApiUser. Memory write handlers then saw None -> anonymous Viewer fallback -> 403 on every POST/PUT/DELETE /api/memory*, breaking the documented default workflow. No-auth + trusted origin (loopback / LIBREFANG_ALLOW_NO_AUTH) is the same trust level as the root master credential, so attribute the same Owner-equivalent user (ROOT_API_KEY_USER_ID). Non-loopback still fails closed. Add integration tests for both: loopback write != 403, non-loopback no-auth still 401. * fix(memory): read-only recall for listing paths; correct decay doc MEDIUM (#5839): list/get/export/list_all read paths called recall(), which unconditionally bumps access_count + accessed_at. A dashboard polling the memory list would perpetually reset accessed_at = now and inflate access_count — the exact signals the C3 decay logic keys idle/popularity off — so polled listings could keep memories from ever decaying (and turned a GET into a 10k-row write). Add recall_readonly() (shared impl, no bump) and route the four listing/export reads through it; genuine semantic recalls still track access. Regression test asserts recall_readonly leaves access_count untouched while recall() bumps by 1. Also correct the decay_confidence doc: the once-per-hour cadence is enforced by the periodic maintenance scheduler, not an internal throttle (a direct call decays immediately). --------- Co-authored-by: Evan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
houko
pushed a commit
that referenced
this pull request
May 29, 2026
…loor, partial-status, dedup-strip, test tightening 8 follow-ups from the code review of the prior commit. All in scope for the same proactive-memory layer. * #1 misattributed doc-comment in `proactive.rs` near `AUTO_CONSOLIDATE_EVERY` / `NEGATION_WORDS`. The `/// Negation/contradiction words …` line was orphaned above `AUTO_CONSOLIDATE_EVERY` when the new const got inserted; both consts now carry their own intended docstring. * #2 lowered `STALE_COUNTER_FLOOR` from `AUTO_CONSOLIDATE_EVERY / 2` (5) to `/ 4` (2). The /2 floor cleaned up "stuck at 1..4" agents but also reset slow-burn agents (single auto_memorize per maintenance tick) before they could climb to 10, so a steady 1-call/hour stream effectively never consolidated. /4 keeps the cold-slot eviction directional while letting low-frequency agents still accumulate to the trigger. * #3 `PATCH /api/memory/config` response shape is now explicit about partial success. `body.status` is `"applied"` when the reload succeeded and `"partial"` when the disk write landed but the live reload failed (e.g. operator hand-edited an unrelated section into an invalid shape between PATCH writes). Clients MUST inspect `status`; the HTTP code stays 200 for both branches since the disk write itself succeeded. 207 / 500 were considered and rejected — 500 misrepresents that the request was rejected (it wasn't) and 207 forces every existing client to re-classify success. Mirrors the `import_agent_memory` partial-success pattern. * #4 documented the M13 cost tradeoff. Up to 4 SQLite roundtrips per insertion on the no-embedding fallback path (versus the pre-fix 1), but the embedding-driver path is unaffected and the loop short-circuits as soon as the union hits fetch_limit, so the common case ("first keyword filled the slate") still runs a single query. Operators on the no-embedding path pay the cost on writes only, against an already-unindexed `content LIKE` scan. * #5 defensively strip the M14 `_update_threshold_*` keys from the enriched item right after `decide_action` returns + assert callers don't pre-populate them. Both insertion branches today build their stored metadata from the original `item.metadata` (not `enriched_item.metadata`), so the threshold keys never reach the column — but stripping + asserting is cheap insurance for the next refactorer who repoints either branch at the enriched copy. * #6 relaxed the `extract_search_keywords_returns_multiple_ordered_longest_first` test: no more exact `kws.len() == 4` — the cap and the "longest distinctive word survives" invariants are what we care about, not the exact count. Future additions to `STOP_WORDS` no longer silently break the test. * #7 dropped the unused `_update_threshold_cross_cat` set in `decide_action_honors_config_update_thresholds`. Both candidate memories share the "preference" category, so the cross-cat threshold branch was unreachable. * #8 moved `STALE_COUNTER_FLOOR` from a `fn`-scope `const` to module scope, alongside `AUTO_CONSOLIDATE_EVERY` from which it derives. Matches the repo convention and lets future readers see the floor / trigger relationship at one site. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 273 passed * cargo test -p librefang-api --test memory_routes_integration — 14 passed
houko
added a commit
that referenced
this pull request
May 29, 2026
…CH, multi-keyword search, configurable UPDATE thresholds (#5850) * fix(memory): MEDIUM follow-ups — counter map sweep, hot-reload on PATCH, multi-keyword search, configurable UPDATE thresholds Continuation of the audit sweep on the proactive-memory subsystem. 4 MEDIUM findings from the same audit, all in scope for this PR. * M11 `consolidation_counters` HashMap is now actively pruned every maintenance tick. Pre-fix it only swept when the map crossed 1000 entries (truncate-to-500 by count DESC), which delayed cleanup until the leak was observable AND deleted the highest-count entries — exactly the agents about to fire a real consolidate. The new sweep drops every counter that hasn't passed the halfway mark (`< AUTO_CONSOLIDATE_EVERY / 2 = 5`) on each maintenance tick: HashMap::retain in-place, evicts cold entries first, never touches an entry that's about to fire. Added named constant `AUTO_CONSOLIDATE_EVERY = 10` so the trigger + the prune floor stay in lockstep. * M12 `PATCH /api/memory/config` now calls `kernel.reload_config()` after writing `config.toml`, so dashboard saves take effect on the running kernel instead of staying disk-only until restart. Pre-fix the response always reported `restart_required: true`, which confused operators who could see GET return the new values while live behaviour (ProactiveMemoryStore::config, decay engine, etc.) stayed on the boot snapshot. `restart_required` now reflects the actual `ReloadPlan` — false when every diff field hot-reloads, true when any field needs a restart. A reload validation failure is surfaced via the new `reload_error` field instead of swallowing the disk write. * M13 `extract_search_keywords` returns a `Vec<String>` of the top 4 distinctive keywords ordered longest-first (post-stop-word filter, post-dedup), and the caller iterates over them unioning per-keyword LIKE recalls until `fetch_limit` is hit. Pre-fix it collapsed the four candidates to the single longest one, wasting the stop-word filter work on the other three and giving the no-embedding fallback path a frequently-too-generic substring (e.g. "analysis") to match against, OR a too-specific compound term that matched nothing. The fallback to the raw-content LIKE is preserved for the no-distinctive-words case so a near-verbatim duplicate is still detectable. * M14 the `decide_action` UPDATE thresholds are now configurable via two new `ProactiveMemoryConfig` fields: `update_threshold_same_category` (default 0.7) and `update_threshold_cross_category` (default 0.8). The trait method signature stays stable — `add_with_decision` stashes the live config values in the new memory's metadata under `_update_threshold_same_cat` / `_update_threshold_cross_cat`, the default `decide_action` reads them out (falling back to the const defaults for direct trait-method callers), and the LLM-backed extractor inherits the same behaviour via its fallback to the default heuristic on driver failure. This separates the per-insertion conflict-resolution threshold (UPDATE vs ADD) from the post-hoc consolidation threshold (`duplicate_threshold`) — pre-fix both were conflated. Drive-by fmt: two stale formatting hunks `cargo fmt` flagged in `runtime/tool_runner/wasm_skill.rs:159` and `api/tests/memory_routes_integration.rs:518` (neither mine, both inherited from main). Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api -p librefang-kernel --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 273 passed (4 new regression tests: `extract_search_keywords_returns_multiple_ordered_longest_first`, `extract_search_keywords_empty_for_all_stop_words`, `decide_action_honors_config_update_thresholds`, and the existing suite re-verified against the new `update_threshold_*_category` config fields) * cargo test -p librefang-runtime --lib proactive_memory — 60 passed * cargo test -p librefang-api --lib --test memory_routes_integration --test agent_kv_authz_integration --test auth_public_allowlist — all green * fix(memory): review-followups on #5850 — doc-comment fix, threshold floor, partial-status, dedup-strip, test tightening 8 follow-ups from the code review of the prior commit. All in scope for the same proactive-memory layer. * #1 misattributed doc-comment in `proactive.rs` near `AUTO_CONSOLIDATE_EVERY` / `NEGATION_WORDS`. The `/// Negation/contradiction words …` line was orphaned above `AUTO_CONSOLIDATE_EVERY` when the new const got inserted; both consts now carry their own intended docstring. * #2 lowered `STALE_COUNTER_FLOOR` from `AUTO_CONSOLIDATE_EVERY / 2` (5) to `/ 4` (2). The /2 floor cleaned up "stuck at 1..4" agents but also reset slow-burn agents (single auto_memorize per maintenance tick) before they could climb to 10, so a steady 1-call/hour stream effectively never consolidated. /4 keeps the cold-slot eviction directional while letting low-frequency agents still accumulate to the trigger. * #3 `PATCH /api/memory/config` response shape is now explicit about partial success. `body.status` is `"applied"` when the reload succeeded and `"partial"` when the disk write landed but the live reload failed (e.g. operator hand-edited an unrelated section into an invalid shape between PATCH writes). Clients MUST inspect `status`; the HTTP code stays 200 for both branches since the disk write itself succeeded. 207 / 500 were considered and rejected — 500 misrepresents that the request was rejected (it wasn't) and 207 forces every existing client to re-classify success. Mirrors the `import_agent_memory` partial-success pattern. * #4 documented the M13 cost tradeoff. Up to 4 SQLite roundtrips per insertion on the no-embedding fallback path (versus the pre-fix 1), but the embedding-driver path is unaffected and the loop short-circuits as soon as the union hits fetch_limit, so the common case ("first keyword filled the slate") still runs a single query. Operators on the no-embedding path pay the cost on writes only, against an already-unindexed `content LIKE` scan. * #5 defensively strip the M14 `_update_threshold_*` keys from the enriched item right after `decide_action` returns + assert callers don't pre-populate them. Both insertion branches today build their stored metadata from the original `item.metadata` (not `enriched_item.metadata`), so the threshold keys never reach the column — but stripping + asserting is cheap insurance for the next refactorer who repoints either branch at the enriched copy. * #6 relaxed the `extract_search_keywords_returns_multiple_ordered_longest_first` test: no more exact `kws.len() == 4` — the cap and the "longest distinctive word survives" invariants are what we care about, not the exact count. Future additions to `STOP_WORDS` no longer silently break the test. * #7 dropped the unused `_update_threshold_cross_cat` set in `decide_action_honors_config_update_thresholds`. Both candidate memories share the "preference" category, so the cross-cat threshold branch was unreachable. * #8 moved `STALE_COUNTER_FLOOR` from a `fn`-scope `const` to module scope, alongside `AUTO_CONSOLIDATE_EVERY` from which it derives. Matches the repo convention and lets future readers see the floor / trigger relationship at one site. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-runtime -p librefang-types -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 273 passed * cargo test -p librefang-api --test memory_routes_integration — 14 passed * fix(memory): review-followups (round 2) — proper M11 idle-window, M12 + M14 regression coverage, comment polish 5 follow-ups from the second review pass on #5850. * B (was #1) M11 done properly. `consolidation_counters` is now `HashMap<String, CounterEntry>` where `CounterEntry` carries both the running count and a `last_touched: DateTime<Utc>` stamp. The maintenance sweep evicts entries whose `last_touched` is older than `STALE_COUNTER_IDLE_WINDOW = 2 hours` (~2× the maintenance rate-limit window), regardless of count. The previous count-threshold fix (followup #2 in the prior commit) mitigated but didn't solve the slow-burn case: any agent firing ≤ 1 × per maintenance window would still be reset before climbing past the count floor. The timestamp-based check closes that gap — an active slot, however slow, is preserved as long as it's been touched within the window; a truly idle slot is reclaimed within ~2 hours of going quiet. * C (was #2) `PATCH /api/memory/config` happy-path test added at `memory_routes_integration::patch_memory_config_hot_reloads_and_reports_applied`. Pre-seeds a minimal `config.toml` (the harness's tempdir previously didn't materialise one, which the file-level docstring flagged as out-of-scope; the docstring updated accordingly) and asserts `body["status"] == "applied"`, `body["reload_error"]` null, and that the PATCHed value round-trips into the response. Without this, the M12 status contract could silently revert and the rest of the suite wouldn't catch it. `RouterHarness._tmp` is exposed as `tmp` to let the test reach the seed location. * D (was #3) M14 strip regression test added at `proactive::tests::add_with_decision_does_not_leak_threshold_keys_to_stored_metadata`. Drives the full `add()` path through `add_with_decision`, then reads back via `list()` and asserts none of the private `_update_threshold_*` / `_embedding` keys leaked into the stored metadata column. Catches the regression where someone repoints the ADD or UPDATE branch at `enriched_item.metadata` (the decision-clone) instead of the original `item.metadata` (the caller's input). * E (was #4) the `debug_assert!` panic messages on the `_update_threshold_*` private keys re-worded from "caller leaked it" to "callers must not pre-populate ..." — neutral phrasing that doesn't presume the caller is buggy. Also added a one-line note that the production path stays safe regardless (the unconditional `insert` overwrites any leaked value before `decide_action` reads it), since `debug_assert!` is compiled out in release. * F (was #5) M13 cost-trade-off comment tightened. The "common case still runs a single query" claim was accurate for agents with sizeable stores (first keyword exhausts fetch_limit) but not for fresh / small stores where no individual keyword has enough matches to short-circuit. The comment now distinguishes the two regimes instead of overgeneralising. Re-stamped `.secrets.baseline` line numbers shifted by the test file edits. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 274 passed (incl. `add_with_decision_does_not_leak_threshold_keys_to_stored_metadata`) * cargo test -p librefang-api --test memory_routes_integration — 15 passed (incl. `patch_memory_config_hot_reloads_and_reports_applied`) * fix(memory): review-followups (round 3) — M12 test key-presence + accepts partial, M14 strips _embedding too, prune rate-limit, chrono::Duration const portability 6 followups from the third review pass on #5850. * #1 M12 test pinned the contract properly. `serde_json::Value` indexed by a missing key returns `Value::Null`, so the prior `assert_eq!(body["reload_error"], Null)` silently passed even if the field had been removed. Now asserts `body.as_object() .contains_key(...)` for `status`, `restart_required`, `reload_error` first, then asserts their values. * #2 strip + test for the `_embedding` private-stash key. `add_with_decision` now also calls `enriched_item.metadata.remove("_embedding")` after `decide_action` returns, so all three private stash keys (`_update_threshold_*` + `_embedding`) get the same defensive treatment. Test renamed to `add_with_decision_does_not_leak_private_stash_keys_to_stored_metadata` and attaches a tiny mock `EmbeddingFn` so the `_embedding` stash path actually fires — the prior assertion against `_embedding` was decorative because the test had no embedding driver configured. * #3 rate-limited the counter prune via a new `last_counter_prune: Arc<Mutex<Option<DateTime<Utc>>>>` and a `maybe_prune_counters` helper that mirrors the `maybe_decay_confidence` / `maybe_cleanup_expired` once-per-hour pattern. Prior to this the prune ran on every `maybe_run_maintenance` call, reachable from `search` / `auto_retrieve` / `consolidate` at potentially many Hz. The retain itself was microseconds, so this is a wash today, but it brings the three maintenance sub-tasks under the same scheduling budget for future scaling. * #4 docstring on `STALE_COUNTER_IDLE_WINDOW_HOURS` re-phrased to describe the slot-keeping guarantee in terms of the maintenance rate-limit window, not "consecutive prune passes" — the old phrasing happened to be true only because the prune wasn't rate-limited yet (now rectified by #3). * #5 M12 test now accepts either `"applied"` or `"partial"` as the body status — both are valid post-fix outcomes; the pre-fix contract had no `status` field at all. The status field's presence (asserted via the #1 fix) is the actual contract we're pinning. Made the test robust to future `KernelConfig::default()` changes that might cause the seeded toml to fail reload validation. * #6 swapped `const STALE_COUNTER_IDLE_WINDOW: chrono::Duration = chrono::Duration::hours(2)` for `const STALE_COUNTER_IDLE_WINDOW_HOURS: i64 = 2` plus `chrono::Duration::hours(STALE_COUNTER_IDLE_WINDOW_HOURS)` at the call site. `chrono::Duration::hours` is a `const fn` at the currently pinned `chrono` minor but the const-ness isn't a stable contract across `0.4.x` versions, so a lockfile bump could silently break the build. The integer-hours + runtime conversion stays valid regardless. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 274 passed (incl. `add_with_decision_does_not_leak_private_stash_keys_to_stored_metadata` with embedding-driver coverage) * cargo test -p librefang-api --test memory_routes_integration — 15 passed (incl. tighter `patch_memory_config_hot_reloads_and_reports_applied` contract assertions) * fix(memory): review-followups (round 4) — docstring honesty + direct strip helper + non-empty partial-error 3 follow-ups from the fourth review pass on #5850. All three are about closing gaps between what the code does and what the docstrings / tests claim it does. * A `STALE_COUNTER_IDLE_WINDOW_HOURS` docstring: the "reclaimed within ~2 hours of going quiet" claim was accurate before the round-3 prune rate-limit landed, after which the worst-case reclaim latency is 2-3 hours (idle window + up to one prune rate-limit period because the prune itself only runs ≤ once per hour). Re-phrased with explicit upper/lower bounds; deleted the duplicate "previous fix (round-1 followup #2)" paragraph that had ended up in the doc twice during the round-2 edit. * B `strip_private_stash_keys` extracted into a module-level function driven by a single `ADD_WITH_DECISION_PRIVATE_STASH_KEYS` const, and unit-tested directly via `strip_private_stash_keys_removes_all_private_keys`. The integration test `add_with_decision_does_not_leak_private_stash_keys_to_stored_metadata` only catches a *coordinated two-step regression* (strip removed AND ADD/UPDATE branch repointed at `enriched_item.metadata`) — the current ADD path bypasses `enriched_item.metadata` entirely, so single-step regressions of either kind pass that test. The prior commit's docstring claimed "the strip code on the post-decide path is the only thing keeping it out of the stored column", which was wrong — `item.metadata` (the caller's input) is what actually keeps the keys out today. Updated the docstring to match reality and added the direct unit test on the helper to cover single-step strip regressions. * C `reload_error` partial-branch assertion in `patch_memory_config_hot_reloads_and_reports_applied` now also rejects empty / whitespace-only strings. `is_string()` alone passed `""` / `" "` / any other zero-info value — operators would see status=partial with a useless error blob and have no actionable diagnostic. Trimmed-non-empty makes the contract honest about what "carries the validator output" means. Verification: * cargo check --workspace --lib — clean * cargo clippy -p librefang-memory -p librefang-api --tests -- -D warnings — clean * cargo test -p librefang-memory --lib — 275 passed (1 new: `strip_private_stash_keys_removes_all_private_keys`) * cargo test -p librefang-api --test memory_routes_integration — 15 passed --------- Co-authored-by: Evan <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps socket2 from 0.5.10 to 0.6.3.
Changelog
Sourced from socket2's changelog.
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)