Skip to content

Commit a96af89

Browse files
Harden shared agent instruction review (#4220)
## Summary - render shared-agent instructions as literal text so Markdown cannot conceal spoiler contents, link destinations, or image sources - reject non-reviewable Unicode controls at every agent-definition boundary while preserving legitimate rendered emoji sequences - verify shared catalog event IDs and signatures before trusting authorship, coordinates, pagination, or executable content - preserve the exact system-prompt bytes between review and execution instead of silently stripping or normalizing content ## Security rationale Shared system prompts are executable configuration. Previously, catalog prompts were projected through the chat Markdown renderer, which could hide text, replace link destinations with benign labels, and turn image syntax into remote loads. Zero-width and bidirectional controls could also make reviewed text differ from what the agent executes. This change establishes a review invariant: the prompt a user sees is the prompt the agent executes. Definitions that cannot be reviewed faithfully are rejected rather than rewritten. Catalog events must also pass Nostr ID/signature verification before they can claim a publisher, coordinate, or cursor. ## What changed - catalog instructions render as exact literal text rather than rich Markdown - catalog relay events are verified on a fresh wire-shaped object before paging, coordinate selection, attribution, or projection - forged content, pubkeys, signatures, and invalid newer heads are ignored and cannot shadow a valid signed definition - TypeScript catalog parsing rejects unsafe remote definitions before they reach the UI - shared Rust validation covers persona create/update/import, inbound relay sync, definition-less managed-agent sync, and catalog publication paths - definition-less managed agents now fail closed on local create, local update, and publication before persistence or relay retention - linked managed agents validate their local name while treating the persona definition as authoritative; their inert record-level prompt is not executed or published - names reject layout controls; prompts retain ordinary newlines and tabs - legitimate emoji composition is supported, including contextual VS16, ZWJ, skin-tone, family, flag, and keycap sequences - detached selectors/joiners, bidirectional controls, tag characters, zero-width concealment, and other default-ignorables remain rejected - names are bounded to 128 characters and prompts to 64 KiB - contributor guidance documents the byte-for-byte review requirement for future sharing paths Validation reports the offending code point and never silently removes it. ## E2E recording [buzz-shared-agent-security-e2e.webm](https://github.com/user-attachments/assets/44d6b75f-0877-490f-bda4-a716fae3f700) The recording demonstrates: - a safe definition remains visible - a prompt containing zero-width `U+200B` is rejected - a name containing bidi override `U+202E` is rejected - the prompt is preserved exactly - spoiler, link, and image syntax remains literal and does not render or load ## Verification Passed locally: - `just test`: all 10 unit and Docker-backed integration stages - desktop frontend unit suite: 4,295 tests - persona catalog relay unit suite: 32 tests, including forged-event and cursor-shadowing cases - focused Rust definition-validation coverage: 3 local create/update tests and 6 publication-filtered tests - complete desktop Tauri library suite after rebase: 2,263 passed, 14 ignored, 0 failed - desktop Tauri clippy with warnings denied and Rust formatting - complete agent Playwright spec: 34 tests - the exact formerly failing `inbox-edit` immediate-attachment smoke test after rebase: 1 test - focused shared-agent publish, literal-review, hidden-control, signature, and cross-member import Playwright coverage - desktop E2E production build and TypeScript typecheck - changed-file formatting/lint and file-size ratchet - pre-commit secret scan and DCO signoff The branch was rebased onto current `main`, which includes the upstream attachment-button label fix. Fresh post-rebase GitHub CI is green for every required and selected check: Desktop Core, all four Desktop Smoke E2E shards, both Desktop E2E Integration shards and their aggregate, Desktop E2E Relay, Desktop Build (macOS), Windows Rust, Rust Lint, DCO, security scanners, and Desktop Release Candidate. The previously failing `Desktop Smoke E2E (3)` shard now passes. The repository-wide desktop check also reports existing CSS formatting/`!important` findings in `components.css` and `terminal.css`; neither file is changed by this PR. GitHub's Desktop Core lint and format stage passes on the rebased branch. --------- Signed-off-by: Alex Rosenzweig <[email protected]>
1 parent 9e0c6b4 commit a96af89

21 files changed

Lines changed: 1179 additions & 142 deletions

‎desktop/src-tauri/src/commands/agent_models.rs‎

Lines changed: 2 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ use serde::Deserialize;
55
use tauri::{AppHandle, State};
66

77
use super::agent_model_process::run_agent_models_command;
8+
use super::managed_agent_definition::apply_model_provider_prompt_update;
89
// The map-only lookup is reached solely from the base-URL helpers that exist for
910
// their unit tests; discovery itself always goes through the process-env variant.
1011
#[cfg(test)]
@@ -696,35 +697,6 @@ use databricks::{
696697
};
697698
use databricks::{discover_databricks_models, DatabricksAuthIntent};
698699

699-
/// Apply an `UpdateManagedAgentRequest`'s model/provider/system_prompt patch
700-
/// to `record`, enforcing the linked-instance write guard: a definition-linked
701-
/// record's model/provider/prompt are definition-authoritative (see
702-
/// `effective_config::resolve_linked`), so writes to these three fields are
703-
/// silently dropped for a linked instance rather than persisting a byte the
704-
/// resolver will never read. Definition-less instances accept the patch
705-
/// as-is. Extracted so the guard is exercised by both `update_managed_agent`
706-
/// and its regression tests — a test that reimplements this check instead of
707-
/// calling it can go green after the real guard is deleted.
708-
fn apply_model_provider_prompt_update(
709-
record: &mut crate::managed_agents::ManagedAgentRecord,
710-
model: Option<Option<String>>,
711-
provider: Option<Option<String>>,
712-
system_prompt: Option<Option<String>>,
713-
) {
714-
if record.persona_id.is_some() {
715-
return;
716-
}
717-
if let Some(model_update) = model {
718-
record.model = model_update;
719-
}
720-
if let Some(provider_update) = provider {
721-
record.provider = provider_update;
722-
}
723-
if let Some(prompt_update) = system_prompt {
724-
record.system_prompt = prompt_update;
725-
}
726-
}
727-
728700
/// Update mutable fields on an existing managed agent record.
729701
///
730702
/// Does NOT auto-restart the agent. Runtime config changes (system prompt,
@@ -769,7 +741,7 @@ pub async fn update_managed_agent(
769741
input.model,
770742
input.provider,
771743
input.system_prompt,
772-
);
744+
)?;
773745
if let Some(parallelism) = input.parallelism {
774746
record.parallelism = parallelism;
775747
}

‎desktop/src-tauri/src/commands/agent_models_tests.rs‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -509,7 +509,8 @@ fn linked_instance_ignores_model_provider_prompt_writes() {
509509
Some(Some("explicit-model".to_string())),
510510
Some(Some("explicit-prov".to_string())),
511511
Some(Some("explicit-prompt".to_string())),
512-
);
512+
)
513+
.unwrap();
513514

514515
assert!(
515516
record.model.is_none(),
@@ -560,7 +561,8 @@ fn definition_less_instance_accepts_model_provider_prompt_writes() {
560561
Some(Some("new-model".to_string())),
561562
Some(Some("new-prov".to_string())),
562563
Some(Some("new-prompt".to_string())),
563-
);
564+
)
565+
.unwrap();
564566

565567
assert_eq!(record.model.as_deref(), Some("new-model"));
566568
assert_eq!(record.provider.as_deref(), Some("new-prov"));

‎desktop/src-tauri/src/commands/agents.rs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
use nostr::{Keys, ToBech32};
22
use tauri::{AppHandle, State};
33

4+
use super::managed_agent_definition::validate_create_definition;
5+
46
use crate::{
57
app_state::AppState,
68
managed_agents::{
@@ -568,15 +570,13 @@ pub async fn create_managed_agent(
568570
state: State<'_, AppState>,
569571
) -> Result<CreateManagedAgentResponse, String> {
570572
let name = input.name.trim().to_string();
571-
if name.is_empty() {
572-
return Err("agent name is required".to_string());
573-
}
574573
let requested_persona_id = input
575574
.persona_id
576575
.as_deref()
577576
.map(str::trim)
578577
.filter(|value| !value.is_empty())
579578
.map(str::to_string);
579+
validate_create_definition(&name, requested_persona_id.as_deref(), &input)?;
580580
if let Some(parallelism) = input.parallelism {
581581
if !(1..=32).contains(&parallelism) {
582582
return Err("parallelism must be between 1 and 32".to_string());
Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
//! Managed-agent definition validation at local mutation boundaries.
2+
3+
use crate::managed_agents::{CreateManagedAgentRequest, ManagedAgentRecord};
4+
5+
pub(super) fn validate_create_definition(
6+
name: &str,
7+
persona_id: Option<&str>,
8+
input: &CreateManagedAgentRequest,
9+
) -> Result<(), String> {
10+
validate_definition_fields(name, persona_id, input.system_prompt.as_deref())
11+
}
12+
13+
fn validate_definition_fields(
14+
name: &str,
15+
persona_id: Option<&str>,
16+
system_prompt: Option<&str>,
17+
) -> Result<(), String> {
18+
crate::managed_agents::validate_managed_agent_definition_text(name, persona_id, system_prompt)
19+
.map_err(|error| format!("Managed agent definition is unsafe: {error}"))
20+
}
21+
22+
/// Apply definition-owned update fields, then validate the complete
23+
/// prospective definition before the caller can persist it.
24+
pub(super) fn apply_model_provider_prompt_update(
25+
record: &mut ManagedAgentRecord,
26+
model: Option<Option<String>>,
27+
provider: Option<Option<String>>,
28+
system_prompt: Option<Option<String>>,
29+
) -> Result<(), String> {
30+
if record.persona_id.is_none() {
31+
if let Some(model_update) = model {
32+
record.model = model_update;
33+
}
34+
if let Some(provider_update) = provider {
35+
record.provider = provider_update;
36+
}
37+
if let Some(prompt_update) = system_prompt {
38+
record.system_prompt = prompt_update;
39+
}
40+
}
41+
42+
validate_definition_fields(
43+
&record.name,
44+
record.persona_id.as_deref(),
45+
record.system_prompt.as_deref(),
46+
)
47+
}
48+
49+
#[cfg(test)]
50+
mod tests {
51+
use super::*;
52+
53+
fn standalone_record() -> ManagedAgentRecord {
54+
serde_json::from_value(serde_json::json!({
55+
"pubkey": "standalone1",
56+
"name": "standalone-agent",
57+
"private_key_nsec": "nsec1fake",
58+
"relay_url": "wss://localhost:3000",
59+
"acp_command": "buzz-acp",
60+
"agent_command": "goose",
61+
"agent_args": [],
62+
"mcp_command": "",
63+
"turn_timeout_seconds": 320,
64+
"system_prompt": "safe prompt",
65+
"model": null,
66+
"provider": null,
67+
"env_vars": {},
68+
"created_at": "2026-01-01T00:00:00Z",
69+
"updated_at": "2026-01-01T00:00:00Z",
70+
"last_started_at": null,
71+
"last_stopped_at": null,
72+
"last_exit_code": null,
73+
"last_error": null
74+
}))
75+
.expect("standalone agent record")
76+
}
77+
78+
fn create_request(system_prompt: &str) -> CreateManagedAgentRequest {
79+
serde_json::from_value(serde_json::json!({
80+
"name": "Reviewer",
81+
"systemPrompt": system_prompt
82+
}))
83+
.expect("create request")
84+
}
85+
86+
#[test]
87+
fn create_rejects_invisible_definition_less_name_or_prompt() {
88+
for (name, prompt, code) in [
89+
("Review\u{200B}er", "Review code.", "U+200B"),
90+
("Reviewer", "Review\u{202E} code.", "U+202E"),
91+
] {
92+
let input = create_request(prompt);
93+
let error = validate_create_definition(name, None, &input)
94+
.expect_err("create must reject unsafe definition text");
95+
assert!(error.contains(code), "unexpected error: {error}");
96+
}
97+
}
98+
99+
#[test]
100+
fn create_accepts_visible_multiline_definition_less_prompt() {
101+
let input = create_request("Review changes.\n\tCall out security risks.");
102+
validate_create_definition("Reviewer 🐝", None, &input)
103+
.expect("visible multiline instructions should remain valid");
104+
}
105+
106+
#[test]
107+
fn update_rejects_invisible_definition_less_name_or_prompt() {
108+
let mut unsafe_prompt = standalone_record();
109+
let error = apply_model_provider_prompt_update(
110+
&mut unsafe_prompt,
111+
None,
112+
None,
113+
Some(Some("Review\u{200B} code.".to_string())),
114+
)
115+
.expect_err("definition-less prompt update must reject invisible text");
116+
assert!(error.contains("U+200B"), "unexpected error: {error}");
117+
118+
let mut unsafe_name = standalone_record();
119+
unsafe_name.name = "Review\u{202E}er".to_string();
120+
let error = apply_model_provider_prompt_update(&mut unsafe_name, None, None, None)
121+
.expect_err("definition-less name update must reject formatting controls");
122+
assert!(error.contains("U+202E"), "unexpected error: {error}");
123+
}
124+
}

‎desktop/src-tauri/src/commands/mod.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ mod identity_archive;
2525
mod join_policy;
2626
mod legacy_storage;
2727
mod link_preview;
28+
mod managed_agent_definition;
2829
pub(crate) mod media;
2930
mod media_animated;
3031
mod media_download;

‎desktop/src-tauri/src/commands/personas/create.rs‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ use uuid::Uuid;
77
use crate::{
88
app_state::AppState,
99
managed_agents::{
10-
apply_persona_behavior, load_personas, save_personas, try_regenerate_nest, AgentDefinition,
11-
CatalogSource, CreatePersonaRequest,
10+
apply_persona_behavior, load_personas, save_personas, try_regenerate_nest,
11+
validate_agent_definition_text, AgentDefinition, CatalogSource, CreatePersonaRequest,
1212
},
1313
util::now_iso,
1414
};
@@ -25,7 +25,10 @@ pub async fn create_persona(
2525
let state = app.state::<AppState>();
2626
let display_name = trim_required(&input.display_name, "Display name")?;
2727
// System prompt optional: core memory is auto-injected. Empty is valid.
28-
let system_prompt = input.system_prompt.trim().to_string();
28+
// Preserve it byte-for-byte: shared/import review surfaces show this
29+
// exact string before the ACP harness executes it.
30+
let system_prompt = input.system_prompt.clone();
31+
validate_agent_definition_text(&display_name, &system_prompt)?;
2932
let avatar_url = trim_optional(input.avatar_url);
3033
let runtime = trim_optional(input.runtime);
3134
let model = trim_optional(input.model);

‎desktop/src-tauri/src/commands/personas/inbound.rs‎

Lines changed: 35 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -102,12 +102,21 @@ fn reconcile_inbound_persona_event_blocking(
102102

103103
// The d-tag identifies the record within its kind. Persona derives it from
104104
// the parsed record (`persona_d_tag`); team/agent carry it as the event's
105-
// d-tag directly. The persona is parsed once here and reused in the apply
106-
// branch below — team/agent content is parsed in-branch since their d-tag
107-
// comes from the event tag, not the content.
105+
// d-tag directly. Definition-bearing content is parsed and validated once
106+
// here, before retention, then reused in the apply branch below. This keeps
107+
// an unsafe event out of both the retention database and the local store.
108108
let inbound_persona = (kind == KIND_PERSONA)
109109
.then(|| persona_from_event(&event))
110110
.transpose()?;
111+
if let Some(persona) = &inbound_persona {
112+
validate_inbound_persona_definition(persona)?;
113+
}
114+
let inbound_managed_agent = (kind == KIND_MANAGED_AGENT)
115+
.then(|| managed_agent_content_from_event(&event))
116+
.transpose()?;
117+
if let Some(managed_agent) = &inbound_managed_agent {
118+
validate_inbound_managed_agent_definition(managed_agent)?;
119+
}
111120
let d_tag = match &inbound_persona {
112121
Some(persona) => persona_d_tag(persona),
113122
None => event_d_tag(&event)?,
@@ -164,11 +173,10 @@ fn reconcile_inbound_persona_event_blocking(
164173
}
165174
KIND_MANAGED_AGENT => {
166175
let mut agents = load_managed_agents(&app)?;
167-
apply_inbound_managed_agent(
168-
&mut agents,
169-
&d_tag,
170-
managed_agent_content_from_event(&event)?,
171-
);
176+
let managed_agent = inbound_managed_agent.ok_or_else(|| {
177+
"managed-agent content was not parsed before retention".to_string()
178+
})?;
179+
apply_inbound_managed_agent(&mut agents, &d_tag, managed_agent);
172180
save_managed_agents(&app, &agents)?;
173181
}
174182
_ => unreachable!("kind gated above"),
@@ -182,6 +190,25 @@ fn reconcile_inbound_persona_event_blocking(
182190
Ok(())
183191
}
184192

193+
fn validate_inbound_persona_definition(persona: &AgentDefinition) -> Result<(), String> {
194+
crate::managed_agents::validate_agent_definition_text(
195+
&persona.display_name,
196+
&persona.system_prompt,
197+
)
198+
.map_err(|error| format!("Inbound persona definition is unsafe: {error}"))
199+
}
200+
201+
fn validate_inbound_managed_agent_definition(
202+
managed_agent: &ManagedAgentEventContent,
203+
) -> Result<(), String> {
204+
crate::managed_agents::validate_managed_agent_definition_text(
205+
&managed_agent.name,
206+
managed_agent.persona_id.as_deref(),
207+
managed_agent.system_prompt.as_deref(),
208+
)
209+
.map_err(|error| format!("Inbound managed-agent definition is unsafe: {error}"))
210+
}
211+
185212
/// Parse an inbound wire event and enforce the signature gate. Everything
186213
/// downstream trusts `event.pubkey` (ownership routing, tombstone scoping,
187214
/// behavioral-quad application), so a forged pubkey must die here — the

‎desktop/src-tauri/src/commands/personas/inbound/inbound_tests.rs‎

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
use super::*;
55
use std::collections::BTreeMap;
66

7-
const UUID: &str = "11111111-2222-3333-4444-555555555555";
7+
const UUID: &str = "11111111-2222-3333-4444-555555555555"; // sadscan:disable sq.pii.cc.visa -- fixed test UUID
88

99
/// A local in-app persona: `source_team_persona_slug` is None, so its d-tag
1010
/// IS its UUID id. Carries env_vars + source_team that must survive a patch.
@@ -673,3 +673,63 @@ fn inbound_gate_accepts_validly_signed_event() {
673673
let parsed = parse_verified_inbound_event(&event.as_json()).unwrap();
674674
assert_eq!(parsed.pubkey, keys.public_key());
675675
}
676+
677+
#[test]
678+
fn inbound_persona_rejects_invisible_definition_text() {
679+
let mut inbound = inbound_for("unsafe", "Remote");
680+
inbound.system_prompt = "Review\u{200B} code.".to_string();
681+
682+
let error = validate_inbound_persona_definition(&inbound)
683+
.expect_err("relay sync must reject invisible instructions");
684+
685+
assert!(error.contains("U+200B"));
686+
}
687+
688+
fn inbound_managed_agent_content(
689+
name: &str,
690+
persona_id: Option<&str>,
691+
system_prompt: Option<&str>,
692+
) -> crate::managed_agents::agent_events::ManagedAgentEventContent {
693+
crate::managed_agents::agent_events::ManagedAgentEventContent {
694+
name: name.to_string(),
695+
persona_id: persona_id.map(str::to_string),
696+
system_prompt: system_prompt.map(str::to_string),
697+
model: None,
698+
provider: None,
699+
persona_source_version: None,
700+
parallelism: 1,
701+
respond_to: crate::managed_agents::RespondTo::OwnerOnly,
702+
respond_to_allowlist: vec![],
703+
}
704+
}
705+
706+
#[test]
707+
fn inbound_definition_less_agent_rejects_invisible_prompt() {
708+
let inbound = inbound_managed_agent_content("Remote Agent", None, Some("Review\u{200B} code."));
709+
710+
let error = validate_inbound_managed_agent_definition(&inbound)
711+
.expect_err("definition-less sync must reject invisible instructions");
712+
713+
assert!(error.contains("U+200B"));
714+
}
715+
716+
#[test]
717+
fn inbound_managed_agent_rejects_bidirectional_name() {
718+
let inbound = inbound_managed_agent_content("Remote\u{202E} Agent", None, None);
719+
720+
let error = validate_inbound_managed_agent_definition(&inbound)
721+
.expect_err("managed-agent sync must reject bidirectional names");
722+
723+
assert!(error.contains("U+202E"));
724+
}
725+
726+
#[test]
727+
fn inbound_definition_less_agent_accepts_visible_multiline_prompt() {
728+
let inbound = inbound_managed_agent_content(
729+
"Remote Agent",
730+
None,
731+
Some("Review code.\n\tCall out security risks."),
732+
);
733+
734+
assert!(validate_inbound_managed_agent_definition(&inbound).is_ok());
735+
}

0 commit comments

Comments
 (0)