Skip to content

Commit 37dad04

Browse files
tlongwell-blockDawn
andcommitted
fix(relay/handlers): seed empty manifest pointer on kind:30617 announce
The architectural split Eva's live e2e caught: 30617 created the on-disk bare repo but no S3 pointer, while the read path (info_refs/upload_pack) now hydrates from the pointer. Pointer-absent ⇒ Ok(None) ⇒ 404 by Max's intentional fail-closed, so announce-then-clone returned 404 for every freshly-announced repo. Fix: establish the invariant "repo announced ⟺ pointer exists" by seeding an empty-manifest pointer at the end of `handle_git_repo_ announcement`. After seed: - `info_refs` stays strictly fail-closed: pointer-absent means never-announced, not "announced but no pushes." Max's blocker preserved exactly. - `live_hydrate_empty_repo` proves clone of an empty hydrated repo works. - First push CASes the seeded pointer via `IfMatch(etag)`, no special-case branch. The `IfNoneMatchStar` arm of `cas_publish` becomes dead code for announced repos. Two new private helpers in `side_effects.rs`: - `seed_manifest_pointer(state, owner, repo)` — `put_manifest(empty)` + `put_pointer(IfNoneMatchStar)`. **Idempotency is constructive, not trusted**: a `CasOutcome::LostRace` is treated as success ONLY if the existing pointer body matches the empty manifest's digest. Any other value (stale from a prior lifecycle, real misconfiguration) surfaces as `anyhow!` — Max's guardrail #1. - `emit_initial_ref_state(state, owner, repo)` — kind:30618 over the seeded empty manifest using Sami's `build_ref_state_event`. Pointer is the commit; this event is the derived "repo exists, empty" signal — Max's guardrail #2. Non-fatal on failure: manifest is truth, 30618 is just notification. Empty manifests across all repos share canonical bytes (deterministic serialization by construction) ⟹ same digest ⟹ `put_manifest` is idempotent at the store level. One blob, many pointers. Pinned by `empty_manifest_validates` test in manifest.rs with byte-stable canonical bytes — locks the digest so future serde version bumps don't silently shift it. Rollback on seed failure: remove the on-disk repo + the name reservation, same pattern as hook-install failure. A successful seed but failed emit leaves the pointer in place (correctly — it's the source of truth). 239/239 relay tests (was 238 + 1 new), clippy + fmt clean. Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@users.noreply.sprout> Signed-off-by: tlongwell-block <[email protected]>
1 parent 11085ba commit 37dad04

2 files changed

Lines changed: 180 additions & 0 deletions

File tree

‎crates/sprout-relay/src/api/git/manifest.rs‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,30 @@ mod tests {
297297
sample().validate().expect("sample manifest must validate");
298298
}
299299

300+
/// The empty manifest is the announce-time seed (`side_effects.rs::
301+
/// seed_manifest_pointer`). It must validate — otherwise repo announce
302+
/// would fail before the pointer can be seeded, and the read path would
303+
/// 404 every freshly-announced repo. This pins that contract.
304+
#[test]
305+
fn empty_manifest_validates() {
306+
let m = Manifest {
307+
version: MANIFEST_VERSION,
308+
head: "refs/heads/main".into(),
309+
refs: BTreeMap::new(),
310+
packs: Vec::new(),
311+
parent: None,
312+
};
313+
m.validate().expect("empty manifest is the announce-seed");
314+
// Canonical bytes must be deterministic + stable so all empty manifests
315+
// share one digest (idempotent put_manifest, one shared S3 object).
316+
let bytes = m.canonical_bytes().expect("serialize");
317+
let s = std::str::from_utf8(&bytes).expect("utf8");
318+
assert_eq!(
319+
s,
320+
r#"{"version":1,"head":"refs/heads/main","refs":{},"packs":[],"parent":null}"#
321+
);
322+
}
323+
300324
#[test]
301325
fn validate_rejects_empty_head() {
302326
let mut m = sample();

‎crates/sprout-relay/src/handlers/side_effects.rs‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1871,13 +1871,169 @@ async fn handle_git_repo_announcement(event: &Event, state: &Arc<AppState>) -> a
18711871
anyhow::anyhow!("failed to install pre-receive hook: {e}")
18721872
})?;
18731873

1874+
// Seed the empty-manifest pointer in object storage. Establishes the
1875+
// invariant "repo announced ⟺ pointer exists" so the read path can rely
1876+
// on pointer-absent meaning never-announced (not just no-pushes-yet),
1877+
// keeping `info_refs`'s fail-closed `Ok(None) → 404` unambiguous.
1878+
// First push CASes the seeded pointer normally — no special-case branch.
1879+
seed_manifest_pointer(state, &owner_hex, &repo_id)
1880+
.await
1881+
.map_err(|e| {
1882+
// Pointer seed failure leaves the on-disk repo + name reservation
1883+
// without a clone-able pointer, which is exactly the broken state
1884+
// the seed exists to prevent. Unwind both so the announce is
1885+
// either fully consummated or fully rolled back.
1886+
let _ = std::fs::remove_dir_all(&repo_dir);
1887+
let _ = std::fs::remove_dir(&reservation);
1888+
anyhow::anyhow!("failed to seed manifest pointer: {e}")
1889+
})?;
1890+
18741891
info!(
18751892
repo_id = %repo_id,
18761893
owner = %owner_hex,
18771894
path = %repo_dir.display(),
18781895
"bare git repo created from kind:30617 announcement"
18791896
);
18801897

1898+
// Derived after the pointer commits: kind:30618 ref-state event over the
1899+
// seeded empty manifest. Pointer is the commit; this event is the
1900+
// notification that the repo exists (with empty refs) so subscribers see
1901+
// a first signal without waiting for the first push.
1902+
if let Err(e) = emit_initial_ref_state(state, &owner_hex, &repo_id).await {
1903+
// Non-fatal: the manifest is the source of truth; this is just the
1904+
// derived notification. A failure here means subscribers miss the
1905+
// "repo now exists" event, but clone/push still works.
1906+
warn!(
1907+
repo_id = %repo_id,
1908+
owner = %owner_hex,
1909+
error = %e,
1910+
"failed to emit initial kind:30618 ref state (non-fatal)"
1911+
);
1912+
}
1913+
1914+
Ok(())
1915+
}
1916+
1917+
/// Default symbolic HEAD for a freshly-announced (empty) repo. Matches
1918+
/// `init.defaultBranch=main` (git ≥ 2.28) and the seed used by
1919+
/// `live_hydrate_empty_repo`. Pinned in one place so the seeded manifest
1920+
/// and the initial kind:30618 emission can't drift.
1921+
///
1922+
/// The first push's `cas_publish` overwrites this with the real symbolic
1923+
/// HEAD observed in the receive-pack workspace via standard CAS, so the
1924+
/// default is a stand-in, not a permanent commitment.
1925+
const DEFAULT_HEAD: &str = "refs/heads/main";
1926+
1927+
/// Seed the manifest-pointer for a newly-announced repo with an empty manifest.
1928+
///
1929+
/// Idempotent: a `CasOutcome::LostRace` is treated as success **only if** the
1930+
/// existing pointer names the same empty manifest digest. Any other pre-existing
1931+
/// pointer body (e.g. a non-empty manifest from a previous announce/push pair
1932+
/// for the same `(owner, repo)`) surfaces as an error rather than silently
1933+
/// succeeding — that would mask a real misconfiguration.
1934+
async fn seed_manifest_pointer(
1935+
state: &Arc<AppState>,
1936+
owner_hex: &str,
1937+
repo_id: &str,
1938+
) -> anyhow::Result<()> {
1939+
use crate::api::git::manifest::{pointer_key, Manifest, MANIFEST_VERSION};
1940+
use crate::api::git::store::{CasOutcome, Precond};
1941+
use std::collections::BTreeMap;
1942+
1943+
// The empty manifest. All empty manifests across all repos share canonical
1944+
// bytes — by design — so `put_manifest` is idempotent at the store level
1945+
// too.
1946+
let empty = Manifest {
1947+
version: MANIFEST_VERSION,
1948+
head: DEFAULT_HEAD.to_string(),
1949+
refs: BTreeMap::new(),
1950+
packs: Vec::new(),
1951+
parent: None,
1952+
};
1953+
empty
1954+
.validate()
1955+
.map_err(|e| anyhow::anyhow!("empty manifest failed validation: {e}"))?;
1956+
let bytes = empty
1957+
.canonical_bytes()
1958+
.map_err(|e| anyhow::anyhow!("empty manifest serialize: {e}"))?;
1959+
let manifest_key = state
1960+
.git_store
1961+
.put_manifest(&bytes)
1962+
.await
1963+
.map_err(|e| anyhow::anyhow!("put_manifest: {e}"))?;
1964+
let digest = manifest_key
1965+
.strip_prefix("manifests/")
1966+
.ok_or_else(|| anyhow::anyhow!("put_manifest returned non-standard key: {manifest_key}"))?;
1967+
1968+
let pkey = pointer_key(owner_hex, repo_id);
1969+
let outcome = state
1970+
.git_store
1971+
.put_pointer(&pkey, digest.as_bytes(), Precond::IfNoneMatchStar)
1972+
.await
1973+
.map_err(|e| anyhow::anyhow!("put_pointer: {e}"))?;
1974+
match outcome {
1975+
CasOutcome::Won(_) => Ok(()),
1976+
CasOutcome::LostRace => {
1977+
// Pointer already exists. Idempotency check: only treat as success
1978+
// if it names the same empty manifest digest. Any other value is
1979+
// either a stale pointer from a prior repo lifecycle for the same
1980+
// (owner, repo) or a real misconfiguration — surface, don't swallow.
1981+
let (_etag, body) = state
1982+
.git_store
1983+
.get_pointer(&pkey)
1984+
.await
1985+
.map_err(|e| anyhow::anyhow!("re-read pointer after LostRace: {e}"))?
1986+
.ok_or_else(|| anyhow::anyhow!("pointer vanished after LostRace race"))?;
1987+
let existing = std::str::from_utf8(&body)
1988+
.map_err(|e| anyhow::anyhow!("pointer body not utf-8: {e}"))?
1989+
.trim();
1990+
if existing != digest {
1991+
return Err(anyhow::anyhow!(
1992+
"repo '{repo_id}' for owner {owner_hex} already has a non-empty pointer \
1993+
({existing}); refusing to overwrite via announce"
1994+
));
1995+
}
1996+
Ok(())
1997+
}
1998+
}
1999+
}
2000+
2001+
/// Emit the initial kind:30618 ref-state event for a freshly-announced repo.
2002+
///
2003+
/// The seeded empty manifest is the source of truth; this event is the
2004+
/// derived notification. Fires once per announce, signed by the relay,
2005+
/// carrying the announcer's pubkey in the `p` tag (sprout extension).
2006+
async fn emit_initial_ref_state(
2007+
state: &Arc<AppState>,
2008+
owner_hex: &str,
2009+
repo_id: &str,
2010+
) -> anyhow::Result<()> {
2011+
use crate::api::git::manifest_event::{build_ref_state_event, RefStateInputs};
2012+
use std::collections::BTreeMap;
2013+
2014+
let empty_refs: BTreeMap<String, String> = BTreeMap::new();
2015+
let inputs = RefStateInputs {
2016+
repo_id,
2017+
head: DEFAULT_HEAD,
2018+
refs: &empty_refs,
2019+
actor_pubkey_hex: owner_hex,
2020+
};
2021+
let event = build_ref_state_event(&inputs, &state.relay_keypair)
2022+
.map_err(|e| anyhow::anyhow!("build_ref_state_event: {e}"))?;
2023+
let (stored, was_inserted) = state
2024+
.db
2025+
.insert_event(&event, None)
2026+
.await
2027+
.map_err(|e| anyhow::anyhow!("insert kind:30618: {e}"))?;
2028+
if was_inserted {
2029+
let matches = state.sub_registry.fan_out(&stored);
2030+
for (conn_id, sub_id) in matches {
2031+
let _ = state.conn_manager.send_to(
2032+
conn_id,
2033+
crate::protocol::RelayMessage::event(&sub_id, &stored.event),
2034+
);
2035+
}
2036+
}
18812037
Ok(())
18822038
}
18832039

0 commit comments

Comments
 (0)