Skip to content

Commit 27910e3

Browse files
shengyfuCopilot
andcommitted
Clarify bounded ancestor observations and publication checks
Co-authored-by: Copilot App <[email protected]> Copilot-Session: 9776b17f-6a2d-4c47-9e89-0f8113d14b95
1 parent f0cd820 commit 27910e3

7 files changed

Lines changed: 129 additions & 34 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -957,6 +957,10 @@ Namespace operation-worker limits do not need to be raised. Exact checkout bytes
957957
identity checks, cancellation and final readiness gates are preserved; clean Git
958958
status never replaces first-attachment verification. Receipts and status expose
959959
[phase timings and resource counters](SHARED_WORKTREE_INDEXES.md#bounded-managed-reconciliation).
960+
Ancestor-ID comparisons detect observed substitutions, but are not lifetime
961+
pins: IDs may be reused after batch handles close. Reconciliation retains its
962+
per-read containment and publication-epoch checks, not an atomic filesystem
963+
snapshot or uninterrupted ancestor-lifetime guarantee.
960964
A [same-binary Windows/ReFS measurement](SHARED_INDEX_BENCHMARKS.md#large-checkout-reconciliation-rc2)
961965
completed a fresh warm-base attachment in 186.144 seconds versus 542.469 seconds
962966
for a standalone full build (2.914x), while still reading every required file.

‎SHARED_INDEX_BENCHMARKS.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,11 @@ error. That correction and its deterministic regressions are not part of the
4848
measured executable; these timings remain attributed to `58fdda8`, not later
4949
PR heads.
5050

51+
The later ancestor-observation naming, documentation and regression follow-up
52+
does not change reconciliation behavior or these measurements. The final ID
53+
comparison detects observed substitutions; it does not establish uninterrupted
54+
ancestor lifetime or an atomic filesystem snapshot.
55+
5156
Standalone ran first; the file cache was **not flushed**. Implementation
5257
builds/tests were paused during measurement, but this was not an exclusive
5358
host. Native watching was disabled to isolate initial attachment. These

‎SHARED_WORKTREE_INDEXES.md‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1018,10 +1018,18 @@ parent rather than accumulating pins for unrelated directories.
10181018

10191019
Siblings share one scoped, handle-rooted ancestor chain per bounded batch,
10201020
including siblings assigned to different workers. Child containment, reparse
1021-
rejection, file identity and before/after read checks remain mandatory. Ancestor
1022-
identities are checked with bounded parallelism when the batch ends and again
1023-
before assembly publishes; completed batches retain identities, not directory
1024-
handles. Compute credits, parent pins and input bytes are admitted in that order.
1021+
rejection, file identity and before/after read checks remain mandatory. Observed
1022+
ancestor IDs are compared with bounded parallelism when the batch ends and again
1023+
before assembly publishes; completed batches retain observations, not directory
1024+
handles. These comparisons detect observed substitutions, not uninterrupted
1025+
ancestor lifetime: filesystems can reuse IDs after handles close, and paths can
1026+
change after the comparison. Read containment uses the live root, batch and child
1027+
handles; root and input-epoch checks independently gate publication. As before
1028+
batching, reconciliation is not an atomic filesystem snapshot. This is not a new
1029+
relaxation of initial verified reads or a consequence of disabling native safety
1030+
rescans.
1031+
1032+
Compute credits, parent pins and input bytes are admitted in that order.
10251033
Native metadata obtained by a verified open is reused, not reacquired redundantly
10261034
before reading. Workers drain on cancellation/error before their reservation or
10271035
checkout handles can be released.

‎tgrep-core/src/rooted.rs‎

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -115,8 +115,8 @@ impl RootedDir {
115115
self.open_file_with(relative, |_| {})
116116
}
117117

118-
/// Pin one ancestor chain for a bounded group of sibling reads. The caller
119-
/// must finish the batch and revalidate its returned evidence at publication.
118+
/// Pin one ancestor chain for a bounded group of sibling reads. Finishing
119+
/// releases the pins; the returned IDs are observations, not lifetime proof.
120120
pub(crate) fn batch(&self, parent: &Path) -> io::Result<RootedBatch<'_>> {
121121
let names = if parent.as_os_str().is_empty() {
122122
Vec::new()
@@ -158,7 +158,7 @@ impl RootedDir {
158158
let resolved = final_path_of(guards.last().unwrap_or(&self.directory))?;
159159
Ok(RootedBatch {
160160
root: self,
161-
evidence: ParentEvidence {
161+
observation: ParentObservation {
162162
path: parent.into(),
163163
identities,
164164
},
@@ -243,14 +243,17 @@ impl RootedDir {
243243
}
244244
}
245245

246-
/// Stores identities, not open handles: completed batches must not keep
247-
/// checkout directories pinned while later batches or checkpoints run.
248-
pub(crate) struct ParentEvidence {
246+
/// Native IDs observed during a rooted batch, not an ancestor lifetime pin.
247+
///
248+
/// IDs may be reused after handles close. A later comparison can reject an
249+
/// observed substitution, but cannot prove continuity through publication.
250+
/// Read containment relies on live rooted handles, not these retained IDs.
251+
pub(crate) struct ParentObservation {
249252
path: PathBuf,
250253
identities: Vec<crate::managed::FileIdentity>,
251254
}
252255

253-
impl ParentEvidence {
256+
impl ParentObservation {
254257
pub(crate) fn verification_memory_bytes(&self, root: &RootedDir) -> u64 {
255258
root.batch_memory_bytes(&self.path)
256259
}
@@ -259,9 +262,9 @@ impl ParentEvidence {
259262
RootedDir::batch_handles(&self.path)
260263
}
261264

262-
pub(crate) fn verify(&self, root: &RootedDir) -> io::Result<()> {
265+
pub(crate) fn check_current_ids(&self, root: &RootedDir) -> io::Result<()> {
263266
let current = root.batch(&self.path)?;
264-
if current.evidence.identities != self.identities {
267+
if current.observation.identities != self.identities {
265268
return Err(Error::new(
266269
ErrorKind::InvalidInput,
267270
"batch ancestor identity changed",
@@ -279,7 +282,7 @@ impl ParentEvidence {
279282

280283
pub(crate) struct RootedBatch<'a> {
281284
root: &'a RootedDir,
282-
evidence: ParentEvidence,
285+
observation: ParentObservation,
283286
guards: Vec<File>,
284287
#[cfg(windows)]
285288
path: PathBuf,
@@ -301,7 +304,7 @@ impl RootedBatch<'_> {
301304
}
302305

303306
pub(crate) fn open_observed(&self, relative: &Path) -> io::Result<OpenedFile> {
304-
if relative.parent() != Some(self.evidence.path.as_path()) {
307+
if relative.parent() != Some(self.observation.path.as_path()) {
305308
return Err(Error::new(
306309
ErrorKind::InvalidInput,
307310
"file is not a batch sibling",
@@ -347,16 +350,16 @@ impl RootedBatch<'_> {
347350
}
348351
}
349352

350-
pub(crate) fn finish(self) -> io::Result<ParentEvidence> {
353+
pub(crate) fn finish(self) -> io::Result<ParentObservation> {
351354
#[cfg(windows)]
352-
debug_assert_eq!(self.guards.len(), self.evidence.identities.len());
355+
debug_assert_eq!(self.guards.len(), self.observation.identities.len());
353356
#[cfg(unix)]
354357
debug_assert_eq!(
355358
self.guards.len(),
356-
usize::from(!self.evidence.identities.is_empty())
359+
usize::from(!self.observation.identities.is_empty())
357360
);
358-
self.evidence.verify(self.root)?;
359-
Ok(self.evidence)
361+
self.observation.check_current_ids(self.root)?;
362+
Ok(self.observation)
360363
}
361364
}
362365

‎tgrep-core/src/rooted/tests.rs‎

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -110,11 +110,40 @@ fn sibling_batches_release_pins_and_recheck_ancestor_identity() {
110110
}
111111
#[cfg(windows)]
112112
assert!(fs::rename(&parent, temp.path().join("saved")).is_err());
113-
let evidence = batch.finish().unwrap();
114-
evidence.verify(&root).unwrap();
113+
let observation = batch.finish().unwrap();
114+
observation.check_current_ids(&root).unwrap();
115115
fs::rename(&parent, temp.path().join("saved")).unwrap();
116116
fs::create_dir(&parent).unwrap();
117-
assert!(evidence.verify(&root).is_err());
117+
assert!(observation.check_current_ids(&root).is_err());
118+
}
119+
120+
#[test]
121+
fn ancestor_observations_are_point_in_time_not_lifetime_pins() {
122+
let temp = tempfile::tempdir().unwrap();
123+
let parent = temp.path().join("parent");
124+
let saved = temp.path().join("saved");
125+
fs::create_dir(&parent).unwrap();
126+
fs::write(parent.join("file"), "original").unwrap();
127+
let root = RootedDir::open(temp.path()).unwrap();
128+
let batch = root.batch(Path::new("parent")).unwrap();
129+
assert_eq!(
130+
contents(batch.open_file(Path::new("parent/file")).unwrap()),
131+
"original"
132+
);
133+
let observation = batch.finish().unwrap();
134+
fs::rename(&parent, &saved).unwrap();
135+
fs::create_dir(&parent).unwrap();
136+
fs::write(parent.join("file"), "replacement").unwrap();
137+
assert!(observation.check_current_ids(&root).is_err());
138+
fs::remove_file(parent.join("file")).unwrap();
139+
fs::remove_dir(&parent).unwrap();
140+
fs::rename(&saved, &parent).unwrap();
141+
// Restoring the original object restores ID equality, not path continuity.
142+
observation.check_current_ids(&root).unwrap();
143+
assert_eq!(
144+
contents(root.open_file(Path::new("parent/file")).unwrap()),
145+
"original"
146+
);
118147
}
119148

120149
#[test]
@@ -132,11 +161,11 @@ fn completed_batch_never_accepts_a_reparse_replacement() {
132161
contents(batch.open_file(Path::new("parent/file")).unwrap()),
133162
"original"
134163
);
135-
let evidence = batch.finish().unwrap();
164+
let observation = batch.finish().unwrap();
136165
fs::rename(&parent, temp.path().join("saved")).unwrap();
137166
link_directory(&other, &parent);
138167
assert!(root.batch(Path::new("parent")).is_err());
139-
assert!(evidence.verify(&root).is_err());
168+
assert!(observation.check_current_ids(&root).is_err());
140169
}
141170

142171
#[cfg(unix)]

‎tgrep-core/src/worktrees.rs‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ use crate::hybrid::HybridIndex;
2222
use crate::live::LiveIndex;
2323
use crate::meta::{ContentId, FileVersion, file_version};
2424
use crate::query::QueryPlan;
25-
use crate::rooted::{OpenedFile, ParentEvidence, RootedBatch, RootedDir};
25+
use crate::rooted::{OpenedFile, ParentObservation, RootedBatch, RootedDir};
2626
use crate::trigram::TrigramMaskMap;
2727
use crate::visibility::PathVisibility;
2828
use crate::walker::{self, MetaWalkOptions};
@@ -578,7 +578,7 @@ impl WorktreeView {
578578
let _memory = permit
579579
.admit_read(parent.verification_memory_bytes(&self.files))
580580
.map_err(crate::Error::from)?;
581-
parent.verify(&self.files)?;
581+
parent.check_current_ids(&self.files)?;
582582
}
583583
Ok(())
584584
})
@@ -594,7 +594,7 @@ impl WorktreeView {
594594
}
595595
} else {
596596
for parent in &prepared.parents {
597-
parent.verify(&self.files)?;
597+
parent.check_current_ids(&self.files)?;
598598
}
599599
}
600600
Ok(())
@@ -1642,7 +1642,7 @@ impl WorktreeSnapshot<'_> {
16421642

16431643
struct Prepared {
16441644
memory: Option<crate::managed::work::MemoryCharge>,
1645-
parents: Vec<ParentEvidence>,
1645+
parents: Vec<ParentObservation>,
16461646
evidence: BTreeMap<String, Evidence>,
16471647
visibility: PathVisibility,
16481648
upserts: BTreeMap<String, TrigramMaskMap>,
@@ -1750,11 +1750,11 @@ impl<'root, 'inputs> VerificationBatch<'root, 'inputs> {
17501750
mut self,
17511751
workers: usize,
17521752
) -> Result<(
1753-
Vec<ParentEvidence>,
1753+
Vec<ParentObservation>,
17541754
Option<crate::managed::work::MemoryCharge>,
17551755
)> {
17561756
let mut groups: Vec<_> = self.parents.into_values().collect();
1757-
let verify = |groups: &mut [ParentBatch<'_>]| -> Result<Vec<ParentEvidence>> {
1757+
let verify = |groups: &mut [ParentBatch<'_>]| -> Result<Vec<ParentObservation>> {
17581758
let mut parents = Vec::with_capacity(groups.len());
17591759
for group in groups {
17601760
if let Some(permit) = self.permit {
@@ -1788,9 +1788,9 @@ impl<'root, 'inputs> VerificationBatch<'root, 'inputs> {
17881788
if let Some(error) = failure {
17891789
return Err(error);
17901790
}
1791-
let mut bytes = parents.capacity() as u64 * std::mem::size_of::<ParentEvidence>() as u64;
1792-
for evidence in &parents {
1793-
bytes += evidence.memory_bytes();
1791+
let mut bytes = parents.capacity() as u64 * std::mem::size_of::<ParentObservation>() as u64;
1792+
for observation in &parents {
1793+
bytes += observation.memory_bytes();
17941794
}
17951795
if let Some(memory) = &mut self.memory {
17961796
memory.resize(bytes).map_err(crate::Error::from)?;

‎tgrep-core/src/worktrees/tests.rs‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1455,6 +1455,52 @@ fn root_replacement_before_publication_keeps_the_view_not_ready() {
14551455
));
14561456
}
14571457

1458+
#[test]
1459+
fn observed_parent_replacement_after_batch_drain_blocks_publication() {
1460+
for managed in [false, true] {
1461+
let fixture = Fixture::new();
1462+
write(&fixture.root, "dir/nested/file.txt", b"original marker");
1463+
commit(&fixture.root);
1464+
let exercise = |view: &WorktreeView, permit: Option<&Arc<crate::managed::WorkPermit>>| {
1465+
let result = view.refresh_controlled_inner(
1466+
|| {},
1467+
|| {
1468+
fs::rename(fixture.root.join("dir"), fixture._temp.path().join("saved"))
1469+
.unwrap();
1470+
write(&fixture.root, "dir/nested/file.txt", b"replacement marker");
1471+
view.files.verify_root().unwrap();
1472+
},
1473+
permit,
1474+
);
1475+
assert!(matches!(result, Err(WorktreeError::Io(_))), "{result:?}");
1476+
let status = view.status().unwrap();
1477+
assert!(!status.ready);
1478+
assert!(status.full_required);
1479+
assert!(status.published_epoch.is_none());
1480+
assert!(matches!(
1481+
view.with_snapshot(|_| ()),
1482+
Err(WorktreeError::NotReady)
1483+
));
1484+
let released = fixture._temp.path().join("after-check");
1485+
fs::rename(fixture.root.join("dir"), &released).unwrap();
1486+
fs::rename(&released, fixture.root.join("dir")).unwrap();
1487+
let repaired = view.refresh_controlled_inner(|| {}, || {}, permit).unwrap();
1488+
assert!(repaired.full);
1489+
assert_eq!(repaired.files_read, 1);
1490+
assert_eq!(repaired.content_reads_avoided, 0);
1491+
assert!(candidates(view, "original").is_empty());
1492+
assert_eq!(matches(view, "replacement"), ["dir/nested/file.txt"]);
1493+
};
1494+
if managed {
1495+
let (_namespace, permit, view) = fixture.controlled();
1496+
exercise(&view, Some(&permit));
1497+
} else {
1498+
let view = fixture.view(&fixture.root, &fixture.generation());
1499+
exercise(&view, None);
1500+
}
1501+
}
1502+
}
1503+
14581504
#[cfg(unix)]
14591505
#[test]
14601506
fn fifo_swap_after_discovery_is_bounded_and_keeps_the_view_not_ready() {

0 commit comments

Comments
 (0)