Skip to content

Commit e16d4bd

Browse files
committed
fix(querier): review feedback — restore RFC 0046 §7, session test, lint rationale
- The squash-rebase had silently reverted #772's resolved RFC 0046 §7 checklist; restored to main's version (Copilot + CodeRabbit caught the regression). - `SessionConfig` is now qualified inside `exec::session()` instead of riding the crate-root import surface it alone needed. - New unit test pins `session()`'s collect_statistics=false so the apache/datafusion#24769 workaround cannot be dropped silently. - The `duration_suboptimal_units` rationale was factually wrong (`from_mins` is stable since 1.91; zero `from_days` suggestions fire). The allow stays — all 26 flagged sites are 86_400 day-sentinels or epoch fixture stamps whose canonical form is seconds — but the comment now says the true reason. - RFC 0021 §5 reconciled with `green`: §5 lists only fired criteria; phase 2b's (.7 and the quick-xml half of .9) moved to a §5b forward-declaration block that re-enters §5 as red stubs when the object_store ≥ 0.14 trigger fires; RFC0021.1's scenario text updated to the re-baselined floors. Signed-off-by: Jens Holdgaard Pedersen <[email protected]>
1 parent 87b8c85 commit e16d4bd

5 files changed

Lines changed: 85 additions & 47 deletions

File tree

‎Cargo.toml‎

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -40,12 +40,6 @@ unsafe_code = "deny"
4040
# All member crates declare `[lints] workspace = true`, so this
4141
# propagates without per-crate edits; `cargo doc --workspace
4242
# --no-deps --all-features` is the local check.
43-
# `duration_suboptimal_units` (new in the toolchain the 1.94 MSRV admits)
44-
# suggests `Duration::from_days` / `from_mins`, which are **unstable**
45-
# (rust-lang/rust#120301) — the lint is un-actionable on stable, and the
46-
# flagged sites are honest anyway: `from_secs(86_400)` day-scale sentinels
47-
# and Unix-epoch fixture stamps, both of which read better in seconds.
48-
# Revisit when the constructors stabilise.
4943
[workspace.lints.rustdoc]
5044
broken_intra_doc_links = "deny"
5145
private_intra_doc_links = "deny"
@@ -54,11 +48,12 @@ bare_urls = "warn"
5448

5549
[workspace.lints.clippy]
5650
pedantic = { level = "warn", priority = -1 }
57-
# `duration_suboptimal_units` suggests `Duration::from_days` / `from_mins`,
58-
# which are **unstable** (rust-lang/rust#120301) — un-actionable on stable.
59-
# The flagged sites are honest in seconds anyway: `from_secs(86_400)`
60-
# day-scale "effectively never" sentinels and Unix-epoch fixture stamps.
61-
# Revisit when the constructors stabilise.
51+
# `duration_suboptimal_units` (clippy 1.95+; our clippy gate runs latest
52+
# stable) flags ~26 sites, every one a `from_secs` of either a
53+
# seconds-per-day sentinel (86_400) or a Unix-epoch fixture stamp. The
54+
# suggested `from_mins(1440)` / `from_mins(29585458)` rewrites lose the
55+
# canonical form each constant is recognised by, so the lint is noise
56+
# here — allowed, not fought site by site.
6257
duration_suboptimal_units = "allow"
6358

6459
# The profile that 'dist' will build with

‎crates/ourios-querier/src/exec.rs‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,11 +53,23 @@ pub(crate) enum SchemaMode<'a> {
5353
/// depend on; the RFC0044.7/.8 and RFC0007.1 pruning tests are the
5454
/// gate that will catch it either way.
5555
pub(crate) fn session() -> SessionContext {
56-
let mut config = SessionConfig::new();
56+
let mut config = datafusion::prelude::SessionConfig::new();
5757
config.options_mut().execution.collect_statistics = false;
5858
SessionContext::new_with_config(config)
5959
}
6060

61+
#[cfg(test)]
62+
mod session_tests {
63+
#[test]
64+
fn the_shared_session_disables_statistics_collection() {
65+
let ctx = super::session();
66+
assert!(
67+
!ctx.state().config().options().execution.collect_statistics,
68+
"collect_statistics must stay off until apache/datafusion#24769 is fixed in a release we depend on (see session())"
69+
);
70+
}
71+
}
72+
6173
/// Register `urls` as listing table `name` on `ctx` and return its
6274
/// `DataFrame`. **Tenancy** is not a parameter here by design: the
6375
/// §3.7 scope is enforced where URLs are *resolved* (the tenant-

‎crates/ourios-querier/src/lib.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ use datafusion::datasource::listing::{
7373
use datafusion::error::DataFusionError;
7474
use datafusion::functions_aggregate::expr_fn::count;
7575
use datafusion::physical_plan::metrics::{MetricValue, MetricsSet};
76-
use datafusion::prelude::{SessionConfig, SessionContext, col, lit};
76+
use datafusion::prelude::{SessionContext, col, lit};
7777
use ourios_core::tenant::TenantId;
7878
use ourios_parquet::columns;
7979
use ourios_parquet::hour_partition_in_window;

‎docs/rfcs/0021-datafusion-arrow-upgrade.md‎

Lines changed: 43 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -237,20 +237,28 @@ DataFusion 54 carries arrow 58 — reintroducing the dual-arrow split
237237

238238
## 5. Acceptance criteria
239239

240-
Scenario ids `RFC0021.<m>`. Phase 1 = `.1`–`.6`; phase 2 = `.7`–`.9`
241-
(**upstream-gated**: their stubs land red only when phase 2 opens).
242-
243-
> **Status note (2026-07-03):** phase 1 is complete — `.1`–`.6` are
244-
> discharged (#339 dependency bump, #340 decoder unification; indicative
245-
> B1/B2 ci-runner run on #340). `green` here covers phase 1; opening
246-
> phase 2 lands `.7`–`.9` as red stubs and moves the status back to
247-
> `red` until they pass.
248-
249-
> **Scenario RFC0021.1 — one arrow.**
250-
> Given the phase-1 bump,
240+
Scenario ids `RFC0021.<m>`. Phase 1 = `.1`–`.6`; phase 2a = `.8` and
241+
the deny half of `.9`. **This section lists only criteria whose
242+
trigger has fired** — `green` means all of them pass, per
243+
`docs/rfcs/README.md`. Phase 2b's criteria live in §5b below and are
244+
**inactive**: they enter this ledger (as red stubs, moving the RFC
245+
back to `red`) the day a released DataFusion accepts object_store
246+
≥ 0.14, and not before.
247+
248+
> **Status note (2026-08-29):** phase 1 discharged 2026-07-03 (#339,
249+
> #340; indicative B1/B2 ci-runner run on #340). Phase 2a discharged
250+
> 2026-08-29 (`.1` re-baselined per the header, `.8` landed, `.9`'s
251+
> deny half green). The 2026-07-03 note's "opening phase 2 moves the
252+
> status back to `red`" now applies to **phase 2b** — 2a opened and
253+
> closed within one PR, so the red window collapsed to nothing.
254+
255+
> **Scenario RFC0021.1 — one arrow (re-baselined, see header).**
256+
> Given the current dependency set,
251257
> When the workspace lockfile is inspected,
252-
> Then exactly one arrow major (58.x) and `datafusion 54.x` are
253-
> present, and the workspace builds with MSRV 1.88.
258+
> Then exactly one arrow major and exactly one `datafusion` version
259+
> are present, with floors arrow ≥ 59 / DataFusion ≥ 55 (a downgrade
260+
> that would drag `thrift` back fails), and the workspace builds with
261+
> the pinned MSRV (1.94).
254262
255263
> **Scenario RFC0021.2 — old files read identically (§3.5).**
256264
> Given Parquet files written by the pre-upgrade writer (committed
@@ -284,25 +292,36 @@ Scenario ids `RFC0021.<m>`. Phase 1 = `.1`–`.6`; phase 2 = `.7`–`.9`
284292
> Then the complete suite passes — including `s3 integration
285293
> (localstack)` (the CAS paths, untouched) and `live-check (weaver)`.
286294
287-
> **Scenario RFC0021.7 (phase 2b — deferred) — CAS survives
288-
> object_store 0.14.**
289-
> Given the object_store bump,
290-
> When the RFC0013.3/.4 conditional-PUT localstack suites run,
291-
> Then concurrent-sweep publish semantics are preserved.
292-
293295
> **Scenario RFC0021.8 (phase 2a — green) — thrift is gone.**
294296
> Given the parquet 59 bump,
295297
> When the lockfile is inspected,
296298
> Then no `thrift` crate is present (#295 closed).
297299
> Asserted by `rfc0021_8_thrift_is_absent_from_the_lockfile`.
298300
299-
> **Scenario RFC0021.9 (split) — supply chain re-cleared.**
301+
> **Scenario RFC0021.9 (phase 2a half) — supply chain re-cleared.**
300302
> Given the new transitive set,
301303
> When `cargo deny check` runs,
302-
> Then it passes — **green for phase 2a** (it does, and the dead
303-
> RUSTSEC-2024-0436 `paste` ignore was removed) — and, **deferred to
304-
> phase 2b**, with the renovate hold lifted (#313) and the quick-xml
305-
> ignores removed iff object_store pins quick-xml ≥ 0.41.
304+
> Then it passes (it does, and the dead RUSTSEC-2024-0436 `paste`
305+
> ignore was removed). The quick-xml half of this criterion is phase
306+
> 2b's — see §5b.
307+
308+
### 5b. Phase 2b criteria (inactive — trigger not fired)
309+
310+
These activate, as red stubs in §5, when a released DataFusion accepts
311+
object_store ≥ 0.14 (epic #314). Until then they are forward
312+
declarations, not acceptance criteria, and do not count against the
313+
status.
314+
315+
> **Scenario RFC0021.7 — CAS survives object_store 0.14.**
316+
> Given the object_store bump,
317+
> When the RFC0013.3/.4 conditional-PUT localstack suites run,
318+
> Then concurrent-sweep publish semantics are preserved.
319+
320+
> **Scenario RFC0021.9 (phase 2b half) — quick-xml ignores retired.**
321+
> Given the object_store ≥ 0.14 transitive set,
322+
> When `cargo deny check` runs with the renovate hold lifted (#313),
323+
> Then it passes with the quick-xml ignores removed iff object_store
324+
> pins quick-xml ≥ 0.41.
306325
307326
## 6. Testing strategy
308327

‎docs/rfcs/0046-out-of-band-tenancy.md‎

Lines changed: 22 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -326,16 +326,28 @@ validation gets its own rejection cases. RFC0046.8 in the CI-only collector inte
326326

327327
## 7. Open questions
328328

329-
- [ ] **Selector length bound (256 B)** — generous for opaque ids; is a
330-
shorter bound wanted so headers stay log-friendly?
331-
- [ ] **Non-ASCII tenant ids over gRPC** — accept the stated caveat, or
332-
define a `-bin` metadata carrier so gRPC parity is total?
333-
- [ ] **Deprecation window for the RFC 0045 registry entries** — deprecate
334-
now and delete at the next minor, or keep indefinitely per semconv
335-
practice? (Leaning: keep deprecated; they were never released.)
336-
- [ ] **RFC 0003 §6.3 text** — amend in place with a "superseded by RFC 0046"
337-
note, or leave and rely on this RFC's `supersedes:` line? (Leaning:
338-
amend in place; readers start from 0003.)
329+
All four resolved (recorded 2026-08-28; three by RFC 0048, one by the
330+
semconv extraction — this RFC was flipped `accepted` while they still
331+
read open, which the resolve-never-waive rule of the ladder sweep does
332+
not allow):
333+
334+
- [x] **Selector length bound** — **RFC 0048 §3.1 pinned it at 1–128
335+
bytes**, tighter than the 256 B floated here, and made it the one
336+
tenant grammar every boundary applies at extraction
337+
(`MAX_SELECTOR_BYTES = MAX_TENANT_BYTES` in the receiver).
338+
- [x] **Non-ASCII tenant ids over gRPC** — **caveat accepted, then
339+
dissolved**: RFC 0048 §3.1's grammar admits only ASCII graphic
340+
characters (minus `:`, `#`, `/`) at *every* boundary, so HTTP and
341+
gRPC now reject the same inputs. No `-bin` carrier is needed;
342+
parity is total because the grammar, not the transport, is the
343+
bound.
344+
- [x] **Deprecation window for the RFC 0045 registry entries** — kept
345+
deprecated, as leaned. `ourios.tenant.watch.key` /
346+
`.first_value` carry `deprecated: {reason: obsoleted}` in the
347+
shared `ourios-semconv` registry, noting they were never emitted
348+
by a released version.
349+
- [x] **RFC 0003 §6.3 text** — amended in place, as leaned: §6.3 opens
350+
with a "Superseded by RFC 0046 (2026-08-17)" banner.
339351

340352
## 8. References
341353

0 commit comments

Comments
 (0)