feat(cayenne): runtime-wide adaptive_tuning and target_* settings - #14844

📝 Summary

• Moves Cayenne's tuning settings out of per-dataset and per-catalog parameters into one runtime-wide switch and runtime-wide targets, without the cayenne_ prefix:

  New setting	Replaces
  runtime.adaptive_tuning: enabled | disabled (default disabled)	dataset and catalog param cayenne_tuning (adaptive → enabled, auto → disabled)
  runtime.params.target_replication_lag, target_freshness, target_query_latency	cayenne_goal_* (global and per-dataset)
  runtime.params.target_convergence_window	per-dataset cayenne_goal_convergence_window
  runtime.params.target_qph	runtime.params.cayenne_goal_qph

  runtime:
    adaptive_tuning: enabled
    params:
      target_freshness: 30s

• runtime.adaptive_tuning is a typed field. Any other value, including true, fails the Spicepod load: Invalid `runtime.adaptive_tuning` value '{value}': expected `enabled` or `disabled`. See: https://spiceai.org/docs/reference/spicepod/runtime.

• Applies to every Cayenne-accelerated dataset and to Cayenne catalogs, which now also receive the target_* values. Targets still do nothing unless adaptive tuning is enabled, and every existing adaptive precondition and fallback is unchanged.

• The old names are no longer read. Each one still in use gets a single startup warning that names its replacement, in either spelling (with or without cayenne_), on datasets, catalogs and runtime.params, and it is not applied. For example: Dataset 'orders' sets `cayenne_tuning`, which is no longer a dataset parameter, so it has no effect. Set `runtime.adaptive_tuning` instead. See: https://spiceai.org/docs/reference/spicepod/runtime. These keys no longer also produce the generic "Ignoring parameter" warning.

• Clearer warnings for ignored targets:

  • Targets set while adaptive tuning is disabled get one warning per dataset or catalog.
  • The compaction fallback (adaptive tuning enabled but cayenne_compaction_background_interval_ms: 0) gets a single warning that also says the targets are ignored.

• A runtime change to runtime.adaptive_tuning is handled like a runtime.params change, so it applies to recreated components.

• Adds ParameterSpec::moved_to so a retired parameter is dropped with its own warning and left out of the generated schema.

• Updates docs/cayenne/cayenne.md, the Cayenne README, and the adaptive benchmark and scenario Spicepods. Cayenne's cayenne_goal_* metrics keep their names.

🔗 Related

• Closes Enhancement: Runtime-wide adaptive tuning (runtime.adaptive_tuning) #14845.

🚨 Breaking Changes

• cayenne_tuning and cayenne_goal_* (dataset, catalog or runtime.params) are no longer applied. Each warns at startup with its replacement. A Spicepod that relied on them runs with static tuning until it is updated.
• Per-dataset target overrides are gone; targets are runtime-wide.
• An invalid runtime.adaptive_tuning fails the load.

📚 Docs

• The Spicepod runtime reference needs adaptive_tuning and params.target_*. The Cayenne accelerator page needs the dataset parameters removed.

👀 Notes for Reviewers

• Before, on trunk's spiced: a runtime-level switch is ignored (runtime.params.tuning is not a recognized runtime parameter; ignoring, then Cayenne auto-tuned config: … tuning="auto"), and only the dataset's cayenne_tuning: adaptive turns adaptive on.

• crates/runtime/tests/cayenne_runtime_tuning.rs loads real Spicepods through AppBuilder::build_from_path and reads each dataset's resolved tuning and targets. It covers:

  • runtime.adaptive_tuning: enabled turns adaptive on, and the default is static;
  • runtime.params.adaptive_tuning does not turn it on;
  • an invalid value fails the load;
  • retired dataset keys in both spellings are not applied and warn once, with no generic warning;
  • a renamed runtime key warns once;
  • the compaction fallback produces a single warning.

  It runs in the make nextest gate. With the accelerator pointed back at the old keys, the enable cases fail.

• crates/runtime/tests/cayenne_catalog_tuning.rs starts a scheduler and an executor and registers a real Cayenne catalog. It checks the enabled and default modes, that the targets reach catalog tables, the ignored-targets warning, and the retired catalog keys. The Cayenne catalog connector exists only with runtime/spicebench, so it runs by hand: cargo test -p runtime --features spicebench --test cayenne_catalog_tuning -- --test-threads=1.

• DataAccelerator::adaptive_tuning_seeds now takes the typed mode and the runtime params. Only Cayenne overrides it; every other engine keeps the default, which has no adaptive controller.

• Not run: the new warnings from a built spiced.

Review gate

Covers only the two trunk merges 27af2f6 and e804b4e pushed by fix-failing-prs (each a Makefile NEXTEST_FILTER conflict, resolved by keeping both sides), not the author's original work.

• Adversarial review: codex, verdict needs-attention, scoped to origin/trunk...27af2f6b5f. No findings on the merge resolution. One unverified finding (code inspection only) on the existing catalog hot-reload path is passed to the author in a PR comment and is not fixed here.
• Security review: the resolution diff adds one nextest filter clause and three comment lines in Makefile. It adds no runtime, input-handling or secret-bearing surface.
• Simplify: n/a (no code written beyond the conflict hunk).