@@ -472,3 +472,95 @@ Data-produced inputs. They used Python `3.12`, pandas `2.3.3`, Polars `2.0.0` an
472472pandas-market-calendars ` 5.4.0 ` and yfinance ` 1.5.2 ` . Older yfinance versions admitted by the
473473former ` >=0.2.0 ` declaration lack the exception API used by Data's empty-download recovery;
474474the Yahoo extra now requires the qualified ` >=1.5.2 ` range.
475+
476+ ## Reconstruct inputs after source pruning
477+
478+ Retain the input tables and declarations instead of referring to mutable source paths. Run the
479+ offline [ portable snapshot example] ( https://github.com/ml4t/data/blob/a98cede0a36ba6856a010456f302e3e987d255de/examples/portable_snapshot_migration.py )
480+ from a source checkout:
481+
482+ ``` bash
483+ uv run python examples/portable_snapshot_migration.py
484+ ```
485+
486+ The recipe reuses the dated-history and individual-futures examples. It captures six tables,
487+ their separate source cutoffs and revisions, stable identity namespaces, units, raw price/share
488+ basis, feed declarations and effective selection configuration. It then prunes source generations,
489+ changes the original futures file, relocates the archive, and reconstructs the same prepared inputs.
490+ The checks compare equity keys, ticker changes, action facts, adjusted prices/volume, selected
491+ contracts and paired roll marks. Missing quotes remain missing. No Reloaded package, credentials
492+ or network is needed. Read the [ snapshot storage contract] ( ../user-guide/storage.md#portable-input-snapshots )
493+ for required metadata and integrity rules.
494+
495+ Effective configuration must be supplied by the caller. A Data snapshot does not establish a
496+ common historical publication time for independently sourced tables, discover omitted execution
497+ policies, or certify an account result. Backtest owns runtime history, execution and account
498+ reconstruction; Engineer owns feature computation and Diagnostic owns analytical fill views.
499+
500+ ### Convert an existing Zipline bundle once
501+
502+ In a separate conversion environment with Zipline Reloaded installed, load the original bundle
503+ through its standard API and export selected stable source IDs and inclusive reader bounds:
504+
505+ ``` python
506+ from zipline.data import bundles
507+ from ml4t.data.compat.zipline import export_bundle
508+
509+ with bundles.load(" my-existing-bundle" ) as bundle:
510+ snapshot = export_bundle(
511+ bundle,
512+ " captured-bundle" ,
513+ namespace = " original-bundle-permanent-ids" ,
514+ source_revision = " original-ingestion-id" ,
515+ sids = [1 , 2 ],
516+ start = " 2024-01-02" ,
517+ end = " 2024-01-04" ,
518+ frequencies = (" daily" ,),
519+ )
520+ ```
521+
522+ Keep the namespace stable across captures; exported identities combine it with the original sid.
523+ The bridge reads standard asset, bar and adjustment readers without modifying the source bundle.
524+ Native tables include raw observations, source history-adjustment factors, dated asset definitions,
525+ symbol lookup intervals, and corporate-action facts. A bounded stock-dividend selection also retains
526+ the payment asset's identity metadata, without creating price observations for it.
527+
528+ Daily bounds are civil session labels. Minute bounds are UTC instants; use ` frequency_bounds `
529+ when selecting daily and minute data together. Minute timestamps retain the source's right-labeled
530+ bar-close convention. Source reader zeros in volume remain zeros without an inferred no-trade
531+ status. Source schedule/provider declarations remain separate from Data's selected calendar;
532+ ` calendar=None ` disables the native schedule without changing the captured source grid.
533+
534+ Zipline split price factors are converted to native new-shares-per-old-share terms. Source
535+ history factors remain a declared, supplied adjustment method; they are not recomputed as a
536+ different vendor's total-return definition. Publication times, quote currency, merger consideration
537+ and original listing endpoints that public readers cannot establish remain explicitly unavailable.
538+ Symbol ownership lookup intervals must not be interpreted as original listing intervals.
539+ Capturing unknown publication times does not make a bundle suitable for causal known-at claims.
540+
541+ For staged migration, ` make_bundle_ingest("captured-bundle") ` returns a standard Zipline bundle
542+ ingestion callback. Register it with the same source calendar and a range covering the captured
543+ bars, then use normal ` bundles.ingest ` and ` bundles.load ` . The callback verifies all files and
544+ prepares all tables before invoking writers. Asset metadata, source action-table indexes and raw
545+ daily/minute values round-trip through standard writers. A target minute writer must have enough
546+ price precision for the source; incompatible scales fail before writes instead of rounding prices.
547+ Configure ` BcolzMinuteBarWriter ` 's per-sid scale when needed. Native snapshots impose no legacy
548+ integer price scale. Reverse ingestion is optional and is not required for native reconstruction.
549+
550+ The actual optional conversion checks use immutable Zipline Reloaded ` 3.2.0 ` source
551+ [ ` cf649ac06 ` ] ( https://github.com/stefan-jansen/zipline-reloaded/commit/cf649ac06fddce76f6caa2403cd8494ac84f72ad ) ,
552+ Python ` 3.12 ` , pandas ` 2.3.3 ` , Specs ` 0.2.0 ` and Polars ` 2.0.0 ` . They construct a real bundle
553+ with renamed equities, two venues, two futures, missing quotes, split/cash/stock actions and daily
554+ and minute observations, then compare native preparation and standard re-ingestion. The native
555+ recipe requires no legacy runtime. This is unreleased source functionality; it does not claim that
556+ published Data ` 0.2.1 ` provides these APIs or that Data replaces Zipline's runtime or broker.
557+
558+ Clean wheel and source-distribution installs qualified Data ` 0.2.2.dev27+g90c06ba9c ` for native
559+ reconstruction, the optional conversion and the selected dependency floors. Reconstructed tables
560+ also passed actual Engineer labeling, Diagnostic forward returns and Backtest contract/position
561+ checks using Engineer ` 0.1.7.dev29+gc69e7bada ` , Diagnostic ` 0.1.9 ` and Backtest
562+ ` 0.1.13.dev22+g12a2c483f ` . That Backtest check verifies a USD500 marked position result, not
563+ cash settlement or corporate-action account execution. The source recipes and input contracts
564+ remain usable without any of those consumers. CI's separate ` Installed bundle migration ` job
565+ keeps the optional conversion qualified against the pinned Zipline source without adding it to
566+ Data's production dependencies.
0 commit comments