Skip to content

Commit 2eb2e62

Browse files
author
Stefan Jansen
committed
docs: identify released migration inputs and consumer boundaries
Correct stale unreleased wording for Data 0.3.0, link release examples, and distinguish historical integration checks from released Data availability. Keep Diagnostic analytics and Backtest runtime/account responsibilities explicit. Closes #137
1 parent d4f33bc commit 2eb2e62

1 file changed

Lines changed: 47 additions & 22 deletions

File tree

‎docs/getting-started/migration.md‎

Lines changed: 47 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,29 @@
11
# Migrate existing data workflows
22

3+
The inputs and adapters on this page are available in
4+
[Data 0.3.0](https://pypi.org/project/ml4t-data/0.3.0/) on Python 3.12 through 3.14.
5+
Install the released package with `uv add ml4t-data`. Native preparation needs no Reloaded runtime.
6+
The optional Zipline bridge runs once in a separate environment with the original bundle runtime.
7+
8+
Run the maintained examples from the matching
9+
[release checkout](https://github.com/ml4t/data/tree/db9cda776b5b346f536b0e8ceb855b9fd60b6ec8/examples).
10+
Clean published-wheel installation checks passed all six native examples; the optional conversion
11+
passed 13 checks using the real Zipline readers and writers. The pinned legacy environment and
12+
historical cross-library checks are described in the relevant sections below. They do not qualify
13+
every version of a consuming library.
14+
15+
| Input needed by a consumer | Released Data API | Boundary |
16+
|---|---|---|
17+
| A pandas price panel or native long price table for Diagnostic | `to_price_panel`, `from_price_panel` | Diagnostic computes returns, factor statistics and analytical fill views. Data retains missing quotes and stable IDs. |
18+
| Captured action facts and dated adjusted research prices | `prepare_action_inputs`, `prepare_adjusted_history`, `InstrumentRegistry` | Backtest owns runtime history, orders, entitlement, payments and positions. Adjusted research prices do not establish broker accounting. |
19+
| Individual futures definitions, eligible fields and paired roll marks | `prepare_futures_inputs` | Backtest owns roll execution, settlement and margin. Data does not infer missing execution marks. |
20+
| Portable captured tables and declarations | `export_snapshot`, `read_snapshot` | Each consumer supplies its own effective configuration and runtime policy. |
21+
| Legacy acquisition imports or a saved Zipline bundle | `ml4t.data.compat.empyrical`, `ml4t.data.compat.zipline` | These adapters preserve the bounded data workflow, not the legacy analytics or execution engine. |
22+
23+
The examples qualify Data's producer contracts. For a combined environment, also use the
24+
consumer's published migration guide and qualification evidence. The historical source checks
25+
below remain limited to their named revisions; they are not current installation requirements.
26+
327
## Move a long price table to a pandas analytical panel
428

529
Keep source observations in Data storage. Use `to_price_panel` when an analytical consumer needs
@@ -56,7 +80,7 @@ the declared label clock; instant bounds use UTC. Read ranges are start-inclusiv
5680
Calendar omission, explicit `None` and a named `24/7` schedule remain distinct. Panel conversion
5781
does not invoke a calendar resolver or create an expected clock.
5882

59-
These adapters are unreleased source functionality. Isolated wheel and source-distribution checks
83+
Before the 0.3.0 release, isolated wheel and source-distribution integration checks
6084
qualified Data `0.2.2.dev14+gbe73c5d86`, Specs `0.2.0`, Diagnostic `0.1.9` and Engineer
6185
`0.1.7.dev29+gc69e7bada` from immutable source
6286
[`c69e7bada`](https://github.com/ml4t/engineer/commit/c69e7badaac012ad98b1c4b8aa9a67a952b08f5c).
@@ -65,11 +89,10 @@ The checks used pandas `3.0.6` and Polars `2.0.0`. Diagnostic's public
6589
B's missing quote. Engineer's public `labeling.fixed_time_horizon_labels` consumed A's complete
6690
price history. Both produced the independently expected one-observation outcomes.
6791

68-
Engineer `0.1.7` is not yet a published release; published `0.1.6` requires Specs `<0.2` and
69-
cannot be installed alongside current Data. Use the qualified immutable Engineer source for this
70-
integration until a compatible Engineer release is available. This is source integration evidence,
71-
not a claim of released-package migration compatibility. Neither consumer nor any Reloaded package
72-
is required for the Data-only example.
92+
Engineer `0.1.6` required Specs `<0.2`; that historical combination is incompatible with Data
93+
0.3.0. Engineer has since published `0.2.0` with Specs `>=0.2.0,<0.3.0`. The source checks above
94+
do not certify that newer consumer's behavior. Neither consumer nor any Reloaded package is
95+
required for the Data-only example.
7396

7497
## Import corporate-action facts without changing their financial definition
7598

@@ -117,13 +140,14 @@ Factor keys must exactly cover the raw observations. Volume is adjusted only whe
117140
is supplied. Raw prices, volume and factor provenance remain available; known conflicting currency
118141
units, reserved output fields and incomplete or duplicate keys fail without input mutation.
119142

120-
These are unreleased source APIs. The Data-only recipe needs no Reloaded, Engineer or Backtest
121-
dependency. Prepared adjusted research prices are not execution marks with an additional copy of
143+
These APIs are included in Data 0.3.0. The Data-only recipe needs no Reloaded, Engineer or
144+
Backtest dependency. Prepared adjusted research prices are not execution marks with an additional copy of
122145
the same dividend cash event. The consuming broker owns entitlement, payment, orders and positions.
123146
Source availability remains separate from effective time; this preparation step makes no causal
124147
as-of claim.
125148

126-
Isolated wheel and source-distribution checks qualified Data `0.2.2.dev23+g0917c8401`,
149+
Pre-release wheel and source-distribution integration checks qualified Data
150+
`0.2.2.dev23+g0917c8401`,
127151
Specs `0.2.0`, Diagnostic `0.1.9` and the immutable Engineer source candidate identified above.
128152
The installed Data recipe produced two versioned event facts. Both consumers' public return/label
129153
APIs produced `0.01` for native adjusted prices and `1/97` for the supplied-factor prices, while
@@ -219,8 +243,8 @@ The recipe retains July1/2 warmup prices `198,200` at the July2 cutoff. A later
219243
does not change that input. The July3 split/cash facts produce the independent native shareholder
220244
return `1.01`; a July7 symbol rename retains ID17. July4 is an exchange closure,not a missing
221245
observation. The example needs no Reloaded,Engineer,Diagnostic or Backtest runtime dependency.
222-
These are unreleased source APIs. Isolated wheel and source-distribution checks qualified Data
223-
`0.2.2.dev35+g2c4ff2050`, Specs `0.2.0`, Diagnostic `0.1.9` and the immutable Engineer source
246+
These APIs are included in Data 0.3.0. Pre-release wheel and source-distribution integration
247+
checks qualified Data `0.2.2.dev35+g2c4ff2050`, Specs `0.2.0`, Diagnostic `0.1.9` and the immutable Engineer source
224248
candidate identified above, with pandas `3.0.6` and Polars `2.0.0`. The installed Engineer public
225249
`features.trend.sma` consumed Data's two pre-entry observations and produced `null,199`; the
226250
price-sensitive decision `200 > 199` remained true after the future correction. Engineer's public
@@ -379,8 +403,8 @@ FX conversion. Read `metadata['normalized_feed_spec']` to consume those fields a
379403
The same calculation after cents-to-dollars conversion uses price change0.1 and multiplier5000.
380404
Neither notional value nor these marks establish cash settlement, margin or fills.
381405

382-
These are unreleased source APIs. Isolated wheel and sdist checks qualified Data
383-
`0.2.2.dev57+ge83e6b086`, Specs `0.2.0`, pandas `3.0.6`, Polars `2.0.0` and immutable Backtest
406+
These APIs are included in Data 0.3.0. Pre-release wheel and source-distribution integration
407+
checks qualified Data `0.2.2.dev57+ge83e6b086`, Specs `0.2.0`, pandas `3.0.6`, Polars `2.0.0` and immutable Backtest
384408
[`12a2c483f`](https://github.com/ml4t/backtest/commit/12a2c483f8f1fda54244af187db8bf44e6f85fc9)
385409
(`0.1.13.dev22+g12a2c483f`) on Python `3.12.11`. Actual Backtest `ContractSpec` and `Position`
386410
consumed the produced raw and converted fields, retaining IDs, paired marks and units; both
@@ -389,7 +413,7 @@ roll fills, settlement, margin or a complete released-package migration.
389413
Data does not replace Backtest's runtime history, account, roll execution or auto-close policy.
390414
The recipe needs no credentials, network, Reloaded package or Backtest installation.
391415

392-
This source candidate requires Polars `>=1.36.1` and pandas-market-calendars `>=5.4.0`.
416+
Data 0.3.0 requires Polars `>=1.36.1` and pandas-market-calendars `>=5.4.0`.
393417
Isolated runs using the former advertised minima failed during ordinary Data import or calendar
394418
construction. The saved-file recipe passes with Polars `1.36.1`, pandas-market-calendars `5.4.0`
395419
and pandas `2.3.3`; those checks do not certify every possible dependency-version combination.
@@ -457,12 +481,13 @@ from ml4t.data.compat.empyrical import default_returns_func
457481
register_return_func(default_returns_func)
458482
```
459483

460-
The offline [acquisition migration example](https://github.com/ml4t/data/blob/3dd71cb394c390316ff0bb36ad1d7d0f3dd6aefc/examples/acquisition_migration.py)
484+
The offline [acquisition migration example](https://github.com/ml4t/data/blob/db9cda776b5b346f536b0e8ceb855b9fd60b6ec8/examples/acquisition_migration.py)
461485
replays retained public response captures through actual parsers, providers and CSV files. It runs
462-
without credentials or network and does not require any Reloaded package. This is an unreleased
463-
source workflow, not a claim that a released successor stack already replaces every legacy option.
486+
without credentials or network and does not require any Reloaded package. This workflow is
487+
included in Data 0.3.0; it does not establish replacement of every legacy option.
464488

465-
Isolated wheel and source-distribution checks qualified Data `0.2.2.dev27+g5ff44e4ac`,
489+
Pre-release wheel and source-distribution integration checks qualified Data
490+
`0.2.2.dev27+g5ff44e4ac`,
466491
Specs `0.2.0`, Diagnostic `0.1.9`, Pyfolio Reloaded `0.10.0` at immutable commit
467492
`365e649f591c39cef5d0170958280e49b8eeb953`, and Alphalens Reloaded `0.5.0` at
468493
`1bc181de4aafbe33437ed0e362ff96aa7ce20acd`. The checks exercised actual Diagnostic
@@ -476,7 +501,7 @@ the Yahoo extra now requires the qualified `>=1.5.2` range.
476501
## Reconstruct inputs after source pruning
477502

478503
Retain the input tables and declarations instead of referring to mutable source paths. Run the
479-
offline [portable snapshot example](https://github.com/ml4t/data/blob/a98cede0a36ba6856a010456f302e3e987d255de/examples/portable_snapshot_migration.py)
504+
offline [portable snapshot example](https://github.com/ml4t/data/blob/db9cda776b5b346f536b0e8ceb855b9fd60b6ec8/examples/portable_snapshot_migration.py)
480505
from a source checkout:
481506

482507
```bash
@@ -552,10 +577,10 @@ The actual optional conversion checks use immutable Zipline Reloaded `3.2.0` sou
552577
Python `3.12`, pandas `2.3.3`, Specs `0.2.0` and Polars `2.0.0`. They construct a real bundle
553578
with renamed equities, two venues, two futures, missing quotes, split/cash/stock actions and daily
554579
and minute observations, then compare native preparation and standard re-ingestion. The native
555-
recipe requires no legacy runtime. This is unreleased source functionality; it does not claim that
556-
published Data `0.2.1` provides these APIs or that Data replaces Zipline's runtime or broker.
580+
recipe requires no legacy runtime. These APIs are included in Data 0.3.0. Data does not replace
581+
Zipline's runtime or broker.
557582

558-
Clean wheel and source-distribution installs qualified Data `0.2.2.dev27+g90c06ba9c` for native
583+
Pre-release wheel and source-distribution installs qualified Data `0.2.2.dev27+g90c06ba9c` for native
559584
reconstruction, the optional conversion and the selected dependency floors. Reconstructed tables
560585
also passed actual Engineer labeling, Diagnostic forward returns and Backtest contract/position
561586
checks using Engineer `0.1.7.dev29+gc69e7bada`, Diagnostic `0.1.9` and Backtest

0 commit comments

Comments
 (0)