Repository navigation
Comparing changes
Open a pull request
base repository: ml4t/data
base: v0.2.0
head repository: ml4t/data
compare: v0.2.1
- 8 commits
- 113 files changed
- 5 contributors
Commits on Sep 25, 2026
-
docs: publish asset-class data-source references (#71)
* docs: split market sources by asset class Closes #68. * docs: add economic and specialized source references Closes #69. * test: enforce source-reference contracts Closes #70. * ci: refresh ecosystem qualification snapshot --------- Co-authored-by: Stefan Jansen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 885bc11 - Browse repository at this point
Copy the full SHA 885bc11View commit details
Commits on Oct 8, 2026
-
feat: prediction-market resolution outcomes, price and trade history …
…(Kalshi, ForecastEx, Polymarket, Polymarket US) (#76) * feat(kalshi): enumerate resolved markets and trades across live and archive tiers KalshiProvider gains iter_markets/fetch_markets and iter_trades/fetch_trades. Both page through the live endpoints (/markets, /markets/trades) and the archive (/historical/markets, /historical/trades), using /historical/cutoff to skip a tier that cannot match and dropping duplicates. Filters the archive does not accept (status, close-time window, multivariate exclusion combined with a ticker filter) are applied on the client. fetch_markets returns a typed frame with a normalized resolution outcome (result in yes/no/void/other or null, settlement_value, settlement_ts), and fetch_trades returns trade_id, ticker, timestamp, price, count, taker_side, is_block_trade. Both the *_dollars/*_fp fields and legacy unsuffixed fields are read. list_markets and the other existing methods are unchanged. BaseProvider gets a shared rate-limited _request/_request_json helper that maps 429/404/5xx/transport failures to the shared exceptions and retries transient ones under the existing provider retry policy, now exposed as provider_retry. prediction_markets.py holds the outcome vocabulary and trade schema shared by the prediction-market adapters. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(providers): add ForecastEx and Polymarket US event-contract providers ForecastExProvider reads the free daily CSVs in the public forecastex-public-data bucket: fetch_prices (daily OHLC, marks, settlement price), fetch_trades (every matched YES/NO pair) and fetch_products. Its fetch_markets derives the resolution outcome: a contract absent from the next day's prices file has expired, and a final YES settlement price of 1.00/0.00 is the payout. Intraday contracts that drop out with a daily mark instead of the payout are reported as expired with a null result rather than guessed. PolymarketUSProvider reads the public gateway of the CFTC-regulated Polymarket US exchange (separate from the global PolymarketProvider, which is unchanged): offset-paginated markets with a normalized outcome taken from the long market side, /settlement, and price history in fixed or 24-hour windows. The gateway's outcomes labels are not always in outcomePrices order, so labels are never zipped with prices. Both are registered as "events" providers outside DataManager routing. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(providers): document prediction-market history and outcome loaders Add provider pages for ForecastEx and Polymarket US, extend the Kalshi page with resolved-market enumeration, the normalized outcome columns and trade history, and list both new venues in the provider index, the prediction-market source reference, provider selection and the navigation. Rename the new offline test modules so their basenames do not collide with the integration modules of the same venue. Co-Authored-By: Claude Opus 5.5 <[email protected]> * build(deps): lock patched multidict, urllib3 and virtualenv The dependency audit now flags multidict 6.7.1 (CVE-2026-104874), urllib3 2.7.0 (PYSEC-2026-4175..4177) and virtualenv 21.7.1 (PYSEC-2026-4011..4014). Upgrade the three transitive pins to 6.9.1, 2.8.0 and 21.14.6; python-discovery moves with virtualenv. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(kalshi): expose mve_filter and keep series, previous price and taker sides Replace the exclude_multivariate flag with Kalshi's mve_filter parameter (None, "exclude", "only") on iter_markets/fetch_markets, so combo (KXMVE*) markets can be included, excluded or selected. /markets receives the value directly; /historical/markets rejects "only" and takes "exclude" only without a ticker filter, so the selection is completed on the client. The normalized market frame gains series (series filter or event-ticker prefix) and previous_price; the trade frame keeps taker_outcome_side and taker_book_side next to taker_side. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(kalshi): tier-aware candlesticks, streaming market frames, live contracts Price history now follows the market's tier. fetch_candles (and fetch_ohlcv, which shares the routing) reads archived markets from /historical/markets/{ticker}/candlesticks and live ones from /series/{series}/markets/{ticker}/candlesticks. An explicit tier or series_ticker skips detection; otherwise a live 404 falls back to the archive, and a second 404 looks the series up from the market's event, which fixes old tickers whose prefix is not their series (FED-23DEC is in KXFED). Ranges beyond Kalshi's 5000-candle limit are split. Prices are converted per field by encoding: *_dollars values and unsuffixed strings are dollars, unsuffixed integers are legacy cents. The previous blanket /100 on unsuffixed fields would have scaled archive candles down 100x. fetch_candles returns trade OHLC, mean and previous, YES bid/ask close, volume, open interest and the serving tier. iter_market_frames streams fetch_markets' normalized rows in frames of chunk_size rows for listings too large to materialize; fetch_markets now delegates to it. The provider-contracts public job runs the new Kalshi, ForecastEx and Polymarket US live smoke tests, and the workflow contract test checks it. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(polymarket): bulk resolved-market listing, windowed price history and trades iter_markets / iter_market_frames / fetch_markets page through Gamma's GET /markets/keyset (GET /markets refuses offsets beyond 2000), passing next_cursor back as after_cursor; any other parameter name silently returns page 1, so a repeated cursor raises instead of looping. The server honours closed, volume_num_min, end_date_min and end_date_max (verified 2026-10-08) and caps pages at 100; the filters are re-applied on the client. include_tag=true brings the market tags, the reliable topic field now that category is mostly null. Rows are normalized to the shared resolution vocabulary. Gamma markets are binary and ordered: the first outcome is the YES side whatever its label, final prices [1,0]/[0,1] are yes/no, [0.5,0.5] is void, a UMA "resolved" market with other prices is "other", the rest is null. settlement_ts is closedTime (else umaEndDate), and the frame carries token ids, event slug/title, neg-risk and the raw fee fields. fetch_candles always sends startTs/endTs with fidelity, split into windows of at most 15 days: interval=max returns nothing for resolved markets below daily fidelity, and longer spans are rejected with 400. fetch_trades pages data-api /trades newest first. Past the offset cap of 10000 it moves the inclusive end bound to the oldest second seen and drops the boundary trades already yielded; a repeated page or a second too dense to page past raises. Prices are converted to the YES price and the taker side to yes/no from outcome index and BUY/SELL. Offline tests use responses recorded on 2026-10-08; each was checked to fail with its defect planted. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(polymarket): fetch_ohlcv returned nothing for past date ranges The CLOB treats interval as a range ending now that overrides startTs and endTs, so fetch_ohlcv's interval=1h chunks came back empty for any past window (reproduced live on 2026-10-08 for a market resolved in 2025). Chunks now send their bounds with the interval's step as fidelity. YYYY-MM-DD bounds were also converted in the machine's local zone; they are UTC days now, and "max" no longer builds a single multi-year window the CLOB rejects. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(polymarket): document resolved history and run its live contracts The Polymarket provider page covers fetch_markets / iter_market_frames (keyset paging, normalized columns, first-outcome-is-YES mapping), fetch_candles (15-day windows, the empty interval=max history of resolved markets) and fetch_trades (offset cap and end-bound paging), and replaces the outdated 7-day range advice. The prediction-market reference lists Polymarket among the venues with normalized outcomes. TestPolymarketResolvedHistory smoke-tests the three calls against a market resolved on 2026-01-01; the provider-contracts public job runs it and the workflow contract test checks that it does. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(polymarket): skip the local-zone test where time.tzset is missing Windows has no time.tzset, so the ecosystem qualification job errored in the fixture that switches the local zone to America/New_York. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Stefan Jansen <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1855154 - Browse repository at this point
Copy the full SHA 1855154View commit details
Commits on Oct 14, 2026
-
ci: bump the github-actions group across 1 directory with 3 updates (#81
) Bumps the github-actions group with 3 updates in the / directory: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv), [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact). Updates `astral-sh/setup-uv` from 10.1.0 to 10.3.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@bec219d...1c37ad0) Updates `actions/upload-artifact` from 7.0.1 to 7.0.2 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@043fb46...cf430e0) Updates `actions/download-artifact` from 8.0.1 to 8.0.2 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@3e5f45b...9000827) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 10.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions - dependency-name: actions/upload-artifact dependency-version: 7.0.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions - dependency-name: actions/download-artifact dependency-version: 8.0.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for dfcce14 - Browse repository at this point
Copy the full SHA dfcce14View commit details
Commits on Oct 15, 2026
-
docs: correct stale guidance and enforce link integrity (#78)
* docs: enforce published link integrity * docs: replace stale contributor guidance * fix: retry deployed documentation probes * docs: remove stale legacy guidance * fix: tolerate transient external link failures * docs: describe supported Massive financial statements * docs: make session validation examples executable --------- Co-authored-by: Stefan Jansen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 8055e70 - Browse repository at this point
Copy the full SHA 8055e70View commit details -
ci: enforce minimum package coverage (#80)
* docs: enforce published link integrity * docs: replace stale contributor guidance * fix: retry deployed documentation probes * docs: remove stale legacy guidance * fix: tolerate transient external link failures * ci: enforce minimum package coverage * docs: align agent test command with coverage gate * docs: document enforced coverage lane * ci: retain current setup-uv pin in coverage job --------- Co-authored-by: Stefan Jansen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for bc24706 - Browse repository at this point
Copy the full SHA bc24706View commit details -
docs: adopt shared focused-contribution policy (#83)
* docs: adopt shared contribution policy * docs: align contribution checks with coverage gate --------- Co-authored-by: Stefan Jansen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 72b5e9b - Browse repository at this point
Copy the full SHA 72b5e9bView commit details -
fix: restore inclusive date ranges and complete Massive histories (#85)
Include the real month end in Fama-French filters, include the stored end-date bar in gap detection, request the inclusive macro end date from Yahoo, interpret Finnhub date bounds in UTC, and follow Massive aggregate pagination. Closes #73 Preserves the five applicable fixes contributed by quantum-like in #74. The Kalshi and Polymarket UTC fixes from that contribution are already supplied by #76. The regressions failed against the original base and passed with the fixes. The replacement retains the original contribution commits for review and passed the supported CI checks in #85. Co-authored-by: quantum-like <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for f2a2252 - Browse repository at this point
Copy the full SHA f2a2252View commit details -
docs: add verified roic.ai fundamentals source (#86)
Addresses #84 with the documented free annual-history restriction and point-in-time caveat. Co-authored-by: Stefan Jansen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 6888629 - Browse repository at this point
Copy the full SHA 6888629View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v0.2.0...v0.2.1