Skip to content

fix: normalize exchange date before filtering equity sessions #88

Closed

Description

@stefan-jansen

The same NYSE trading instant is classified differently depending on the caller's timestamp timezone. In published ml4t-engineer 0.1.6 with pandas-market-calendars 5.5.0, EquityCalendar.filter_sessions retains the open row in UTC and America/New_York but incorrectly drops it in Asia/Tokyo and Australia/Sydney.

Reproduction through the public filtering workflow:

from datetime import UTC, datetime
import polars as pl
from ml4t.engineer.core.calendars import EquityCalendar

calendar = EquityCalendar("NYSE")
frame = pl.DataFrame({
    "timestamp": [datetime(2024, 7, 2, 19, tzinfo=UTC), datetime(2024, 7, 2, 23, tzinfo=UTC)],
    "state": ["open", "closed"],
})
for zone in ("UTC", "America/New_York", "Asia/Tokyo", "Australia/Sydney"):
    converted = frame.with_columns(pl.col("timestamp").dt.convert_time_zone(zone))
    print(zone, calendar.filter_sessions(converted)["state"].to_list())

Actual: UTC and New York return ["open"]; Tokyo and Sydney return []. Expected: every representation returns ["open"]. is_session also returns false for the same instant in the last two timezones.

The provider schedule is queried using the input's date before timezone normalization. For this instant, Tokyo and Sydney have already crossed into July 3, but the actual NYSE session is July 2. Query the real exchange's local date after interpreting/normalizing the timestamp, retaining the existing naive-timestamp convention. Add end-user filtering regressions for winter/summer, midnight crossing, holidays and explicit presentation-timezone overrides, without changing next_open/previous_close or the no-provider fallback contract.

This is separate from the CME zero-duration break repair in #87/#86. A focused repair should retain that work rather than mix unrelated calendar APIs into its dependency PR.

AI assistance: Codex reproduced the published-package behavior through actual Polars timezone conversion and public filtering, inspected the implementation and drafted this report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions