Skip to content

fix: remove undeclared SciPy dependency from core Hurst feature #96

Closed

Description

@stefan-jansen

Core Hurst feature must not require undeclared SciPy through Numba least squares

Affected verified source: main0e7865273cd462c507de02a9d3b64de7c6cbb099, installed via the exact-tree qualified development wheel. Python3.12.11, NumPy2.5.4, Numba0.68.0, Polars2.0.0. Fresh17-package base installation has no SciPy.

Public workflow
import numpy as np
import polars as pl
from datetime import datetime, timedelta
from ml4t.engineer import compute_features
close = 100 + np.cumsum(np.random.default_rng(42).normal(0, .1, 300))
data = pl.DataFrame({'timestamp': [datetime(2025,1,1)+timedelta(hours=i) for i in range(300)], 'open': close, 'high': close+1, 'low': close-1, 'close': close, 'volume': np.ones(300)})
compute_features(data, ['hurst_exponent'], calendar=None)

Observed
ImportError: scipy0.16+ is required for linear algebra, from Numba np.linalg.lstsq used by hurst_exponent_nb. The actual fresh core environment reproduces it without mocking or changing imports. Full shared3.15 qualification independently found the same dependency problem in Hurst and registry/public property workflows. This is not specific to prereleases.

Expected and proposed focused repair
Core features work with declared core dependencies. Preserve the existing R/S definition, lag selection, warmup and intercept-inclusive OLS slope. Compute that scalar slope by centered sums instead of calling Numba LAPACK/SciPy; do not add a heavyweight runtime dependency or change the numerical definition. Keep the owning Python3.15 qualification PR93 separate, with this repair as a concrete source prerequisite.

Acceptance
Fail on the actual installed base core package; pass on a clean installed repair wheel without SciPy. Permanent eager/lazy public-workflow tests must expose missing SciPy and preserve checked numerical controls. Run focused tests, all applicable hosted supported source/wheel/platform/docs/security/package gates and ordinary reviewed-green merge. No release/deployment is requested. Resource coordination excludes heavy local full suites.

AI assistance: Codex inspected failed hosted logs, reproduced the bug in an actual minimal installed environment, and prepared the focused source/test repair and evidence. The maintainer remains accountable for the mathematical definition and checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions