Skip to content

BUG: Use the integer part for the pilot lag of the automatic bandwidth - #983

Merged
bashtage merged 2 commits into
mainfrom
wip-nw-pilot-lag
Oct 29, 2026
Merged

bashtage merged 2 commits into
mainfrom
wip-nw-pilot-lag

Conversation

@bashtage

Copy link
Copy Markdown
Owner

Target 9.0. xref #964, #887. Found while reviewing #964: the default bandwidth of the kernel estimators did not match Newey and West (1994) or sandwich::bwNeweyWest.

The nonparametric bandwidth of Newey and West (1994) estimates its scale from the
autocovariances up to the lag n = [4 (T/100)^r], the integer part (eq. 2-2 of the
paper). arch.unitroot.auto_bandwidth and sandwich::bwNeweyWest in R do this. The
kernel covariance estimators in arch.covariance.kernel rounded up, which used one more
autocovariance whenever 4 (T/100)^r is not an integer.

  • _alpha_q now uses int(4 * (T / 100) ** r), and the opt_bandwidth docstring
    says so.
  • The bandwidths now equal those of bwNeweyWest. A new test compares them for the
    Bartlett, Parzen and quadratic-spectral kernels on the univariate and multivariate test
    data (R 4.6.1, sandwich 3.1.3, rtol 1e-11). The test fails on main.
  • The stored automatic bandwidths and weights in test_covariance.py and the formulas
    in two tests that restate the documented estimate are updated.
  • Release note in "Changes that alter results". Every estimate with an automatic
    bandwidth changes, including PreWhitenedRecolored, DOLS, FMOLS, CCR and
    phillips_ouliaris without bandwidth. For the daily S&P 500 percent returns in
    arch.data.sp500, the Bartlett, Parzen and QS bandwidths change from 22.88, 31.99 and
    13.25 to 31.96, 35.50 and 13.61.

Tests: the fast suite has 27,021 passed and 19 skipped. flake8, ruff, black, isort
and numpydoc lint are clean.

🤖 Generated with Claude Code

The nonparametric bandwidth of Newey and West (1994) estimates its scale
from the autocovariances up to the integer part of 4 (T/100)^r (eq. 2-2),
which auto_bandwidth and sandwich::bwNeweyWest use. The kernel covariance
estimators rounded up, which used one more autocovariance whenever the
value is not an integer. The bandwidths now equal those of bwNeweyWest,
which a new test checks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.98%. Comparing base (00eb3d1) to head (f932759).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #983      +/-   ##
==========================================
+ Coverage   98.95%   98.98%   +0.02%     
==========================================
  Files         138      140       +2     
  Lines       30148    30898     +750     
  Branches     2339     2390      +51     
==========================================
+ Hits        29834    30583     +749     
- Misses        256      257       +1     
  Partials       58       58              
Flag Coverage Δ
adder 98.92% <100.00%> (+0.02%) ⬆️
subtractor 98.92% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

With the new pilot lag, a weight of the quadratic-spectral kernel of the
VAR residuals is near one of its zeros, and the weights of the two
estimates differed by 4.4e-13 on Linux, which is 1.5e-7 relative to that
weight. The weights are compared with atol 1e-11 as well.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@bashtage
bashtage merged commit 4e47465 into main Oct 29, 2026
26 checks passed
jngod2011 pushed a commit to jngod2011/arch that referenced this pull request Oct 31, 2026
- Highlights and new features: NGARCH (bashtage#966), the VaR backtests (bashtage#965), the
  local block bootstrap (bashtage#967) and bandwidth_method (bashtage#964, and
  phillips_ouliaris here).
- Changes that alter results: the automatic bandwidth entry of bashtage#983 now
  also describes the bandwidth of the kernels with bounded support, the
  fractional bandwidths and the Tukey-Parzen constant of bashtage#964, its S&P 500
  bandwidths are those of the bandwidth attribute (30.96 and 34.50, where it
  said the 31.96 and 35.50 of opt_bandwidth), and it links phillips_ouliaris
  in arch.unitroot.cointegration. The ADF-Z tables of bashtage#972, bashtage#973 and bashtage#980.
- Backward-incompatible changes: strings in object data.
- Bug fixes: bashtage#954, bashtage#970, bashtage#974, bashtage#975, bashtage#977, bashtage#978, bashtage#982, and the constant
  data of the Newey-West bandwidth and mackinnonp at -inf and inf.
- Maintenance: the CI changes and the version file.
- Contributors: galafis, jackiectl2, mkzung and wavebyrd.
@bashtage
bashtage deleted the wip-nw-pilot-lag branch November 3, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant