Skip to content

fix: preserve rows when adding trends to duplicate-index data - #978

Merged
bashtage merged 1 commit into
bashtage:mainfrom
jackiectl2:fix-add-trend-duplicate-index
Oct 25, 2026
Merged

bashtage merged 1 commit into
bashtage:mainfrom
jackiectl2:fix-add-trend-duplicate-index

Conversation

@jackiectl2

Copy link
Copy Markdown
Contributor

Summary

add_trend joins generated columns on the DataFrame index. If the index contains duplicate labels, pandas performs a many-to-many join: three observations with index a, b, a become five rows, and trend values attach to the wrong observations.

Concatenate the original and generated columns in their existing row order. The result keeps the original index, row count, values, and prepend order.

Verification

  • Added regression tests for both prepend modes with a repeated, noncontiguous index. Both fail on main with a 5-row result instead of 3 rows.
  • The target test file passes after the fix: 16 passed. The test run loaded the actual arch.utility source while bypassing the package initializer because compiled extensions are unavailable here.
  • Ruff lint, Black format, and git diff --check passed. Ruff format on the source file still reports a pre-existing unrelated blank line; the new test file passes Ruff format.
  • The full repository test suite was not run.

Concatenate trend columns by position instead of joining on index
labels. A duplicated DataFrame index otherwise multiplies rows and
mixes trend values between observations.

Signed-off-by: Tianlang (Jackie) Chen <[email protected]>
@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.95%. Comparing base (d638400) to head (ee6f775).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #978   +/-   ##
=======================================
  Coverage   98.95%   98.95%           
=======================================
  Files         138      138           
  Lines       30141    30148    +7     
  Branches     2339     2339           
=======================================
+ Hits        29827    29834    +7     
  Misses        256      256           
  Partials       58       58           
Flag Coverage Δ
adder 98.90% <100.00%> (+<0.01%) ⬆️
subtractor 98.90% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bashtage
bashtage merged commit d0c59dd into bashtage:main Oct 25, 2026
18 checks passed
@bashtage

Copy link
Copy Markdown
Owner

Thanks

jngod2011 pushed a commit to jngod2011/arch that referenced this pull request Oct 31, 2026
- Highlights and new features: NGARCH (bashtage#966), the VaR backtests (bashtage#965), the
  local block bootstrap (bashtage#967) and bandwidth_method (bashtage#964, and
  phillips_ouliaris here).
- Changes that alter results: the automatic bandwidth entry of bashtage#983 now
  also describes the bandwidth of the kernels with bounded support, the
  fractional bandwidths and the Tukey-Parzen constant of bashtage#964, its S&P 500
  bandwidths are those of the bandwidth attribute (30.96 and 34.50, where it
  said the 31.96 and 35.50 of opt_bandwidth), and it links phillips_ouliaris
  in arch.unitroot.cointegration. The ADF-Z tables of bashtage#972, bashtage#973 and bashtage#980.
- Backward-incompatible changes: strings in object data.
- Bug fixes: bashtage#954, bashtage#970, bashtage#974, bashtage#975, bashtage#977, bashtage#978, bashtage#982, and the constant
  data of the Newey-West bandwidth and mackinnonp at -inf and inf.
- Maintenance: the CI changes and the version file.
- Contributors: galafis, jackiectl2, mkzung and wavebyrd.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants