Skip to content

BUG: describe/Description handles 0-row (empty) input gracefully (#9891)#9899

Merged
bashtage merged 2 commits into
statsmodels:mainfrom
PranavAchar01:fix-describe-empty-input-9891
Jul 20, 2026
Merged

BUG: describe/Description handles 0-row (empty) input gracefully (#9891)#9899
bashtage merged 2 commits into
statsmodels:mainfrom
PranavAchar01:fix-describe-empty-input-9891

Conversation

@PranavAchar01

Copy link
Copy Markdown
Contributor

Closes #9891.

Problem

describe / Description raised an opaque error, deep from pandas/numpy, when given a 0-row DataFrame — even though pandas.DataFrame.describe() handles the same input fine and returns a frame of NaNs. The message depended on the column dtype, hinting at two separate unhandled spots:

  • numeric columnValueError: Length of values (2) does not match length of index (1)
  • categorical columnKeyError: 2

Root cause

Both live in Description.numeric (which describe computes by default):

  1. df.apply(_mode) on a 0-row frame takes pandas' empty-result path, which calls _mode on an empty column; _mode returns a 2-tuple, so pandas tries to fit 2 values into the 1-length column index → ValueError.
  2. For all-categorical input the numeric frame has no columns, so the Jarque–Bera result jb has no columns and jb[2]/jb[3] (skew/kurtosis) → KeyError: 2.

Fix

Short-circuit both undefined computations to NaN when there are no observations:

  • when df.shape[0] == 0, set mode/mode_counts to NaN of the right length instead of calling apply;
  • when there are no observations or no numeric columns, build a NaN Jarque–Bera frame with the expected four columns.

describe now returns an empty/NaN summary (nobs=0, missing=0, other stats NaN), matching pandas.describe. Non-empty inputs are completely unchanged.

Tests

Added a parametrized test_empty_rows covering 0-row numeric, categorical, and mixed frames (asserts no error, correct columns, nobs/missing = 0, NaN stats, and describe == Description(...).frame). Full test_descriptivestats.py passes (18 tests).

Description.numeric raised on 0-row input: df.apply(_mode) hit pandas'
empty-result path and returned a 2-length result for a 1-length index
(ValueError), and for all-categorical input the empty Jarque-Bera frame
had no columns so jb[2]/jb[3] raised KeyError. Short-circuit both undefined
computations to NaN when there are no observations, so describe now returns
an empty/NaN summary (matching pandas.DataFrame.describe) instead of an
opaque error. Adds a parametrized regression test.
Copilot AI review requested due to automatic review settings July 10, 2026 11:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@bashtage bashtage left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some simplification and clean up would help.

Comment thread statsmodels/stats/descriptivestats.py Outdated
Comment thread statsmodels/stats/descriptivestats.py Outdated
- Remove the legacy 'pandas before 1.0' branch in the mode computation;
  df.apply(...).T is always a DataFrame on supported pandas.
- Replace 'df.shape[0] > 0 and df.shape[1] > 0' with 'df.size'.
Addresses bashtage's review on statsmodels#9899.
@bashtage

Copy link
Copy Markdown
Member

Thanks.

@bashtage
bashtage merged commit c83f818 into statsmodels:main Jul 20, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: describe/Description raises unhelpful error on empty (0-row) input

3 participants