Skip to content

fix: skip empty ByteStream in HTMLToDocument to avoid noisy lxml logs#11670

Merged
bogdankostic merged 1 commit into
deepset-ai:mainfrom
Srivatsa03:fix/html-empty-bytestream-logs
Jun 18, 2026
Merged

fix: skip empty ByteStream in HTMLToDocument to avoid noisy lxml logs#11670
bogdankostic merged 1 commit into
deepset-ai:mainfrom
Srivatsa03:fix/html-empty-bytestream-logs

Conversation

@Srivatsa03

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

When HTMLToDocument receives a ByteStream with empty data (b""), it passes the empty content to trafilatura's extract(). Internally lxml then emits several ERROR/WARNING log lines (e.g. Document is empty, discarding data: None) that surface as noise in production logs, even though the component already skips the source gracefully.

The existing try/except around extract() cannot suppress these because lxml logs the problem internally rather than raising an exception.

This PR adds a guard that skips any ByteStream whose data is empty before extract() is called, so lxml is never invoked and no noisy logs are produced. The empty source is skipped with a single WARNING-level message, consistent with the component's other skip paths.

How did you test it?

  • Added a unit test test_run_empty_bytestream that runs the converter on an empty ByteStream and asserts:
    • no documents are produced,
    • trafilatura.extract is never called (mock_extract.assert_not_called()),
    • the skip is logged.
  • Ran locally:
    • hatch run test:unit test/components/converters/test_html_to_document.py — 14 passed
    • hatch run test:types — success, no issues
    • hatch run fmt — all checks passed

Notes for the reviewer

The fix is intentionally minimal and placed right after the source is read, mirroring the existing skip-on-error pattern in run(). A release note has been added under releasenotes/notes/.

Checklist

When HTMLToDocument received a ByteStream with empty data (b""), it passed
the empty content to trafilatura's extract(), which made lxml emit several
ERROR/WARNING log lines internally (e.g. "Document is empty"). The existing
try/except could not suppress these because lxml logs rather than raises.

Empty ByteStream objects are now skipped before extraction, so lxml is never
invoked and no noisy logs are produced.
@Srivatsa03 Srivatsa03 requested a review from a team as a code owner June 17, 2026 22:12
@Srivatsa03 Srivatsa03 requested review from bogdankostic and removed request for a team June 17, 2026 22:12
@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

@Srivatsa03 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/converters
  html.py
Project Total  

This report was generated by python-coverage-comment-action

@bogdankostic bogdankostic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks @Srivatsa03! :)

@bogdankostic bogdankostic merged commit a820fc5 into deepset-ai:main Jun 18, 2026
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: HTMLToDocument generates noisy ERROR logs when receiving empty ByteStream objects

2 participants