Repository navigation
fix(importer): keep subtitle cues that follow two blank lines - #1627
Merged
akitaonrails merged 1 commit intoOct 23, 2026
Merged
akitaonrails merged 1 commit into
akitaonrails merged 1 commit into
Conversation
extract_transcript splits VTT/SRT text into blocks on "\n\n". A gap of two blank lines (or any even number) leaves one "\n" at the start of the next block, and the block's first line was required to be non-empty, so the whole cue was skipped. A WebVTT header followed by two blank lines lost its first cue the same way, and a file with a single cue was then refused as containing no extractable text. WebVTT separates blocks with one or more line terminators after each block's own, so any number of blank lines is valid, and SRT files in the wild do the same. Skip the blank lines a block opens with before reading its first line. Co-Authored-By: Claude Opus 5.5 <[email protected]>
akitaonrails
added a commit
that referenced
this pull request
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The document importer's VTT/SRT reader (
extract_transcriptincompanions/ai-memory-importer/src/document.rs) now skips the blank lines a cue block starts with before it reads the block's first line. Until now, a cue that followed two blank lines (or any even number of them) was dropped.Before, with SRT
1\n00:00:01,000 --> 00:00:02,000\nFirst fact\n\n\n2\n00:00:03,000 --> 00:00:04,000\nSecond fact\nthe reader returnedFirst fact. After, it returnsFirst fact\n\nSecond fact. A WebVTT file whose header is followed by two blank lines lost its first cue. When that was its only cue (WEBVTT\n\n\n00:00:01.000 --> 00:00:02.000\nOnly fact\n),documentrefused the file with "document contained no extractable text". Files with single blank lines between cues produce the same output as before.Why
The reader splits the normalized text into blocks on
"\n\n"(document.rs:1031onmain). With three newlines in a row, the next block starts with a"\n", so its first line is empty. That block then failedlines.first().copied().filter(|line| !line.is_empty())(document.rs:1033), and the whole cue was skipped bycontinue.Extra blank lines are valid input. The WebVTT file structure separates blocks by "one or more WebVTT line terminators" after each block's own terminator, and lets "two or more" follow the header line (https://www.w3.org/TR/webvtt1/#file-structure). The spec's parser also skips any run of line feeds between blocks. SRT has no formal spec, but hand-edited and concatenated subtitle files often contain extra blank lines.
The fix adds
.skip_while(|line| line.is_empty())to the iterator that builds the block's lines. The empty-line filter onfirstis then redundant, so it goes. No related issue or open PR was found. #1601 fixed two other cases in this function (NOTE-prefixed identifiers and CR-only files).Test plan
cargo fmt --check --manifest-path companions/ai-memory-importer/Cargo.tomlgit diff --checkpassescargo clippy --locked --manifest-path companions/ai-memory-importer/Cargo.toml --all-targets -- -D warningscargo test --locked --manifest-path companions/ai-memory-importer/Cargo.tomlThe change was not compiled locally: no Rust toolchain was available where this was prepared, so the three cargo boxes are unchecked and CI's companions job is the first build. The new test
transcripts_keep_cues_after_more_than_one_blank_linecopies the structure oftranscripts_accept_each_supported_line_terminator. It feeds a WebVTT and an SRT input, each with two blank lines before a cue, through LF, CRLF and CR, and expectsFirst fact\n\nSecond fact.I checked the logic with a line-by-line Python port of
extract_transcriptand its helpers:extract_transcriptunit tests, with and without the change. For the inputs ofsubtitle_cues_are_sanitized_before_the_document_plan, the extractor output is the same before and after.""(WebVTT) andFirst fact(SRT) formain, and the expected text with the change.scripts/check-changelog-sections.shandscripts/changelog_sections.py frozen origin/mainboth pass.Commit attribution
any unintended identity before requesting merge. See
commit attribution guidance.
Release impact
Patch: bug fix, no new surface
Minor: additive (new flag/subcommand/MCP tool/config key,
new agent harness or provider)
Major (breaking): on-disk format, removed/renamed surface, or
breaking MCP schema change, called out in "What changed" above
The base branch matches:
mainfor a patch, the standing featurebranch (currently
release/3.2) for minor or major work.I checked that this does not redo something already on
mainor thefeature branch, or fixed by a closed issue.
CHANGELOG (merge gate)
CHANGELOG.md[Unreleased]entry under### Fixed.Notes for reviewers
The change affects only how a block's first lines are read. Size bounds, sanitization and the NOTE/STYLE/REGION handling are unchanged, so row 14k of
docs/security-boundaries.mdstill describes the importer correctly. The importer README now says that any number of blank lines may separate cues.Some pages may already have been imported from such a file. Each one keeps its shorter body. A re-run plans the full body for the same digest-derived path, and the existing conflict check refuses to replace the stored page, as #1601 noted for its own fix.
🤖 Generated with Claude Code