Repository navigation
docx: a bare < in a text run silently truncates the document and returns OK #187
Description
Summary
When a docx text run contains an unescaped < (e.g. <w:t>Pump A < pump B</w:t>), toMarkdownBytes returns OK with the text up to the < and nothing after it — the rest of that paragraph and every later paragraph are dropped. If the < sits in the first paragraph, the result is OK and empty. No error, no warning, so a caller cannot tell the conversion lost content.
A bare < makes document.xml ill-formed, so this is bad input — but it does occur in documents written by non-Word generators, and anydoc already treats the sibling case strictly: a bare & with no later ; is rejected as malformed document: unparseable xml. The < case should behave the same way (an error), or recover the text.
Reproduction
A minimal docx with five paragraphs, the third holding a bare < (Python stdlib only):
import zipfile
def docx(path, paragraphs):
body = "".join(f'<w:p><w:r><w:t xml:space="preserve">{p}</w:t></w:r></w:p>' for p in paragraphs)
parts = {
"[Content_Types].xml": '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
'<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">'
'<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>'
'<Default Extension="xml" ContentType="application/xml"/>'
'<Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/>'
"</Types>",
"_rels/.rels": '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
'<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
'<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>'
"</Relationships>",
"word/document.xml": '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
'<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">'
f"<w:body>{body}</w:body></w:document>",
}
with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as z:
for name, xml in parts.items():
z.writestr(name, xml)
before = ["First paragraph.", "Second paragraph."]
after = ["Fourth paragraph.", "Fifth paragraph."]
docx("escaped-lt.docx", before + ["Pump A < pump B in flow rate."] + after)
docx("bare-lt.docx", before + ["Pump A < pump B in flow rate."] + after)
docx("bare-lt-lead.docx", ["< First paragraph.", "Second paragraph.", "Third paragraph."] + after)
docx("bare-amp.docx", before + ["Pump A & pump B in flow rate."] + after)import { readFileSync } from "node:fs"
import { toMarkdownBytes } from "@firecrawl/anydoc"
for (const f of ["escaped-lt.docx", "bare-lt.docx", "bare-lt-lead.docx", "bare-amp.docx"]) {
try {
console.log(f, JSON.stringify(await toMarkdownBytes(new Uint8Array(readFileSync(f)), "docx")))
} catch (e) {
console.log(f, "threw", e.message)
}
}Output on 0.2.4:
escaped-lt.docx "First paragraph.\n\nSecond paragraph.\n\nPump A < pump B in flow rate.\n\nFourth paragraph.\n\nFifth paragraph.\n"
bare-lt.docx "First paragraph.\n\nSecond paragraph.\n\nPump A\n"
bare-lt-lead.docx ""
bare-amp.docx threw malformed document: unparseable xml: ill-formed document: entity or character reference not closed: `;` not found before end of input
xmllint --noout on bare-lt.docx's word/document.xml reports StartTag: invalid element name, so the part is ill-formed; anydoc appears to stop reading at that point and return what it has so far as a success.
Expected
Either:
- an error, as for the bare
&(malformed document: unparseable xml: …), so the caller can fall back to another converter; or - recovery: read the
<as a literal character and keep the remaining text.
Silently returning a truncated or empty document as success is the one outcome a caller cannot detect.
Also seen
- pptx: a bare
<in an<a:t>loses the rest of that paragraph, also without an error. - xlsx: a bare
<inxl/sharedStrings.xmlreturns OK with later shared strings run together into one cell. - docx
w:altChunk: a body whose content is analtChunk(imported HTML part) converts to OK and empty, also without an error. Possibly a separate issue — happy to split it out.
Version
@firecrawl/anydoc 0.2.4 (Node binding, darwin-arm64), Node 24.
Activity