Repository navigation
Tracked changes: RTF and DOC print deleted text as live text; DOCX applies them silently #202
Description
Summary
Tracked changes (revision marks) are read differently per format, and in two of them the result misstates the document:
| Format | Deleted text | Inserted text | Signal that revisions exist |
|---|---|---|---|
| DOCX | dropped (w:del skipped) |
kept | none |
| RTF | printed as live text | kept, no mark | none |
| DOC | printed as live text | kept, no mark | none |
So the same Word document gives a different text depending on the format it was saved in. For RTF and DOC the output contains both the old and the new wording side by side, and nothing in it says that one of them was deleted. We use anydoc to feed contracts and court documents to an LLM that quotes them; a deleted wording that reads as in force is the worst case for that use. DOCX's "final document" reading is reasonable (content.rs treats ins/moveTo as part of the final document), but the caller has no way to learn that it was applied, i.e. that the file on disk is not the text a reader of it sees in Word.
Reproduction (anydoc 0.2.4, Python)
The script builds a DOCX with one w:del and one w:ins, and an RTF with one \deleted and one \revised run:
--- docx
The parties agreed NEWTERM terms.
--- rtf
The parties agreed OLDTERM NEWTERM terms.
DOC saved from Word 365 (Document.Revisions.Count == 2) gives the same output as RTF; it is binary, so it is not in the script — happy to attach one if useful. In DOC the marks are sprmCFRMarkDel / sprmCFRMarkIns in the CHPX.
repro.py
"""Minimal reproductions for anydoc 0.2.4: tracked changes and ODT lists/annotations."""
import io, zipfile, anydoc
W = 'xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"'
def docx(body):
b = io.BytesIO()
with zipfile.ZipFile(b, "w") as z:
z.writestr("[Content_Types].xml", '<?xml version="1.0"?><Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>')
z.writestr("_rels/.rels", '<?xml version="1.0"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="r1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/></Relationships>')
z.writestr("word/document.xml", f'<?xml version="1.0"?><w:document {W}><w:body>{body}</w:body></w:document>')
return b.getvalue()
tracked = ('<w:p><w:r><w:t xml:space="preserve">The parties agreed </w:t></w:r>'
'<w:del w:id="1" w:author="A" w:date="2026-01-01T00:00:00Z"><w:r><w:delText xml:space="preserve">OLDTERM </w:delText></w:r></w:del>'
'<w:ins w:id="2" w:author="A" w:date="2026-01-01T00:00:00Z"><w:r><w:t xml:space="preserve">NEWTERM </w:t></w:r></w:ins>'
'<w:r><w:t>terms.</w:t></w:r></w:p>')
rtf = (r"{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}}{\*\revtbl{Unknown;}{A;}}"
r"\f0 The parties agreed {\deleted\revauthdel1 OLDTERM }{\revised\revauth1 NEWTERM }terms.\par}").encode()
ODT_NS = ('xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" '
'xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" '
'xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0" '
'xmlns:dc="http://purl.org/dc/elements/1.1/"')
def odt(text):
styles = (f'<office:automatic-styles><text:list-style style:name="L1">'
'<text:list-level-style-number text:level="1" style:num-format="1" style:num-suffix="."/>'
'<text:list-level-style-number text:level="2" style:num-format="1" style:num-suffix="." text:display-levels="2"/>'
'</text:list-style></office:automatic-styles>')
content = (f'<?xml version="1.0" encoding="UTF-8"?><office:document-content {ODT_NS} office:version="1.2">'
f'{styles}<office:body><office:text>{text}</office:text></office:body></office:document-content>')
b = io.BytesIO()
with zipfile.ZipFile(b, "w") as z:
z.writestr(zipfile.ZipInfo("mimetype"), "application/vnd.oasis.opendocument.text")
z.writestr("META-INF/manifest.xml", '<?xml version="1.0"?><manifest:manifest xmlns:manifest="urn:oasis:names:tc:opendocument:xmlns:manifest:1.0"><manifest:file-entry manifest:full-path="/" manifest:media-type="application/vnd.oasis.opendocument.text"/><manifest:file-entry manifest:full-path="content.xml" manifest:media-type="text/xml"/></manifest:manifest>')
z.writestr("content.xml", content)
return b.getvalue()
# The shape Word writes when it saves a two-level list as ODT: every nested list
# carries text:continue-numbering="true".
lists = ('<text:list text:style-name="L1" text:continue-numbering="true">'
'<text:list-item><text:p>Subject</text:p><text:list text:continue-numbering="true">'
'<text:list-item><text:p>Duty A</text:p></text:list-item><text:list-item><text:p>Duty B</text:p></text:list-item></text:list></text:list-item>'
'<text:list-item><text:p>Price</text:p><text:list text:continue-numbering="true">'
'<text:list-item><text:p>Payment</text:p></text:list-item></text:list></text:list-item></text:list>')
annot = ('<text:p>Text with a comment on Lamb<office:annotation office:name="0"><dc:creator>Author</dc:creator>'
'<dc:date>2026-01-01T00:00:00</dc:date><text:p>COMMENTTEXT</text:p></office:annotation>daword'
'<office:annotation-end office:name="0"/> here.</text:p>')
from importlib.metadata import version
print("anydoc", version("firecrawl-anydoc"))
for name, data in [("docx", docx(tracked)), ("rtf", rtf), ("odt-list", odt(lists)), ("odt-annotation", odt(annot))]:
path = f"repro-{name}." + name.split("-")[0]
open(path, "wb").write(data)
print(f"--- {name}\n{anydoc.to_markdown(path)}")Expected
At least consistency: RTF and DOC skip deleted runs the way DOCX skips w:del. Better still, a signal on the Document (for example counts of insertions and deletions, or a flag) so a caller can refuse, warn, or render the marks, instead of silently getting one reading. The same signal for content that is excluded by policy (headers/footers in RTF, comments — #191) would let callers say what a conversion left out.
Activity