PPTX: images silently dropped in two mc:AlternateContent structures (oleObj preview not walked; first_descendant walks through AlternateContent)

Summary

pptx conversion silently drops images in two MC-related structures. Both are in the
pptx reader only, and both lose content, not formatting:

<p:oleObj> children are never walked. For an embedded OLE object, the preview
image lives in a <p:pic> inside <p:oleObj>. parse_shapes has no "oleObj"
arm, so it falls through and the preview image is never emitted. (The OLE payload
itself is not an image — it is an OLE2 compound document.)
parse_graphic_frame uses first_descendant, which walks through
mc:AlternateContent. When a graphicData wraps its content in
mc:AlternateContent, first_descendant returns the node from whichever branch
happens to come first in document order — typically mc:Choice (the VML variant,
which carries the OLE payload and no preview image) instead of mc:Fallback
(which carries the <p:pic> preview). Choosing the branch per the MC spec fixes it.

Both were verified against real corporate .pptx decks.

Version

anydoc 0.1.9 (built from the published crate; weknora-docreader/WeKnora pins
anydoc via the local third_party/anydoc-go bindings).

Reproduction

A minimal fixture for (1):

<p:graphicFrame>
  <a:graphic><a:graphicData uri=".../ole">
    <mc:AlternateContent>
      <mc:Choice Requires="v">
        <p:oleObj r:id="rId5" progId="Excel.Sheet.12" name="Sheet1">
          <p:embed/>            <!-- OLE2 payload: NOT a raster image -->
        </p:oleObj>
      </mc:Choice>
      <mc:Fallback>
        <p:oleObj r:id="rId5" progId="Excel.Sheet.12" name="Sheet1">
          <p:pic>                <!-- ← the preview image that should be emitted -->
            <p:blipFill><a:blip r:embed="rId6"/></p:blipFill>
            <p:spPr><a:xfrm>…</a:xfrm></p:spPr>
          </p:pic>
        </p:oleObj>
      </mc:Fallback>
    </mc:AlternateContent>
  </a:graphicData></a:graphic>
</p:graphicFrame>

Current behaviour: no image block for the preview (case 1), and in case 2 the payload
is picked up as if it were the image.

Measurements (this corpus)

Census over our decks (ppt/slides/slide*.xml):

	count
`.pptx` with `mc:AlternateContent` on a slide	12
`.pptx` with `p:oleObj` on a slide	11

Impact, measured end-to-end by reconciling "images referenced by a slide" against
"images the pipeline actually produced" for all 235 indexed decks:

this class accounted for 19 images across 10 documents that were missing from the
index (the payload was being surfaced as an asset; the preview image never was);
after the two changes below, the reconciliation for the whole corpus is 0 missing,
and the not a raster image retry failures stopped appearing in the logs.

Symptom is silent: the document still reports as successfully indexed, the pages just
have no content for those objects. Downstream (a vision model) additionally fails on the
OLE payload with not a raster image, burning retries against a payload that can never
succeed.

Proposed fix

Two changes, both in src/formats/pptx/mod.rs (diff below, applies to 0.1.9 with
-p1). crate::shared::mc::alternate_branch already exists and is used correctly in
parse_shapes; parse_graphic_frame just needs to use it too.

@@ fn parse_shapes @@
         if child.is(ns::MC, "AlternateContent") {
             if let Some(branch) = crate::shared::mc::alternate_branch(child, SUPPORTED_NS) {
                 parse_shapes(branch, ctx, blocks)?;
+            } else {
             }
             continue;
         }
@@
             "graphicFrame" => parse_graphic_frame(child, ctx, blocks)?,
+            // OLE wrapper nodes (`<p:oleObj>`) keep their preview image in a child
+            // `<p:pic>`. Skipping the wrapper means that image is never seen.
+            "oleObj" => parse_shapes(child, ctx, blocks)?,
             "pic" => { … }

@@ fn parse_graphic_frame @@
-    if let Some(tbl) = frame.first_descendant(ns::A, "tbl") {
+    // `graphicData` can be wrapped in `mc:AlternateContent` directly (PowerPoint's
+    // version-compat wrapper for embedded objects/charts). Pick the branch per the MC
+    // spec first: `first_descendant` walks *through* the wrapper and returns whichever
+    // branch comes first in document order — typically `mc:Choice` (VML, payload only,
+    // no preview image).
+    let scope = match frame.first_descendant(ns::MC, "AlternateContent") {
+        Some(alt) => crate::shared::mc::alternate_branch(alt, SUPPORTED_NS).unwrap_or(frame),
+        None => frame,
+    };
+    if let Some(tbl) = scope.first_descendant(ns::A, "tbl") {
         parse_table(tbl, ctx, blocks)?;
         return Ok(());
     }
-    // Embedded OLE objects: retain identity, media type, and payload.
-    if let Some(ole) = frame.first_descendant(ns::P, "oleObj") {
+    // An embedded object holds two things, and only one of them is an image:
+    //   · the payload (`r:id` → an OLE2 compound document) — NOT a raster image;
+    //   · the preview image (`<p:pic>`, a sibling of `<p:embed>`) — what the reader
+    //     is supposed to see.
+    // So do not surface the payload as an image asset; keep the caption and walk into
+    // the wrapper so the preview image is picked up like any other picture.
+    if let Some(ole) = scope.first_descendant(ns::P, "oleObj") {
         let prog_id = ole.attr(ns::P, "progId").unwrap_or("object");
         let name = ole.attr(ns::P, "name").unwrap_or("").trim();
         let alt =
             if name.is_empty() { format!("Embedded object: {prog_id}") } else { name.to_string() };
-        let source = match ole.attr_qualified(ns::R, "id") {
-            Some(rid) => match ctx.rel_part(rid)? {
-                Some((part, bytes)) => Some(ImageSource::Asset(ctx.assets.borrow_mut().add(
-                    "application/vnd.ms-ole-object".into(),
-                    part,
-                    &bytes,
-                )?)),
-                None => None,
-            },
-            None => None,
-        };
         blocks.push(Block::Paragraph(vec![Inline::Image {
             alt,
-            source: source.unwrap_or(ImageSource::Unavailable),
+            source: ImageSource::Unavailable,
         }]));
+        parse_shapes(ole, ctx, blocks)?;
         return Ok(());
     }
-    if let Some(chart_ref) = frame.first_descendant(ns::CHART, "chart")
+    if let Some(chart_ref) = scope.first_descendant(ns::CHART, "chart")

Notes

We are carrying this as a downstream patch. Happy to open a PR if you'd like —
but the oleObj change is arguably a product decision (it replaces "show the OLE
payload as an asset" with "show the preview image"), so I'd rather ask first.
The else {} in the first hunk is cosmetic in the original patch; the functional
change there is only the added "oleObj" arm.

Both defects still reproduce on current main (261fc257d17c3eab0f673be31c408fd9fdc2171a, anydoc 0.2.4), so this is not confined to 0.1.9. parse_graphic_frame now does have an oleObj arm, but it closes neither hole: it reaches the wrapper with first_descendant (walking straight through mc:AlternateContent) and it never walks the wrapper for the <p:pic> preview, emitting the OLE2 payload as the image instead.

I built two minimal fixtures, each with a control <p:pic> (rId7 -> ppt/media/control1.png) next to an embedded object whose preview is rId6 -> ppt/media/preview1.png and whose payload is rId5 -> ppt/embeddings/oleObject1.bin:

ole-nested-ac.pptx - mc:AlternateContent inside a:graphicData; mc:Choice Requires="v" holds a payload-only p:oleObj, mc:Fallback holds the same p:oleObj with the <p:pic> preview.
ole-plain.pptx - the same p:oleObj with its <p:pic> preview and no mc:AlternateContent at all, to separate defect 1 from branch selection.

The harness converts each deck and reconciles the parts the slide references against document.assets:

$ cargo build --release --example olerepro
$ ./target/release/examples/olerepro ole-nested-ac.pptx ole-plain.pptx
=== ole-nested-ac.pptx ===
--- markdown ---
## OLE preview repro

control picture

Sheet1
--- assets (2) ---
  id=0 media_type=image/png origin_part=ppt/media/control1.png bytes=69
  id=1 media_type=application/vnd.ms-ole-object origin_part=ppt/embeddings/oleObject1.bin bytes=512
  referenced ppt/media/control1.png -> produced=true
  referenced ppt/media/preview1.png -> produced=false
  referenced ppt/embeddings/oleObject1.bin -> produced=true

=== ole-plain.pptx ===
--- markdown ---
## OLE preview repro

control picture

Sheet1
--- assets (2) ---
  id=0 media_type=image/png origin_part=ppt/media/control1.png bytes=69
  id=1 media_type=application/vnd.ms-ole-object origin_part=ppt/embeddings/oleObject1.bin bytes=512
  referenced ppt/media/control1.png -> produced=true
  referenced ppt/media/preview1.png -> produced=false
  referenced ppt/embeddings/oleObject1.bin -> produced=true

The control picture survives, so the pipeline is fine; the preview is dropped in both shapes, and the OLE2 compound document is what lands in Inline::Image. Applying only the MC branch selection (so mc:Fallback is chosen correctly) does not recover it, because the oleObj arm still never looks at p:pic:

  id=0 media_type=image/png origin_part=ppt/media/control1.png bytes=69
  id=1 media_type=application/vnd.ms-ole-object origin_part=ppt/embeddings/oleObject1.bin bytes=512
  referenced ppt/media/control1.png -> produced=true
  referenced ppt/media/preview1.png -> produced=false
  referenced ppt/embeddings/oleObject1.bin -> produced=true

One thing worth checking before the patch lands as written. Dropping the payload asset is not only a product decision, it is an invariant the suite currently asserts. Your diff applied verbatim to main gives:

thread 'embedded_ole_payload_is_retained' panicked at tests/snapshots.rs:141:10:
OLE payload retained as an asset

failures:
    embedded_ole_payload_is_retained

test result: FAILED. 8 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out

and tests/snapshots.rs says it out loud two tests down: "Standard Word OLE markup places a VML preview image next to the o:OLEObject; the object payload (not the preview) must be retained."

An additive variant satisfies both: select the MC branch, retain the payload exactly as today, and prefer the wrapper's own a:blip for the image source when a preview exists.

@@ -563,17 +563,26 @@ fn parse_graphic_frame(
     ctx: &SlideCtx,
     blocks: &mut Vec<Block>,
 ) -> Result<(), ConvertError> {
-    if let Some(tbl) = frame.first_descendant(ns::A, "tbl") {
+    // `graphicData` content can be wrapped in `mc:AlternateContent`; pick the branch
+    // per the MC spec, because `first_descendant` walks *through* the wrapper and
+    // returns whichever branch is first in document order (usually `mc:Choice`).
+    let scope = match frame.first_descendant(ns::MC, "AlternateContent") {
+        Some(alt) => crate::shared::mc::alternate_branch(alt, SUPPORTED_NS).unwrap_or(frame),
+        None => frame,
+    };
+    if let Some(tbl) = scope.first_descendant(ns::A, "tbl") {
         parse_table(tbl, ctx, blocks)?;
         return Ok(());
     }
-    // Embedded OLE objects: retain identity, media type, and payload.
-    if let Some(ole) = frame.first_descendant(ns::P, "oleObj") {
+    // Embedded OLE objects: retain identity, media type, and payload. The payload is
+    // an OLE2 compound document, so when the wrapper also carries a `<p:pic>` preview
+    // that preview is the image; the payload stays retained as an asset either way.
+    if let Some(ole) = scope.first_descendant(ns::P, "oleObj") {
         let prog_id = ole.attr(ns::P, "progId").unwrap_or("object");
         let name = ole.attr(ns::P, "name").unwrap_or("").trim();
         let alt =
             if name.is_empty() { format!("Embedded object: {prog_id}") } else { name.to_string() };
-        let source = match ole.attr_qualified(ns::R, "id") {
+        let payload = match ole.attr_qualified(ns::R, "id") {
             Some(rid) => match ctx.rel_part(rid)? {
                 Some((part, bytes)) => Some(ImageSource::Asset(ctx.assets.borrow_mut().add(
                     "application/vnd.ms-ole-object".into(),
@@ -584,13 +593,26 @@ fn parse_graphic_frame(
             },
             None => None,
         };
+        let preview = match ole
+            .first_descendant(ns::A, "blip")
+            .and_then(|b| b.attr_qualified(ns::R, "embed").or_else(|| b.attr_qualified(ns::R, "link")))
+        {
+            Some(rid) => crate::shared::assets::rel_image_source(
+                ctx.pkg,
+                ctx.rels,
+                ctx.base_part,
+                ctx.assets,
+                rid,
+            )?,
+            None => None,
+        };
         blocks.push(Block::Paragraph(vec![Inline::Image {
             alt,
-            source: source.unwrap_or(ImageSource::Unavailable),
+            source: preview.or(payload).unwrap_or(ImageSource::Unavailable),
         }]));
         return Ok(());
     }
-    if let Some(chart_ref) = frame.first_descendant(ns::CHART, "chart")
+    if let Some(chart_ref) = scope.first_descendant(ns::CHART, "chart")
         && let Some(rid) = chart_ref.attr_qualified(ns::R, "id")
         && let Some((part, bytes)) = ctx.rel_part(rid)?
     {
@@ -601,7 +623,7 @@ fn parse_graphic_frame(
         }
         return Ok(());
     }
-    if let Some(rel_ids) = frame.first_descendant(ns::DGM, "relIds")
+    if let Some(rel_ids) = scope.first_descendant(ns::DGM, "relIds")
         && let Some(rid) = rel_ids.attr_qualified(ns::R, "dm")
         && let Some((part, bytes)) = ctx.rel_part(rid)?
     {

=== ole-nested-ac.pptx ===
--- assets (3) ---
  id=0 media_type=image/png origin_part=ppt/media/control1.png bytes=69
  id=1 media_type=application/vnd.ms-ole-object origin_part=ppt/embeddings/oleObject1.bin bytes=512
  id=2 media_type=image/png origin_part=ppt/media/preview1.png bytes=69
  referenced ppt/media/control1.png -> produced=true
  referenced ppt/media/preview1.png -> produced=true
  referenced ppt/embeddings/oleObject1.bin -> produced=true

=== ole-plain.pptx ===   (identical: 3 assets, all three referenced parts produced)

$ cargo test --release
running 286 tests ... test result: ok. 286 passed; 0 failed
running 1 test   ... test result: ok. 1 passed; 0 failed        (tests/robustness.rs)
running 10 tests ... test result: ok. 9 passed; 0 failed; 1 ignored  (tests/snapshots.rs)

That keeps application/vnd.ms-ole-object in document.assets (so embedded_ole_payload_is_retained and its docx sibling stay green) while the image inline points at a real raster, which also removes the not a raster image retry burn you described at its source rather than downstream.

The part that makes this expensive is the reporting: a deck with unreachable images still converts Ok, so the only way to see it is the referenced-vs-produced reconciliation you already ran. We keep a catalog of that failure shape - work that finishes green with content missing (ours): https://github.com/aron-intframe/agent-guards

Navigation Menu

Repository navigation

PPTX: images silently dropped in two mc:AlternateContent structures (oleObj preview not walked; first_descendant walks through AlternateContent) #182

Summary

Version

Reproduction

Measurements (this corpus)

Proposed fix

Notes

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

PPTX: images silently dropped in two mc:AlternateContent structures (oleObj preview not walked; first_descendant walks through AlternateContent) #182

Description

Summary

Version

Reproduction

Measurements (this corpus)

Proposed fix

Notes

Activity

aron-intframe commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions