Skip to content

reflect: add DynamicMessage::try_from_message and BridgeError - #414

Merged
iainmcgin merged 4 commits into
anthropics:mainfrom
rioyu123:feat/try-from-message
Sep 3, 2026
Merged

iainmcgin merged 4 commits into
anthropics:mainfrom
rioyu123:feat/try-from-message

Conversation

@rioyu123

@rioyu123 rioyu123 commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Addresses items 1 and 2 of #413. Item 3, a reflection path that does not re-encode, remains a separate design topic.

Summary

  • Adds DynamicMessage::try_from_message(msg, pool), which resolves the descriptor through M::FULL_NAME and returns a structured BridgeError instead of panicking when the type is missing or the encoded bytes do not match the pool schema.
  • Adds DynamicMessage::try_from_message_with_index(msg, pool, idx) for callers that already hold a MessageIndex, returning the underlying DecodeError and mapping over-limit encoding to DecodeError::MessageTooLarge.
  • Keeps from_message source-compatible while sharing one bridge decode path across the fallible and panicking entry points.
  • Re-exports the non-exhaustive BridgeError from buffa_descriptor::reflect and the crate root, with Display and std::error::Error::source() support.
  • Documents the pool-identity invariant of the explicit-index APIs: MessageIndex carries no pool identity, so a foreign index may panic when out of range or silently select a different descriptor at the same position.

Value

The new name-resolving entry point gives runtime-schema users a practical, diagnosable bridge from generated messages to DynamicMessage. Missing types and incompatible descriptors become ordinary errors that can be propagated or logged with the generated type name, while existing generated reflection bridges keep their current API.

The bridge uses encoded-length-scaled memory bounds for bytes the process has just encoded from an already-materialized message, matching the existing generated ReflectMessage::to_dynamic policy. This prevents valid large repeated, packed, or map fields from failing against the fixed default while keeping the second representation bounded; the recursion limit and protobuf 2 GiB size limit remain in force.

Compatibility and boundaries

  • from_message is not deprecated; generated reflection code and checked-in WKT output still use it. Its signature and decode-failure panic contract remain, while valid large self-encoded messages no longer hit the fixed default memory budget.
  • The name-resolving try_from_message is recommended whenever MessageName is available because the index is obtained from the same pool.
  • The explicit-index forms require the caller to keep the pool and index associated. The current MessageIndex representation cannot validate that association without a broader API change.
  • Wire-compatible schema drift can still decode successfully, as required by protobuf wire compatibility; only missing types and actual decode failures are reported.
  • Element-memory and unknown-field budgets scale with encoded length and retain their existing default floors. The recursion limit remains enabled.
  • The final bounded policy follows reflect: stop to_dynamic panicking on views that decoded cleanly #341 exactly (encoded_len * 128 for element memory and encoded_len for unknown fields, each floored at its default); no usize::MAX escape hatch remains.

Tests

Ten colocated bridge tests cover:

  • by-name resolution and parity with the explicit-index path;
  • MessageNotFound, Decode, Display, and Error::source() behavior;
  • schema mismatch through both fallible entry points;
  • unchanged from_message decode-failure panic behavior;
  • a large self-encoded message that exceeds the fixed default but succeeds under the scaled finite budget;
  • scaling, default floors, and finite memory-budget values;
  • two-pool out-of-range index panic;
  • empty-encoding deferred descriptor panic;
  • same-position foreign index silently selecting the target pool's different descriptor, contrasted with the safe name-resolving path.

Verification

Windows 11, pinned Rust 1.95.0, protoc 33.5.

  • cargo fmt --all --check: clean.
  • cargo clippy --workspace --all-targets -- -D warnings: clean.
  • RUSTDOCFLAGS=-D warnings cargo doc --workspace --all-features --no-deps: clean.
  • cargo test --workspace -- --skip systemtime_roundtrip_post_epoch: clean.
  • The skipped test is an unchanged Windows baseline failure: SystemTime rounds 123456789 ns to 123456700 ns at 100 ns resolution.
  • Full bridge module: 10 passed.
  • docs/guide.md passes markdownlint. Repository-wide markdownlint has one unchanged baseline finding: AGENTS.md lacks a trailing newline.

Changelog

Includes separate Added and Fixed fragments under .changes/unreleased/, updates the reflection guide with the fallible bridge example and bounded-memory policy, and corrects the existing #341 fragment's wording from "lifted" to the finite scaled-and-floored behavior its code already implements.

@github-actions

github-actions Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@rioyu123

rioyu123 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Sep 2, 2026

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try_from_message_with_index is presented as the fallible bridge, but it still inherits MessageIndex's cross-pool hazard. An index obtained from another pool can panic when its ordinal is out of range, or silently select the wrong descriptor when that ordinal exists in both pools. Since this API accepts the pool and index separately, could we validate/associate them or at least document this remaining panic/silent-mismatch case and cover it with two pools?

@rioyu123

rioyu123 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — confirmed. MessageIndex does not carry pool identity, so the same-ordinal mismatch cannot be validated without changing its representation. I documented that invariant and both the panic and silent-mismatch cases on the explicit-index bridge (and its panicking sibling), corrected the referenced DescriptorPool::message panic contract, and added two-pool regressions for out-of-range, deferred empty-message, and same-ordinal behavior. The name-resolving try_from_message path remains the recommended option when MessageName is available.

rioyu123 and others added 4 commits September 2, 2026 23:34
Fallible, name-resolving bridge from a generated message: resolves the descriptor via MessageName::FULL_NAME and returns BridgeError instead of panicking when the pool lacks the type or rejects the encoded bytes. try_from_message_with_index is the fallible index-taking form; from_message keeps its signature and panics and shares the decode step, which now lifts the untrusted-input element-memory and unknown-field budgets for the process's own bytes.

Refs anthropics#413 (items 1 and 2).
… BridgeError::Decode, scale to_message too

The 128 element-bytes-per-encoded-byte factor was duplicated between
this crate and buffa-codegen's view conversion with nothing pinning it
to size_of::<Value>(); it is now a named constant with a const
assertion. An over-2 GiB message surfaced as "failed to round-trip
through the pool's descriptor: message too large", blaming the
descriptor for an encode-side failure; BridgeError::MessageTooLarge
names it. to_message re-decoded with the fixed untrusted-input budgets,
the mirror of the bug this PR fixes inbound, so a snapshot the bridge
admitted could fail to convert back; it uses the same scaled budgets.
The large-message test builds one budget's worth of values plus one
rather than two. Fragments in house style; the anthropics#341 fragment carries
its PR number.
@iainmcgin
iainmcgin force-pushed the feat/try-from-message branch from 918e105 to c2f9b39 Compare September 3, 2026 06:36
@iainmcgin

Copy link
Copy Markdown
Collaborator

[claude code]

This is a well-built bridge: from_message keeps its signature and panic contract (pinned by test), try_from_message resolves by FULL_NAME from the pool it is given so the cross-pool hazard is confined to the _with_index form and documented and tested there, the scaled budgets match the view conversion from #341, and BridgeError follows AnyError's hand-written shape. Pushed c2f9b39 (rebased on main):

  • The 128 element-bytes-per-encoded-byte factor was duplicated here and in buffa-codegen/src/reflect_view.rs with nothing tying it to size_of::<Value>(); it is now BRIDGE_ELEMENT_BYTES_PER_ENCODED_BYTE with a const assertion on Value and MapKey + Value, so a widening of Value fails to compile rather than silently reintroducing the false rejection on both paths.
  • An over-2 GiB message surfaced as "pkg.M" failed to round-trip through the pool's descriptor: message too large, blaming the descriptor for an encode-side failure. BridgeError::MessageTooLarge { full_name } names it (the _with_index form keeps DecodeError::MessageTooLarge, as to_message does).
  • to_message re-decoded with the fixed untrusted-input budgets — the mirror of the bug this fixes inbound — so a snapshot the bridge admitted could fail to convert back; it now uses the same scaled budgets.
  • bridge_scales_the_untrusted_input_element_budget built ~1M values and held two 64 MiB snapshots alive (it also runs under the 32-bit job); one budget's worth plus one still clears the default and keeps the assertion meaningful.
  • Fragments in house style (bold lead, PR number first); your correction to the reflect: stop to_dynamic panicking on views that decoded cleanly #341 fragment is right and now carries its number.

One thing for @iainmcgin rather than you: bridge-mode reflect() on a message decoded from untrusted input now snapshots at up to 128× the encoded length instead of a hard 32 MiB. It replaces a panic with a bounded allocation and matches the view policy already shipped, but it is a deliberate loosening on the owned path going out in a patch release.

@iainmcgin
iainmcgin added this pull request to the merge queue Sep 3, 2026
Merged via the queue into anthropics:main with commit 71fed8c Sep 3, 2026
11 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 3, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants