Skip to content

bevy_reflect: Deserialize dynamics#15131

Open
MrGVSV wants to merge 6 commits intobevyengine:mainfrom
MrGVSV:mrgvsv/reflect/serializable-dynamic-types
Open

bevy_reflect: Deserialize dynamics#15131
MrGVSV wants to merge 6 commits intobevyengine:mainfrom
MrGVSV:mrgvsv/reflect/serializable-dynamic-types

Conversation

@MrGVSV
Copy link
Copy Markdown
Member

@MrGVSV MrGVSV commented Sep 10, 2024

Objective

Say we have the following code:

#[derive(Reflect)]
#[reflect(from_reflect = false)]
struct TestStruct {
    a: i32,
    b: DynamicStruct,
}

#[derive(Reflect)]
struct OtherStruct {
    c: f32,
}

let my_struct = TestStruct {
    a: 123,
    b: OtherStruct { c: 456.0 }.clone_dynamic(),
};

Using the ReflectSerializer, we can serialize this to get the following output:

{
    "bevy_reflect::serde::tests::TestStruct": (
        a: 123,
        b: (
            c: 456.0,
        ),
    ),
}

However, if we try to deserialize this output, we get hit with an error:

no registration found for type `bevy_reflect::DynamicStruct`

It's understandable why this errors: we can't deserialize the DynamicStruct because we don't know what type it represents...

...or do we?

Solution

This PR makes it so that nested dynamic proxies are serialized as a type map so that we can easily deserialize it back into a dynamic type.

What do I mean by "nested dynamic proxies"? Well, "dynamic proxies" here refers to the fact that these are dynamic types (e.g. DynamicStruct, DynamicArray, etc.) that contain the TypeInfo and structure of a concrete type, thus proxying (or representing) that concrete type.

For example, a DynamicList containing the TypeInfo of Vec<i32> would be a proxy of a Vec<i32>.

And I'm using "nested" to imply that these dynamic types are embedded within another type. Serializing and deserializing a dynamic type as the root already works. It's when the dynamic type is nested as a struct field or list element that it doesn't work.

By serializing a dynamic type as a type map, we give the deserializer knowledge of how to deserialize the dynamic data.

So if we serialize our data again, we get this output:

{
    "bevy_reflect::serde::tests::TestStruct": (
        a: 123,
        b: {
            "bevy_reflect::serde::tests::OtherStruct": (
                c: 456.0,
            ),
        },
    ),
}

With the type information embedded, the deserializer can then know that we need to deserialize a DynamicStruct with the structure of OtherStruct.

Future Work

Originally, I wanted to explore making dynamic types implement FromReflect by simply calling .clone_dynamic() on them.

However, I feel like that would be a lot more controversial (if it even worked), so I thought it would be best to save that for a future PR.

Testing

You can test locally by running:

cargo test --package bevy_reflect

Showcase

You can now serialize and deserialize nested dynamic type data!

#[derive(Reflect)]
#[reflect(from_reflect = false)]
struct TestStruct {
    a: i32,
    // This can only be deserialized if `DynamicStruct`
    // contains the `TypeInfo` of a concrete type
    b: DynamicStruct,
}

#[derive(Reflect)]
struct OtherStruct {
    c: f32,
}

let my_struct = TestStruct {
    a: 123,
    // We use `clone_dynamic` to create the proxy
    // since it internally calls `DynamicStruct::set_represented_type`
    // with the `TypeInfo` of `OtherStruct`
    b: OtherStruct { c: 456.0 }.clone_dynamic(),
};

let serializer = ReflectSerializer::new(&value, &registry);
let output = ron::ser::to_string_pretty(&serializer, Default::default()).unwrap();
// {
//     "bevy_reflect::serde::tests::TestStruct": (
//         a: 123,
//         b: {
//             "bevy_reflect::serde::tests::OtherStruct": (
//                 c: 456.0,
//             ),
//         },
//     ),
// }

let mut deserializer = ron::de::Deserializer::from_str(&output).unwrap();
let reflect_deserializer = ReflectDeserializer::new(&registry);

let result = reflect_deserializer.deserialize(&mut deserializer).unwrap();
assert!(my_struct.reflect_partial_eq(result.as_partial_reflect()).unwrap());

Migration Guide

The serialized representation of dynamic types have changed. Dynamic types will now serialize as a type map rather than just as the type itself.

{
    "my_crate::MyStruct": (
-        dynamic_struct_field: (
-            c: 1.23,
-        ),
+        dynamic_struct_field: {
+            "my_crate::MyOtherStruct": (
+                c: 1.23,
+            ),
+        },
    ),
}

Note that this only affects types where they are strongly-typed as a dynamic type. For example, Vec<MyStruct> would not be affected, but Vec<DynamicTuple> would be affected.

@MrGVSV MrGVSV added C-Usability A targeted quality-of-life change that makes Bevy easier to use A-Reflection Runtime information about types D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes labels Sep 10, 2024
@alice-i-cecile alice-i-cecile added the S-Needs-Review Needs reviewer attention (from anyone!) to move forward label Sep 10, 2024
@MrGVSV MrGVSV added the M-Migration-Guide A breaking change to Bevy's public API that needs to be noted in a migration guide label Sep 10, 2024
@MrGVSV MrGVSV force-pushed the mrgvsv/reflect/serializable-dynamic-types branch from fdbe638 to e8a0c79 Compare September 15, 2024 22:56
@yrns
Copy link
Copy Markdown
Contributor

yrns commented Sep 21, 2024

It seems surprising that this doesn't work either, currently. That you need the original type information to de/serialize any dynamic types:

#[derive(Reflect)]
#[reflect(from_reflect = false)]
struct TestStruct {
    a: i32,
    b: DynamicStruct,
}

fn main() {
    let mut registry = TypeRegistry::default();
    registry.register::<TestStruct>();

    let mut dyn_struct = DynamicStruct::default();
    dyn_struct.insert("c", 456.0);

    let my_struct = TestStruct {
        a: 123,
        // b: OtherStruct { c: 456.0 }.clone_dynamic(),
        b: dyn_struct,
    };

    let reflect_serializer = ReflectSerializer::new(&my_struct, &registry);
    let output = ron::to_string(&reflect_serializer).unwrap();
}

Results in: "type bevy_reflect::DynamicStruct does not represent any type (stack: dyn_ser::TestStruct)".

@MrGVSV
Copy link
Copy Markdown
Member Author

MrGVSV commented Sep 21, 2024

It seems surprising that this doesn't work either, currently. That you need the original type information to de/serialize any dynamic types:

#[derive(Reflect)]
#[reflect(from_reflect = false)]
struct TestStruct {
    a: i32,
    b: DynamicStruct,
}

fn main() {
    let mut registry = TypeRegistry::default();
    registry.register::<TestStruct>();

    let mut dyn_struct = DynamicStruct::default();
    dyn_struct.insert("c", 456.0);

    let my_struct = TestStruct {
        a: 123,
        // b: OtherStruct { c: 456.0 }.clone_dynamic(),
        b: dyn_struct,
    };

    let reflect_serializer = ReflectSerializer::new(&my_struct, &registry);
    let output = ron::to_string(&reflect_serializer).unwrap();
}

Results in: "type bevy_reflect::DynamicStruct does not represent any type (stack: dyn_ser::TestStruct)".

The reason this doesn't work is because there is no associated type for dyn_struct. This means we'd have to encode the full type information/structure of dyn_struct along with its value.

This could be a lot of information depending on how much nested data dyn_struct contains.

It might be something we do, but it would be a much larger and more controversial change. On top of that, the exact format and manner in which we serialize that type information will likely need proper design.

.unwrap_err();
assert_eq!(
error,
ron::Error::Message(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we really assert the message? Wouldn't a matches! here do the work?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is specifically checking that the debug stack correctly handles dynamic types. And the way we check this is the error message

@BenjaminBrienen BenjaminBrienen added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 31, 2024
@cart cart added this to Reflection Feb 12, 2026
@github-project-automation github-project-automation bot moved this to Needs SME Triage in Reflection Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Reflection Runtime information about types C-Usability A targeted quality-of-life change that makes Bevy easier to use D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes M-Migration-Guide A breaking change to Bevy's public API that needs to be noted in a migration guide S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged

Projects

Status: Needs SME Triage
Status: In Progress

Development

Successfully merging this pull request may close these issues.

6 participants