asdict(msgspec.Struct) doesn't apply Struct settings

Question

class Example(msgspec.Struct, omit_defaults=True):
    bootstrap_servers: str = field(name="bootstrap.servers")
    retries: int = 5

calling

d = msgspec.structs.asdict(Example("some.server"))

you'd expect {"bootstrap.servers": "some.server"} however you actually get

{"bootstrap_servers": "some.server", "retries": 5} -> meaning neither omit_defaults or rename is applied.

Same thing for rename like

class Example(msgspec.Struct, rename="camel"):
    bootstrap_servers: str

asdict gives you {"bootstrap_servers": "some.server"} and not, as expected {"bootstrapServers": "some.server"}

Am I missing something obvious here to achieve this behaviour? It's a bit "just have to know" and not as expected usage if you need the struct as a dict instance for whatever reason (such internal library passing config etc). I mean, dumping it to bytes and then json.load kinda works but its a bit of a hack 🤷‍♂

like:

class Example(msgspec.Struct, rename="camel"):
    bootstrap_servers: str
    
    def to_dict(self) -> dict[str, str]:
        return json.loads(msgspec.json.encode(self))

From the documentation:

If the default values are known on the decoding end (making serializing them redundant), it may be beneficial and desired to omit default values from the encoded message. This can be done by configuring omit_defaults=True as part of the Struct definition

This means that omit_defaults only works when you're encoding/decoding, and is not affected by asdict.

import msgspec
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
    bootstrap_servers: str = field(name="bootstrap.servers")
    retries: int = field(name="retries", default=5)

print(msgspec.json.encode(Example("some.server")))

# output
b'{"bootstrap.servers":"some.server"}'

From the API, asdict doesn't seem to discard any field when omit_defaults is set. I think it's the intended behavior since it's a way to minimize the data to be transmitted in case a configuration parameter is shared between encoder and decoder.

From the documentation:

If the default values are known on the decoding end (making serializing them redundant), it may be beneficial and desired to omit default values from the encoded message. This can be done by configuring omit_defaults=True as part of the Struct definition

This means that omit_defaults only works when you're encoding/decoding, and is not affected by asdict.

import msgspec
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
bootstrap_servers: str = field(name="bootstrap.servers")
retries: int = field(name="retries", default=5)

print(msgspec.json.encode(Example("some.server")))
# output
b'{"bootstrap.servers":"some.server"}'
From the API, asdict doesn't seem to discard any field when omit_defaults is set. I think it's the intended behavior since it's a way to minimize the data to be transmitted in case a configuration parameter is shared between encoder and decoder.

Yup, it makes sense, asdict is a dict representation of the Python msgspec.Struct - not a dict representation of the message it represents (i.e what is to be encoded). Would be a neat feature/option though :)

From what I can see, msgspec seem to support typing.NotRequired, which is usually used in typing.TypedDict to provide optional fields for a dictionary. I'm currently trying to run your example as follows:

from __future__ import annotations

import msgspec
from typing import NotRequired
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
    bootstrap_servers: str = field(name="bootstrap.servers")
    retries: NotRequired[int] = field(name="retries")

print(msgspec.structs.asdict(Example("some.server")))

But I'm getting:

Traceback (most recent call last):
  File "c:\sandbox\python\sandbox.py", line 11, in <module>
    print(msgspec.structs.asdict(Example("some.server")))
                                 ^^^^^^^^^^^^^^^^^^^^^^
TypeError: Missing required argument 'retries'

The only thing I can see for support of NotRequired is this entry in the changelog of version 0.15.1:

Work around an upstream bug in CPython to properly support typing.Required/typing.NotRequired in typing.TypedDict when future.annotations is enabled (PR #410)

But I get the feeling the intended usage of NotRequired is only to provide support for serializing a field which is a typing.TypedDict with required/not required fields.

From what I can see, msgspec seem to support typing.NotRequired, which is usually used in typing.TypedDict to provide optional fields for a dictionary. I'm currently trying to run your example as follows:

from future import annotations

import msgspec
from typing import NotRequired
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
bootstrap_servers: str = field(name="bootstrap.servers")
retries: NotRequired[int] = field(name="retries")

print(msgspec.structs.asdict(Example("some.server")))
But I'm getting:
Traceback (most recent call last):
  File "c:\sandbox\python\sandbox.py", line 11, in <module>
    print(msgspec.structs.asdict(Example("some.server")))
                                 ^^^^^^^^^^^^^^^^^^^^^^
TypeError: Missing required argument 'retries'
The only thing I can see for support of NotRequired is this entry in the changelog of version 0.15.1:

Work around an upstream bug in CPython to properly support typing.Required/typing.NotRequired in typing.TypedDict when future.annotations is enabled (PR #410)

But I get the feeling the intended usage of NotRequired is only to provide support for serializing a field which is a typing.TypedDict with required/not required fields.

If you run this, it will be more clear what I mean:

import msgspec
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
    bootstrap_servers: str = field(name="bootstrap.servers")
    retries: int = field(name="retries", default=5)

print(msgspec.structs.asdict(Example("some.server")))
print(msgspec.json.encode(Example("some.server")))

Yeah, I get the picture. I'm guessing a possible feature would be to replicate the behavior of omit_defaults and pass it as an optional parameter, but I don't really see a use case for this to be honest.

Yeah, I get the picture. I'm guessing a possible feature would be to replicate the behavior of omit_defaults and pass it as an optional parameter, but I don't really see a use case for this to be honest.

Nvm, I think I found it; msgspec.to_builtins does exactly what I needed!

import msgspec
from msgspec import field, Struct

class Example(Struct, omit_defaults=True):
    bootstrap_servers: str = field(name="bootstrap.servers")
    retries: int = field(name="retries", default=5)

print(msgspec.structs.asdict(Example("some.server"))) # >> {'bootstrap_servers': 'some.server', 'retries': 5}
print(msgspec.json.encode(Example("some.server")))    # >> b'{"bootstrap.servers":"some.server"}'
print(msgspec.to_builtins(Example("some.server")))    # >> {'bootstrap.servers': 'some.server'}

Navigation Menu

Repository navigation

asdict(msgspec.Struct) doesn't apply Struct settings #817

Question

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

asdict(msgspec.Struct) doesn't apply Struct settings #817

Description

Question

Activity

jacopoabramo commented on Feb 18, 2025

idatsy commented on Feb 18, 2025

jacopoabramo commented on Feb 18, 2025

idatsy commented on Feb 18, 2025

jacopoabramo commented on Feb 18, 2025

idatsy commented on Feb 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions