Repository navigation
Description
This will depend on the fate of PEP 661.
Things to consider:
-
We probably should push down constraints, e.g.
int | MISSING = Field(gt=1)should be equivalent toAnnotated[int, Gt(1)] | MISSING. What aboutint | None | MISSING? Push down constraints in unions involvingMISSINGsentinel #12908 -
When validation fails for
<typ> | MISSING, we shouldn't mentionMISSINGas an allowed value in the validation error. This may require some changes in the core schema logic (currently this is a union, which shouldn't have special knowledge aboutMISSINGbeing one of the union members, so we probably need something similar to the'nullable'schema). Do not mentionMISSINGas a valid value in validation errors #13779 -
Related to previous point, should
MISSINGactually be allowed as a value? Is there a use case where explicitly allowingMISSING(only possible from Python, not JSON) makes sense? Shouldn't it be some kind of implementation detail? Final decision: we decided it should be allowed, as it would otherwise contradict static type checkers, and we can see how it can be useful (e.g. setting a field toMISSINGafter validation to exclude it).class Model(BaseModel): a: int | MISSING = MISSING model = Model(a=MISSING) # This currently works # As per the type checking rules for sentinels, this is a static type checker error: class Model(BaseModel): a: int = MISSING # .. if this was possible, then the sentinel is only relevant as a default value. Conceptually, it means # *not* having a default value is a logical error? class Model(BaseModel): a1: int # equivalent to?: a2: int | MISSING
Activity
pycaw commentedon Jul 30, 2025
Doesn't it make sense to make it falsy? As in, derive from
Sentinel, add,and use that as our
MISSING's class?As of yet though, pyright considers this kind of Sentinel as
UNKNOWNinstead ofMISSING.Viicos commentedon Jul 30, 2025
That's not for me to say, but up to the PEP author. There is some discussion about truthiness in the linked discussion thread (my personal opinion is that truthiness shouldn't be supported and
__bool__()should raise an error to avoid unexpected behavior).As mentioned in the documentation,
enableExperimentalFeaturesneeds to be set.pycaw commentedon Jul 30, 2025
I meant if you subclass Sentinel then it will break.
It seems to me it's a pretty good use case for falsiness but I will indulge in those discussions.
pycaw commentedon Sep 15, 2025
I noticed the behavior below.
I assume this is not intended. Why don't we have
patternas part ofClassWithMissing's schema as withClassWithNone?Removing the default values will produce the same outcome regarding
pattern's existence. And usingstr | None | MISSINGas type hint will also result in nullified constraints.bjmc commentedon Sep 16, 2025
I'm guessing this is a foreseeable limitation, but being unable to Pickle
MISSINGunfortunately also means you can't usemodel_copy(deep=True)on any models with MISSING fields:Is there any chance this could be fixed or worked around with special logic inside
model_copy()?Viicos commentedon Sep 16, 2025
Pydantic has some special casing on
<typ> | Nonetypes, where constraints applied usingAnnotatedto the whole union (and not just<typ>) are applied to<typ>directly. I don't like this special casing as it is inconsistent with the rest of the logic, and we then face the issue for theMISSINGsentinel that could behave the same. I'll discuss with the team shortly.@bjmc This will depend on the semantics of PEP 661 once it is submitted. There is some ongoing discussion about sentinels being pickleable in the discussion thread.
bjmc commentedon Sep 16, 2025
Even if it turns out that sentinels are non-pickleable, would it be possible for
model_copy(deep=True)to treatMISSINGas a special case singleton? (i.e. don't try to copy it) The fact that deepcopy relies on pickle machinery seems like an implementation detail callers shouldn't have to be concerned with.Viicos commentedon Sep 16, 2025
If sentinels end up being non-pickleable (I hope they will), we will find an alternative (may require not using PEP 661 sentinels) that will support it.
pycaw commentedon Sep 16, 2025
For the sake of syntactic brevity, not requiring in each cases where MISSING is used in conjunction with some constrained type to have that be wrapped in a RootModel would be appreciated. Road is already paved with None's treatment. The two would have better synergy as well it looks.
Viicos commentedon Sep 17, 2025
I fail to understand what you mean. Do you have an example?
pycaw commentedon Sep 17, 2025
I meant this:
But I don't know where I got the idea. Version
Bdoesn't preserve the constraint either actually.pycaw commentedon Sep 21, 2025
Is the example above not a gigantic bug with this feature? Should I open an issue about it?--constrwas used erroneously here -- And what is the currently advised way of applying constraints together withMISSING?Viicos commentedon Sep 21, 2025
You are not using
constr()the expected way. Thecon*functions are meant to be used directly as type annotations, this way:Usage of these functions are discouraged anyway (and we'll actually properly deprecate them in 2.12 -- it's a shame that we didn't properly remove them from the documentation). Instead, the
Field()function or theStringConstraintsmetadata class can be used:Regarding applying constraints to a union (instead of the specific member of the union), Pydantic has some special casing if the union is a
<typ> | Noneform. I'm not a fan of this behavior (see #11500 (comment) for an example), but we unfortunately can't deprecate this behavior as we noticed widespread usage from Github searches. We are going to document it, and most likely also apply this special casing for theMISSINGsentinel.20 remaining items
ari-bou commentedon Apr 27, 2026
Small update: PEP 661 has since been accepted and the implementation is slated for python 3.15.
Use pydantic `MISSING` sentinel to mark optional but non-nullable fields
Use pydantic `MISSING` sentinel to mark optional but non-nullable fields
Use pydantic `MISSING` sentinel to mark optional but non-nullable fields
lumpi101 commentedon Aug 29, 2026
I really like that feature but there is still one little thing I'd like to see changed:
I think
a: int | MISSINGshould suffice to declare values to be non-required, without= MISSING.MISSINGalready states "this field may be missing". Requiring= MISSINGon top means stating the same fact twice, through two different mechanisms (annotation and default).Semantically, the default on top is actually a no-op. A default replaces absence with a value; this one replaces absence with absence. It's technically not wrong (and should not lead to an error) but it should not be necessary.
I'd guess
int | MISSINGwithout a default would be more intuitive for people (at least it is for me). People who forget the= MISSINGpart will get an error at initialization and the error messageField required [type=missing, input_value={}, input_type=dict]is not really helpful IMHO.Viicos commentedon Aug 29, 2026
Not sure I agree: this goes strongly against type checking behavior for dataclass-like constructs such as Pydantic models. If no default value is present, the field is required. We moved away from this pattern with
Nonein V1 already (https://pydantic.dev/docs/validation/latest/get-started/migration/#required-optional-and-nullable-fields), I don't think it's a good idea to reintroduce it. Besides, adding the default value costs nothing to the user.We are most likely going to change the behavior before stabilization, as per my OP in this issue.
lumpi101 commentedon Aug 29, 2026
I see your point and for the
Nonecase I totally agree.Noneornullin json is an explicit value just like any other value in that sense and should not be assumed to be present in the data when it's not. But I thinkMISSINGis still another case because it denotes non-requiredness. It's its sole purpose (a non-required value is allowed to be missing).Yes, defaults used to be the means to denote that a field is non-required but with the downside that it is always replaced with a default value. It coupled the two concepts of "non-required values" and "default values".
Now
MISSINGsteps in to decouple these concepts, and one don't need a default anymore to represent a non-required field. An absent field in the data directly translates into the explicitMISSINGsingleton inside pydantic - no default value is needed here.I'm not sure I could make my point. But I guess there are different views possible on that topic.
Viicos commentedon Aug 30, 2026
I kind of get the point, related to my last bullet in OP:
If we went this way, with
f: int | MISSING, only allowingintwould mean it is equivalent (at least on the validation side) to justint. But I still don't think it's a good idea: what prevents users from defining the= MISSINGdefault, apart from saving a few keystrokes? How often are we going to have reports about users not specifying the default and complaining about "my static type checker raises an error even though Pydantic fills the default, why are you allowing this in the first place?"?I also think there could be a use case with
int | MISSINGwithout a default: the field is required (and -- depending on whether we allow explicitMISSING-- only anintcan be provided), but can later be set toMISSINGso that it's excluded from serialization. With your current proposal, this would no longer be possible.lumpi101 commentedon Aug 30, 2026
It's not quite what I mean. I think our misunderstanding comes from very different points of view. I guess you are very close to the implementation/code while I only looked at how a model definition with
MISSINGshould look like for me to make most sense. Not so much about saving keystrokes.What I did not see before was the point with the type checkers. If I got you right, they generally assume values to be required when they have no default defined, right? So static type checking would fail with my suggestion. And that would be bad. That's the point where you convinced me :)
So yes you are right, it would be a bad idea.
Viicos commentedon Aug 31, 2026
Yes so in:
and my point was that unless I'm missing something, users can just add the default to solve it:
Viicos commentedon Sep 9, 2026
#13779 updates the validation logic so that
MISSINGisn't reported as a possible value anymore (the runtime still allows explicitly passingMISSING, although it isn't the intended primary case).With that merged, we are ready to stabilize this feature for the next release.
lenard-mosys commentedon Sep 9, 2026
I'm not sure I like that. Does that mean that such a model can be used to serialize to a json document that does not validate with the same model or the corresponding json schema?
It would be nice if roundtripping a model instance through model_dump_json() -> model_validate_json() were safe by default, but maybe it is already broken in other ways.
Viicos commentedon Sep 9, 2026
In this particular case, it wouldn't roundtrip, but not setting a default
= MISSINGfor such fields is pretty contrived to begin with.