Skip to content

Stabilize MISSING sentinel feature #12090

Closed

Description

@Viicos

This will depend on the fate of PEP 661.

Things to consider:

  • We probably should push down constraints, e.g. int | MISSING = Field(gt=1) should be equivalent to Annotated[int, Gt(1)] | MISSING. What about int | None | MISSING? Push down constraints in unions involving MISSING sentinel #12908

  •  When validation fails for <typ> | MISSING, we shouldn't mention MISSING as an allowed value in the validation error. This may require some changes in the core schema logic (currently this is a union, which shouldn't have special knowledge about MISSING being one of the union members, so we probably need something similar to the 'nullable' schema). Do not mention MISSING as a valid value in validation errors #13779

  • Related to previous point, should MISSING actually be allowed as a value? Is there a use case where explicitly allowing MISSING (only possible from Python, not JSON) makes sense? Shouldn't it be some kind of implementation detail? Final decision: we decided it should be allowed, as it would otherwise contradict static type checkers, and we can see how it can be useful (e.g. setting a field to MISSING after validation to exclude it).

    class Model(BaseModel):
        a: int | MISSING = MISSING
    
    model = Model(a=MISSING)  # This currently works
    
    # As per the type checking rules for sentinels, this is a static type checker error:
    class Model(BaseModel):
        a: int = MISSING
    
    # .. if this was possible, then the sentinel is only relevant as a default value. Conceptually, it means
    # *not* having a default value is a logical error?
    class Model(BaseModel):
        a1: int
        # equivalent to?:
        a2: int | MISSING

Activity

  1. pycaw commented on Jul 30, 2025

    @pycaw

    Doesn't it make sense to make it falsy? As in, derive from Sentinel, add,

    def __bool__(self) -> Literal[False]:
        return False

    and use that as our MISSING's class?
    As of yet though, pyright considers this kind of Sentinel as UNKNOWN instead of MISSING.

  2. Viicos commented on Jul 30, 2025

    @Viicos
    MemberAuthor

    Doesn't it make sense to make it falsy?

    That's not for me to say, but up to the PEP author. There is some discussion about truthiness in the linked discussion thread (my personal opinion is that truthiness shouldn't be supported and __bool__() should raise an error to avoid unexpected behavior).

    As of yet though, pyright considers this kind of Sentinel as UNKNOWN instead of MISSING.

    As mentioned in the documentation, enableExperimentalFeatures needs to be set.

  3. pycaw commented on Jul 30, 2025

    @pycaw

    enableExperimentalFeatures needs to be set.

    I meant if you subclass Sentinel then it will break.

    It seems to me it's a pretty good use case for falsiness but I will indulge in those discussions.

  4. pycaw commented on Sep 15, 2025

    @pycaw

    I noticed the behavior below.

    I assume this is not intended. Why don't we have pattern as part of ClassWithMissing's schema as with ClassWithNone?

    Removing the default values will produce the same outcome regarding pattern's existence. And using str | None | MISSING as type hint will also result in nullified constraints.

    import json
    import warnings
    from typing import Annotated
    
    from pydantic import BaseModel, Field, PydanticExperimentalWarning
    
    warnings.filterwarnings("ignore", category=PydanticExperimentalWarning)
    
    from pydantic.experimental.missing_sentinel import MISSING  # noqa: E402
    
    
    class ClassWithMissing(BaseModel):
        value: Annotated[str | MISSING, Field(pattern=r"\d?")] = MISSING
    
    
    class ClassWithNone(BaseModel):
        value: Annotated[str | None, Field(pattern=r"\d?")] = None
    
    
    def test():
            assert (
            json.dumps(ClassWithMissing.model_json_schema(), indent=4)
            == """{
        "properties": {
            "value": {
                "title": "Value",
                "type": "string"
            }
        },
        "title": "ClassWithMissing",
        "type": "object"
    }"""
        )
        assert (
            json.dumps(ClassWithNone.model_json_schema(), indent=4)
            == r"""{
        "properties": {
            "value": {
                "anyOf": [
                    {
                        "pattern": "\\d?",
                        "type": "string"
                    },
                    {
                        "type": "null"
                    }
                ],
                "default": null,
                "title": "Value"
            }
        },
        "title": "ClassWithNone",
        "type": "object"
    }"""
        )
    
    test()
  5. bjmc commented on Sep 16, 2025

    @bjmc
    Contributor

    I'm guessing this is a foreseeable limitation, but being unable to Pickle MISSING unfortunately also means you can't use model_copy(deep=True) on any models with MISSING fields:

    from pydantic.experimental.missing_sentinel import MISSING
    from pydantic import BaseModel
    
    class ExampleModel(BaseModel):
        name: str | MISSING = MISSING
    
    
    unnamed = ExampleModel()
    # Shallow copy is okay
    shallow_copy = unnamed.model_copy(deep=False)  # ExampleModel(name=<MISSING>)
    # But deep copy relies on pickle:
    deep_copy = unnamed.model_copy(deep=True)
    # Traceback (most recent call last):
    #   File "/home/bjmc/Sandbox/example.py", line 12, in <module>
    #     deep_copy = unnamed.model_copy(deep=True)
    #   File "/home/bjmc/.cache/pypoetry/virtualenvs/zPbyUnCb-py3.10/lib/python3.10/site-packages/pydantic/main.py", line 406, in model_copy
    #     copied = self.__deepcopy__() if deep else self.__copy__()
    #   File "/home/bjmc/.cache/pypoetry/virtualenvs/zPbyUnCb-py3.10/lib/python3.10/site-packages/pydantic/main.py", line 941, in __deepcopy__
    #     _object_setattr(m, '__dict__', deepcopy(self.__dict__, memo=memo))
    #   File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
    #     y = copier(x, memo)
    #   File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    #     y[deepcopy(key, memo)] = deepcopy(value, memo)
    #   File "/usr/lib/python3.10/copy.py", line 161, in deepcopy
    #     rv = reductor(4)
    #   File "/home/bjmc/.cache/pypoetry/virtualenvs/zPbyUnCb-py3.10/lib/python3.10/site-packages/typing_extensions.py", line 4245, in __getstate__
    #     raise TypeError(f"Cannot pickle {type(self).__name__!r} object")
    # TypeError: Cannot pickle 'Sentinel' object

    Is there any chance this could be fixed or worked around with special logic inside model_copy()?

  6. Viicos commented on Sep 16, 2025

    @Viicos
    MemberAuthor

    I assume this is not intended. Why don't we have pattern as part of ClassWithMissing's schema as with ClassWithNone?

    Removing the default values will produce the same outcome regarding pattern's existence. And using str | None | MISSING as type hint will also result in nullified constraints.

    Pydantic has some special casing on <typ> | None types, where constraints applied using Annotated to the whole union (and not just <typ>) are applied to <typ> directly. I don't like this special casing as it is inconsistent with the rest of the logic, and we then face the issue for the MISSING sentinel that could behave the same. I'll discuss with the team shortly.


    I'm guessing this is a foreseeable limitation, but being unable to Pickle MISSING unfortunately also means you can't use model_copy(deep=True) on any models with MISSING fields:

    @bjmc This will depend on the semantics of PEP 661 once it is submitted. There is some ongoing discussion about sentinels being pickleable in the discussion thread.

  7. bjmc commented on Sep 16, 2025

    @bjmc
    Contributor

    Even if it turns out that sentinels are non-pickleable, would it be possible for model_copy(deep=True) to treat MISSING as a special case singleton? (i.e. don't try to copy it) The fact that deepcopy relies on pickle machinery seems like an implementation detail callers shouldn't have to be concerned with.

  8. Viicos commented on Sep 16, 2025

    @Viicos
    MemberAuthor

    If sentinels end up being non-pickleable (I hope they will), we will find an alternative (may require not using PEP 661 sentinels) that will support it.

  9. pycaw commented on Sep 16, 2025

    @pycaw

    For the sake of syntactic brevity, not requiring in each cases where MISSING is used in conjunction with some constrained type to have that be wrapped in a RootModel would be appreciated. Road is already paved with None's treatment. The two would have better synergy as well it looks.

  10. Viicos commented on Sep 17, 2025

    @Viicos
    MemberAuthor

    For the sake of syntactic brevity, not requiring in each cases where MISSING is used in conjunction with some constrained type to have that be wrapped in a RootModel would be appreciated.

    I fail to understand what you mean. Do you have an example?

  11. pycaw commented on Sep 17, 2025

    @pycaw

    For the sake of syntactic brevity, not requiring in each cases where MISSING is used in conjunction with some constrained type to have that be wrapped in a RootModel would be appreciated.

    I fail to understand what you mean. Do you have an example?

    I meant this:

    class A(BaseModel):
        value: Annotated[str | MISSING, constr(min_length=1)] = MISSING
    
    
    class Value(RootModel):
        root: Annotated[str, constr(min_length=1)]
    
    
    class B(BaseModel):
        value: Value | MISSING = MISSING
    
    
    print(json.dumps(A.model_json_schema(), indent=4))
    print(json.dumps(B.model_json_schema(), indent=4))

    But I don't know where I got the idea. Version B doesn't preserve the constraint either actually.

  12. pycaw commented on Sep 21, 2025

    @pycaw

    Is the example above not a gigantic bug with this feature? Should I open an issue about it? -- constr was used erroneously here -- And what is the currently advised way of applying constraints together with MISSING?

  13. Viicos commented on Sep 21, 2025

    @Viicos
    MemberAuthor

    You are not using constr() the expected way. The con* functions are meant to be used directly as type annotations, this way:

    class Model(BaseModel):
        a: constr(min_length=1)

    Usage of these functions are discouraged anyway (and we'll actually properly deprecate them in 2.12 -- it's a shame that we didn't properly remove them from the documentation). Instead, the Field() function or the StringConstraints metadata class can be used:

    class Model(BaseModel):
        a: Annotated[str, StringConstraints(min_length=1)]

    Regarding applying constraints to a union (instead of the specific member of the union), Pydantic has some special casing if the union is a <typ> | None form. I'm not a fan of this behavior (see #11500 (comment) for an example), but we unfortunately can't deprecate this behavior as we noticed widespread usage from Github searches. We are going to document it, and most likely also apply this special casing for the MISSING sentinel.

  14. 20 remaining items

  15. ari-bou commented on Apr 27, 2026

    @ari-bou

    Can this Pydantic feature get reworked so that it does not depend on PEP 661? PEP 661 is deferred.

    Small update: PEP 661 has since been accepted and the implementation is slated for python 3.15.

  16. lumpi101 commented on Aug 29, 2026

    @lumpi101

    I really like that feature but there is still one little thing I'd like to see changed:

    # works
    class Foo(BaseModel):
        a: int | MISSING = MISSING
     
    # IMHO should work, too but fails on initialization without `a`
    class Bar(BaseModel):
        a: int | MISSING

    I think a: int | MISSING should suffice to declare values to be non-required, without = MISSING.
    MISSING already states "this field may be missing". Requiring = MISSING on top means stating the same fact twice, through two different mechanisms (annotation and default).
    Semantically, the default on top is actually a no-op. A default replaces absence with a value; this one replaces absence with absence. It's technically not wrong (and should not lead to an error) but it should not be necessary.
    I'd guess int | MISSING without a default would be more intuitive for people (at least it is for me). People who forget the = MISSING part will get an error at initialization and the error message Field required [type=missing, input_value={}, input_type=dict] is not really helpful IMHO.

  17. Viicos commented on Aug 29, 2026

    @Viicos
    MemberAuthor

    I think a: int | MISSING should suffice to declare values to be non-required, without = MISSING.

    Not sure I agree: this goes strongly against type checking behavior for dataclass-like constructs such as Pydantic models. If no default value is present, the field is required. We moved away from this pattern with None in V1 already (https://pydantic.dev/docs/validation/latest/get-started/migration/#required-optional-and-nullable-fields), I don't think it's a good idea to reintroduce it. Besides, adding the default value costs nothing to the user.

    People who forget the = MISSING part will get an error at initialization and the error message Field required [type=missing, input_value={}, input_type=dict] is not really helpful IMHO.

    We are most likely going to change the behavior before stabilization, as per my OP in this issue.

  18. lumpi101 commented on Aug 29, 2026

    @lumpi101

    I see your point and for the None case I totally agree. None or null in json is an explicit value just like any other value in that sense and should not be assumed to be present in the data when it's not. But I think MISSING is still another case because it denotes non-requiredness. It's its sole purpose (a non-required value is allowed to be missing).

    If no default value is present, the field is required.

    Yes, defaults used to be the means to denote that a field is non-required but with the downside that it is always replaced with a default value. It coupled the two concepts of "non-required values" and "default values".
    Now MISSING steps in to decouple these concepts, and one don't need a default anymore to represent a non-required field. An absent field in the data directly translates into the explicit MISSING singleton inside pydantic - no default value is needed here.

    I'm not sure I could make my point. But I guess there are different views possible on that topic.

  19. Viicos commented on Aug 30, 2026

    @Viicos
    MemberAuthor

    I kind of get the point, related to my last bullet in OP:

    Related to previous point, should MISSING actually be allowed as a value? Is there a use case where explicitly allowing MISSING (only possible from Python, not JSON) makes sense? Shouldn't it be some kind of implementation detail?

    If we went this way, with f: int | MISSING, only allowing int would mean it is equivalent (at least on the validation side) to just int. But I still don't think it's a good idea: what prevents users from defining the = MISSING default, apart from saving a few keystrokes? How often are we going to have reports about users not specifying the default and complaining about "my static type checker raises an error even though Pydantic fills the default, why are you allowing this in the first place?"?

    I also think there could be a use case with int | MISSING without a default: the field is required (and -- depending on whether we allow explicit MISSING -- only an int can be provided), but can later be set to MISSING so that it's excluded from serialization. With your current proposal, this would no longer be possible.

  20. lumpi101 commented on Aug 30, 2026

    @lumpi101

    It's not quite what I mean. I think our misunderstanding comes from very different points of view. I guess you are very close to the implementation/code while I only looked at how a model definition with MISSING should look like for me to make most sense. Not so much about saving keystrokes.

    What I did not see before was the point with the type checkers. If I got you right, they generally assume values to be required when they have no default defined, right? So static type checking would fail with my suggestion. And that would be bad. That's the point where you convinced me :)
    So yes you are right, it would be a bad idea.

  21. Viicos commented on Aug 31, 2026

    @Viicos
    MemberAuthor

    Yes so in:

    class Model(BaseModel):
        a: int | MISSING
    
    Model()  # type checker error: a is required

    and my point was that unless I'm missing something, users can just add the default to solve it:

    class Model(BaseModel):
        a: int | MISSING = MISSING  # equivalent according to your proposal
  22. Viicos commented on Sep 9, 2026

    @Viicos
    MemberAuthor

    #13779 updates the validation logic so that MISSING isn't reported as a possible value anymore (the runtime still allows explicitly passing MISSING, although it isn't the intended primary case).

    With that merged, we are ready to stabilize this feature for the next release.

  23. lenard-mosys commented on Sep 9, 2026

    @lenard-mosys

    I also think there could be a use case with int | MISSING without a default: the field is required (and -- depending on whether we allow explicit MISSING -- only an int can be provided), but can later be set to MISSING so that it's excluded from serialization. With your current proposal, this would no longer be possible.

    I'm not sure I like that. Does that mean that such a model can be used to serialize to a json document that does not validate with the same model or the corresponding json schema?

    It would be nice if roundtripping a model instance through model_dump_json() -> model_validate_json() were safe by default, but maybe it is already broken in other ways.

  24. Viicos commented on Sep 9, 2026

    @Viicos
    MemberAuthor

    I'm not sure I like that. Does that mean that such a model can be used to serialize to a json document that does not validate with the same model or the corresponding json schema?

    It would be nice if roundtripping a model instance through model_dump_json() -> model_validate_json() were safe by default, but maybe it is already broken in other ways.

    In this particular case, it wouldn't roundtrip, but not setting a default = MISSING for such fields is pretty contrived to begin with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    deferredDeferred until future release or until something else gets donefeature request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions