Constraining a TypeVar to be not None seems impossible.

First of all, I'm using Mypy 0.770.

I'd like to annotate a function in the following way:

T = TypeVar('T')

class JSONSerialized(str, Generic[T]):
    """Strong type for the JSON serialized version of a type."""

@overload
def json_serialize(value: None) -> None:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: T) -> JSONSerialized[T]:
    """Serialize value to its strong-typed JSON string type."""

def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""
    if value is None:
        return None

    return cast(JSONSerialized[T], json.dumps(to_json(value)))

The idea here is that I know that a string has been serialized from a given type (converted to a json-compatible type by the to_json function which is not depicted here.

The overloads don't work, however. I'd like them to inform mypy that this only returns None if None goes in, otherwise it doesn't. Unfortunately, since T is an unconstrained TypeVar, it also matches None, and mypy gives me the following error:

error: Overloaded function signatures 1 and 2 overlap with incompatible return types

Thus, I need a way to say "anything but None" (as a TypeVar in this case), but I can't seem to find any way to do so, so I believe this may end up being a feature request. I've seen similar behaviour being asked about in StackOverflow:

https://stackoverflow.com/questions/57854871/exclude-type-in-python-typing-annotation
https://stackoverflow.com/questions/47215682/what-type-represents-typing-any-except-none?noredirect=1&lq=1
https://stackoverflow.com/questions/58612057/how-to-make-a-generic-typevar-in-mypy-be-non-optional?noredirect=1&lq=1

Also, I'm not using any flags that change mypy's behaviour about None.

+1. I think introducing a NotNone type is the way to go here.

In terms of implementation, I'd imagine NotNone would become the primary base type (except for None), and object would be treated as Union[NotNone, None] (aka Optional[NotNone]). Note that this is also how Kotlin is designed, where Any is the superclass of everything except Null, and the true top-type is Any? (aka Any | Null).

If that existed, the example above could be rewritten like this:

TNotNone = TypeVar('TNotNone', bound=NotNone)
T = TypeVar('T')

class JSONSerialized(str, Generic[TNotNone]):
    """Strong type for the JSON serialized version of a type."""

@overload
def json_serialize(value: None) -> None:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: TNotNone) -> JSONSerialized[TNotNone]:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""

def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""
    if value is None:
        return None

    return cast(JSONSerialized[T], json.dumps(to_json(value)))

(The third overload is needed for calling with a value of type object, which doesn't match None or TNotNone. In that case, T would be inferred as NotNone.)

@russelldavis

@russelldavis I don't think NotNone solves the general problem, which seems to be constraining a TypeVar that's bound to a Union.

Would have to be something along the lines of

class A:
    pass

class B(A):
    pass

class C(A):
    pass

class D(A):
    pass

class E(A):
    pass

class F(E):
    pass

_TANotDOrE = TypeVar("_TANotDOrE", bound=A, exclude=(D, E))

Where _TANotDOrE is essentially equivalent to TypeVar("_TANotDOrE", A, B, C).

Class def updates

Has the best solution been invented for this case? I have just run into the same 'unconstrained TypeVar contains None' pitfall.

Allow None vs TypeVar overlap for overloads (#15846)

Should probably link the addition of Not as the ideal solution: python/typing#801

Navigation Menu

Repository navigation

Constraining a TypeVar to be not None seems impossible. #8881

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Constraining a TypeVar to be not None seems impossible. #8881

Description

Activity

russelldavis commented on May 25, 2020

NeilGirdhar commented on Sep 6, 2020

relsunkaev commented on Feb 2, 2022

johnslavik commented on Jul 5, 2023

NeilGirdhar commented on Aug 14, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions