I am not sure if this is a mypy issue or is a limitation of PEP 586. I want to use boolean literals to forbid a specific combination of function parameter types and boolean flag values. In my example data is AnyStr, but if data_must_be_bytes=True it must be bytes. (In my real use case data_must_be_bytes is actually data_is_image; and I want to allow either bytes for images, or AnyStr for non images)

As indicated in PEP 586 and mypy docs, I need to add a fallback to allow dealing with the original bool type. However, this fallback will also allow the forbidden combination where data is str and data_must_be_bytes=True

Assuming that the the necessity of the fallback is out of this discussion's scope, I guess I either need a way explicitly state in the fallback that data_must_be_bytes is bool except Literal[True], or have mypy not do the fallback from Literal[True] to bool when a narrower overload is given. Without a way to overcome this issue, there seems to be no benefit in using overloading and literals here, over simpler type annotations. It would also be useful to have the docs call out such cases where the added complexity is redundant.

from typing import overload, Literal, AnyStr


@overload
def f(data: bytes, data_must_be_bytes: Literal[True]) -> int: ...


# redundant due to following fallback:
#@overload
#def f(data: AnyStr, data_must_be_bytes: Literal[False]) -> int: ...


# necessary fallback for non literals:
@overload
def f(data: AnyStr, data_must_be_bytes: bool) -> int: ...


def f(data, data_must_be_bytes):
    return 0

f(data='string', data_must_be_bytes=False)
f(data=b'bytes', data_must_be_bytes=False)
f(data='string', data_must_be_bytes=True) # this passes but i want it to fail
f(data=b'bytes', data_must_be_bytes=True)
f(data='string', data_must_be_bytes=bool(0))
f(data=b'bytes', data_must_be_bytes=bool(0))
f(data='string', data_must_be_bytes=bool(1))
f(data=b'bytes', data_must_be_bytes=bool(1))

mypy-play