Type-narrowing based on x in y

[Azureblade3808]

Converted from discussion (#9337).

---

We can now have following snippet pass type-checking -

from typing_extensions import assert_type

def foo(x: float = 0.0, y: list[int] = [0]):
    if x in y:
        _ = assert_type(x, "int")  # !!!

The type-narrowing of x can be a false negative, as the default value 0.0 of x is obviously not an instance of int.

On the other hand, type-narrowing based on x == L seems to work soundly -

from typing_extensions import Literal, assert_type

def foo(x0: float = 0.0, x1: int = 0, L: Literal[0] = 0):
    if x0 == L:
        assert_type(x0, "float")  # No type-narrowing.
    if x1 == L:
        assert_type(x1, "Literal[0]")  # Safe.

My suggestion is that type-narrowing based of x in y only take effect when the element type of y is a literal type (or maybe a union of literal types that shares a same runtime type) and type of x is the runtime type of y(or some related union types).

My expected behavior would be like following examples -

from typing_extensions import Literal, assert_type

def f0(x: int, y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0, 1]")  # Narrowed.

def f1(x: Literal[-1, 0], y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0]")  # Narrowed.

def f2(x: Literal[0, 1, 2], y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0, 1]")  # Narrowed.

def f3(x: float, y: list[int]):
    if x in y:
        _ = assert_type(x, "float")  # Not narrowed, because `int` is not a literal type or a union of literal types.

def f4(x: float, y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "float")  # Not narrowed, because `float` is not `int`.

def f5(x: int, y: list[Literal[0, True]]):
    if x in y:
        _ = assert_type(x, "int")  # Not narrowed, because `Literal[0]` and `Literal[True]` don't share a same runtime type.

def f6(x: bool, y: list[Literal[0, 1]]):
    if x in y:  # Assuming this is allowed.
        _ = assert_type(x, "bool")  # Not narrowed, because `bool` is not `int`.

[tusharsadhwani]

May be related, but I'm trying to do something like this:

import typing


class MyType(typing.TypedDict):
    a: int
    b: typing.NotRequired[int]


MyEnum = typing.Literal["a", "b"]


def foo(t1: MyType, t2: MyType) -> None:
    for key in t1.keys():
        if key not in t2:
            continue

        key = typing.cast(MyEnum, key)
        print(t2[key])

is it possible to do something like this currently, where pyright doesn't raise an issue?

[JodhwaniMadhur]

I want to contribute to this even though I am here for the first time. I hope that is fine.

[tusharsadhwani]

This is probably a design change, which may or may not be wanted depending on the spec, what other checkers do etc., so I'd suggest wait for confirmation from maintainers. MyPy also doesn't do this right now for example, but it has been on the roadmap for a long time:

• Narrowing types to Literal using in syntax python/mypy#12535
• Infer type from in container check python/mypy#10977

[erictraut]

@tusharsadhwani, the behavior you're seeing is not related to this issue. The OP has identified a bug in the x in y type guard form. Your code uses something closer to the S in D type guard form, although it doesn't quite match that because S must be a string literal for this form to apply. In any event, these are different cases. See this documentation for a list of supported type guard forms. Pyright is working as intended in your case. If you have questions about this, feel free to open a new discussion topic.

[wyattscarpenter]

I've just run into this myself today, I think.

Code sample in pyright playground

from typing import assert_type

def g(a: int | None):
    if a is not None:
        assert_type(a, int) #works fine

def f(a: int | None):
    if a not in [None]:
        assert_type(a, int) # "assert_type" mismatch: expected "int" but received "int | None"  (reportAssertTypeFailure)

[erictraut]

@wyattscarpenter, that's not related to this issue. Type narrowing in the negative case in your code sample will not eliminate None from the union. That would require extra special-casing for None. I generally don't add special cases like this unless there's a strong signal from pyright users that it's a common use case. You'll need to find a workaround if you want this to type check without errors.

[wyattscarpenter]

Ah, thanks!

[Sxderp]

I believe I recently ran into this. From the playground.

from typing import Literal

def f(a: Literal['AND', '&', 'OR', '|']):
    if a in {'AND', '&'}:
        reveal_type(a) # Type of "a" is "Literal['AND', '&', 'OR', '|']" # Seems wrong
    if a in {'OR', '|'}:
        reveal_type(a) # Type of "a" is "Literal['AND', '&', 'OR', '|']" # Seem wrong

However, if I change from set to tuple it works.

from typing import Literal

def f(a: Literal['AND', '&', 'OR', '|']):
    if a in ('AND', '&'):
        reveal_type(a) # Type of "a" is "Literal['AND', '&']"
    if a in ('OR', '|'):
        reveal_type(a) # Type of "a" is "Literal['OR', '|']"

[erictraut]

@Sxderp, the behavior you're seeing here is expected. Pyright's inference behaviors for set, list and dict expressions do not retain literals because these types are mutable. Tuples are immutable, so pyright retains literals (in most cases). For details, refer to the pyright documentation here and here.

[Sxderp]

Maybe the documentation for type-guards should be clarified or a link should be added to the type-inference page?

https://github.com/microsoft/pyright/blob/main/docs/type-concepts-advanced.md

x in y or x not in y (where y is instance of list, set, frozenset, deque, tuple, dict, defaultdict, or OrderedDict)

This is the line for my source of confusion.