[Dr_Birdbrain]

I actually don’t like python type hints!

At my work we have a jit compiler that requires type hints under some conditions.

Aside from that, I avoid them as much as possible. The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.

For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.

So, if the jit compiler needs the annotation I am happy to provide it, but otherwise I will proactively not provide any, and I will sometimes even delete existing annotations when they are devolving into silliness.

[9dev]

That's the same complaints people had about TypeScript in the beginning, when libraries such as Express used to accept a wide range of input options that would be a pain to express in types properly. If you look at where the ecosystem is now, though, you'll see proper type stubs, and most libraries get written in TS in the first place anyway. When editing TS code, you get auto-completion out of the box, even for deeply nested properties or conditional types. You can rely on types being what the compiler says they are, and runtime errors are a rarity now (in properly maintained code bases).

> The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.

I'll admit that this is what I hate Python, and it's probably this spirit of the language as you call it. I never really know what parameters a function takes. Library documentation often shows a few use cases, but doesn't really provide a reference; so I end up having to dig into the source code to figure it out on my own. Untyped and undocumented kwargs? Everywhere. I don't understand how someone could embrace so much flexibility that it becomes entirely undiscoverable for anyone but maintainers.

[zelphirkalt]

Because the flexibility has been a boon and not a problem. The problem only comes when you try to express everything in the type system, that is third party (the type checkers for it) and added on top.

[Izkata]

Except Typescript embraces duck typing. You can say "accept any object with a quack() method", for example, and it'll accept an unexpected quacking parrot. It can even tell when two type definitions are close enough and merge them.

[tdeck]

I like Python a lot, and have been using it for personal projects since about 2010. It was only once I started working and encountering long-lived unfamiliar Python codebases regularly that I understood the benefits of type hints. It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something. It's even less fun to find out that someone made a mistake and it's actually two different incompatible things depending on the execution path.

That era of Python codebases were miserable to work in, and often ended up in the poorly though out "we don't know how this works and it has too many bugs, let's just rewrite it" category.

[rlpb]

> It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something.

My position is that what is intended must be made clear between type hints and the docstring. Skipping this makes for difficult to read code and has no place in a professional setting in any non-trivial codebase.

This doesn't require type hints to achieve. :param and :rtype in the docstring are fine if type hints aren't present, or for complex cases, plain English in the docstring is usually better.

[boltzmann64]

I am sorry, but whats wrong with doing something like, `print(type(var)); exit()` and just running it once instead of digging through 5-6 stack frames?

[kketch]

They don’t violate the spirit of the language. They are optional. They don’t change the behaviour at runtime.

Type annotations can seem pointless indeed if you are unwilling to learn how to use them properly. Using a giant union to type your (generic) function is indeed silly, you just have to make that function generic as explained in another comment or I guess remove the type hints

[pansa2]

> They don’t violate the spirit of the language. They are optional.

That in itself violates the spirit of the language, IMO. “There should be one obvious way to do it”.

[yladiz]

Actually in Python it can. Since the type hints are accessible at runtime, library authors can for example change which values in kwargs are allowed based on the type of the argument.

So on the language level it doesn’t directly change the behavior, but it is possible to use the types to affect the way code works, which is unintuitive. I think it was a bad decision to allow this, and Python should have opted for a TypeScript style approach.

[zelphirkalt]

Yeah, but then you get into the issues with when and where generic types are bound and narrowed, which can then make it more complicated, at which point one might be better off stepping back, redesigning, or letting go of perfect type hint coverage, for dynamic constructs, that one couldn't even write in another type safe language.

[sevensor]

I don’t know anything about your jit compiler, but generally the value I get from type annotations has nothing to do with what they do at runtime. People get so confused about Python’s type annotations because they resemble type declarations in languages like C++ or Java. For the latter, types tell the compiler how to look up fields on, and methods that apply to, an object. Python is fine without that.

Python’s types are machine-checkable constraints on the behavior of your code.. Failing the type checker isn’t fatal, it just means you couldn’t express what you were doing in terms it could understand. Although this might mean you need to reconsider your decisions, it could just as well mean you’re doing something perfectly legitimate and the type checker doesn’t understand it. Poke a hole in the type checker using Any and go on with your day. To your example, there are several ways described in comments by me and others to write a succinct annotation, and this will catch cases where somebody tries to use a dict keyed with strings or something.

Anyway, you don’t have to burn a lot of mental energy on them, they cost next to nothing at runtime, they help document your function signatures, and they help flag inconsistent assumptions in your codebase even if they’re not airtight. What’s not to like?

[TZubiri]

So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence, similar to Iterable.

>from typing import Sequence

>def third(something: Sequence):

> return indexable[3]

however if all you are doing is just iterate over the thing, what you actually need is an Iterable

>from typing import Iterable

>def average(something:Iterable):

> for thing in something:

> ...

Statistically, the odds of a language being wrong, are much lower than the programmer being wrong. Not to say that there aren't valid critiques of python, but we must think of the creators of programming languages and their creations as the top of the field. If a 1400 chess elo player criticizes Magnus Carlsen's chess theory, it's more likely that the player is missing some theory rather than he found a hole in Carlsen's game, the player is better served by approaching a problem with the mentality that he is the problem, rather than the master.

[pansa2]

> we must think of the creators of programming languages and their creations as the top of the field

The people at the top of the type-system-design field aren’t working on Python.

[dragonwriter]

> So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence

Sequence involves more than just __getitem__ with an int index, so if it really is anything int indexable, a lighter protocol with just that method will be more accurate, both ar conveying intent and at avoiding needing to evolve into an odd union type because you have something that a satisfies the function’s needs but not the originally-defined type.

[bgwalter]

That is sort of ironic because the Pythonistas did not leave out any opportunity to criticize Java. Java was developed by world class experts like Gosling and attracted other type experts like Philip Wadler.

No world class expert is going to contribute to Python after 2020 anyway, since the slanderous and libelous behavior of the Steering Council and the selective curation of allowed information on PSF infrastructure makes the professional and reputational risk too high. Apart from the fact that Python is not an interesting language for language experts.

Google and Microsoft have already shut down several failed projects.

[coldtea]

>you better believe that every single admissible type will eventually be fed to this function

That's your problem right there. Why are random callers sending whatever different input types to that function?

That said, there are a few existing ways to define that property as a type, why not a protocol type "Indexable"?

[knome]

>why not a protocol type

it was a sin that python's type system was initially released as a nominal type system. they should have been the target from day one.

being unable to just say "this takes anything that you can call .hello() and .world() on" was ridiculous, as that was part of the ethos of the dynamically typed python ecosystem. typechecking was generally frowned upon, with the idea that you should accept anything that fit the shape the receiving code required. it allowed you to trivially create resource wrappers and change behaviors by providing alternate objects to existing mechanisms. if you wanted to provide a fake file that read from memory instead of an actual file, it was simple and correct.

the lack of protocols made hell of these patterns for years.

[oivey]

> That's your problem right there. Why are random callers sending whatever different input types to that function?

Because it’s nice to reuse code. I’m not sure why anyone would think this is a design issue, especially in a language like Python where structural subtyping (duck typing) is the norm. If I wanted inheritance soup, I’d write Java.

Ironically, that’s support for structural subtyping is why Protocols exist. It’s too bad they aren’t better and the primary way to type Python code. It’s also too bad that TypedDict actively fought duck typing for years.

[david422]

Yes. It's not the type system that's broken, it's the design. Fix the design, and the type system works for you, not against you.

[dragonwriter]

> Why are random callers sending whatever different input types to that function?

Probably because the actual type it takes is well-understood (and maybe even documented in informal terms) by the people making and using it, but they just don’t understand how to express it in the Python type system.

[rtpg]

> For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.

You are looking for protocols. A bit futzy to write once but for a heavily trafficked function it's woth it.

If your JIT compiler doesn't work well with protocols... sounds like a JIT problem not a Python typing problem

[zenkey]

In my experience, the right tooling makes Python typing a big win. Modern IDEs give comprehensive real-time feedback on type errors, which is a big productivity boost and helps catch subtle bugs early (still nowhere near Rust, but valuable nonetheless). Push it too far though, and you end up with monsters like Callable[[Callable[P, Awaitable[T]]], TaskFunction[P, T]]. The art is knowing when to sprinkle types just enough to add clarity without clutter.

[ddavis]

When you hit types like that type aliases come to the rescue; a type alias combined with a good docstring where the alias is used goes a long way

[fyrn_]

You can use a Protocol type for that, makes a lot mote sense than nominal typing for typing use case.

[vjerancrnjak]

Exactly, sounds like misuse of unions.

Although Python type hints are not expressive enough.

[j1elo]

No idea about Python type system, but doesn't it have anything like this?

  interface IntIndexable {
    [key: number]: any
  }

[sdeframond]

It does!

You can specify a protocol like this:

  class IntIndexable(Protocol[T]):
    def __getitem__(self, index: int, /) -> T: ...

(Edit: formatting)

[jonas21]

> eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.

No, this is the great thing about gradual typing! You can use it to catch errors and provide IDE assistance in the 90% of cases where things have well-defined types, and then turn it off in the remaining 10% where it gets in the way.

[oivey]

I feel pretty similarly on this. Python’s bolted on type system is very poor at encoding safe invariants common in the language. It’s a straight jacketed, Java-style OOP type system that’s a poor fit for many common Python patterns.

I would love it if it were better designed. It’s a real downer that you can’t check lots of Pythonic, concise code using it.

[unethical_ban]

It sounds like that function is rightfully eligible to be ignored or to use the Any designation. To me that's why the system is handy. For functions that have specific inputs and outputs, it helps developers keep things straight and document code.

For broad things, write Any or skip it.

[thr0w4w4y1337]

from typing import Protocol, TypeVar

T_co = TypeVar("T_co", covariant=True)

class Indexable(Protocol[T_co]): def __getitem__(self, i: int) -> T_co: ...

def f(x: Indexable[str]) -> None: print(x[0])

I am failing to format it proprely here, but you get the idea.

[whilenot-dev]

Just fyi: https://news.ycombinator.com/formatdoc

> Text after a blank line that is indented by two or more spaces is reproduced verbatim. (This is intended for code.)

If you'd want monospace you should indent the snippet with two or more spaces:

  from typing import Protocol, TypeVar
  
  T_co = TypeVar("T_co", covariant=True)
  
  class Indexable(Protocol[T_co]):
    def __getitem__(self, i: int) -> T_co: ...
  
  def f(x: Indexable[str]) -> None:
    print(x[0])

[amluto]

I give Rust a lot of points for putting control over covariance into the language without making anyone remember which one is covariance and which one is contravariance.

[matusp]

There is also bunch of prepackaged types, such as collections.abc.Sequence that could be used in this case.

[dragonwriter]

As of Python 3.12, you don’t need separately declared TypeVars with explicit variance specifications, you can use the improved generic type parameter syntax and variance inference.

So, just:

  class Indexable[T](Protocol):
    def __getitem__(self, i: int,/) -> T: ... 

is enough.

[fragmede]

> something that is indexable by ints. > ...Dict[int, Any]...

If that is exactly what you want, then define a Protocol: from __future__ import annotations from typing import Protocol, TypeVar

    T = TypeVar("T")
    K = TypeVar("K")
    
    class GetItem(Protocol[K, T]):
        def __getitem__(self, key: K, /) -> T: ...
    
    
    def first(xs: GetItem[int, T]) -> T:
        return xs[0]

Then you can call "first" with a list or a tuple or a numpy array, but it will fail if you give it a dict. There is also collections.abc.Sequence, which is a type that has .__getitem__(int), .__getitem__(slice), .__len__ and is iterable. There are a couple of other useful ones in collections.abc as well, including Mapping (which you can use to do Mapping[int, t], which may be of interest to you), Reversible, Callable, Sized, and Iterable.

[mhitza]

Sounds like the ecosystem needs an "indexable" type annotation. Make it an "indexable<int>" for good measure.

[theptip]

Right, this was my thought.

Can’t you just use a typing.Protocol on __getitem__ here?

https://typing.python.org/en/latest/spec/protocol.html

Something like

    from typing import Protocol

    class Indexable(Protocol):
        def __getitem__(self, i: int) -> Self: ...

Though maybe numpy slicing needs a bit more work to support

[chlorion]

>The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types!

That's not how you are supposed to use static typing? Python has "protocols" that allows for structural type checking which is intended for this exact problem.

[burnerRhodov2]

You explained some hyper niche instance where type hints should be ignored. 99% of the time, they are extremely helpful.

[chlorion]

It's not even a niche instance, protocols solve their problem lol.

[detaro]

Can't you define your own hint for "type that has __getitem__ taking int"?

[crabbone]

The way I understand parent is that such a type would be too broad.

The bigger problem is that the type system expressed through hints in Python is not the type system Python is actually using. It's not even an approximation. You can express in the hint type system things that are nonsense in Python and write Python that is nonsense in the type system implied by hints.

The type system introduced through typing package and the hints is a tribute to the stupid fashion. But, also, there is no syntax and no formal definitions to describe Python's actual type system. Nor do I think it's a very good system, not to the point that it would be useful to formalize and study.

In Russian, there's an expression "like a saddle on a cow", I'm not sure what the equivalent in English would be. This describes a situation where someone is desperately trying to add a desirable feature to an exiting product that ultimately is not compatible with such a feature. This, in my mind, is the best description of the relationship between Python's actual type system and the one from typing package.

[humdaanm]

the issue of having multiple inputs able to be indexable by ints, is exactly why i prefer that type hints remain exactly as "hints" and not as mandated checks. my philosophy for type hints is that they are meant to make codebases easier to understand without getting into a debugger. their functional equivalence should be that of comments. it's a cleaner more concise way of describing a variable instead of using a full on docstring.

though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type(i.e. an argument can be any indexable type, except a string)

[true_blue]

>though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type

I think this is called a negation type, and it acts like a logical NOT operator. I'd like it too, and I hear that it works well with union types (logical OR) and intersection types (logical AND) for specifying types precisely in a readable way.

[albrewer]

Define a protocol[0] that declares it implements `__getitem__` and type annotate with that protocol. Whatever properties are needed inside the function can be described in other protocols.

These are similar to interfaces in C# or traits in Rust - you describe what the parameter _does_ instead of what it _is_.

[0]: https://typing.python.org/en/latest/spec/protocol.html

[the__alchemist]

I like your point! I think the advantage in its light is this: People often use Python because it's convention in the domain, the project already uses it, or it's the language the rest of the team uses. So, you are perhaps violating the spirit, but that's OK. You are making the most of tools available. It's not the Platonic (Pythonic??) ideal, but good enough.

[bobbiechen]

Isn't this supported by typing.SupportsIndex? https://docs.python.org/3/library/typing.html#typing.Support...

Mind you, I haven't used it before, but it feels very similar to the abstract Mapping types.

[detaro]

__index__ is not what you think it is

[nurettin]

Why not do Indexable = Any and pass that? Even if it doesn't help your jit or the IDE, at least it is more explicit than

    def lol(blarg):  # types? haha you wish. rtfc you poor sod. Pytharn spirit ftw!!!
        ...
        return omg[0].wtf["lol freedom"].pwned(Good.LUCK).figuring * out

[bartread]

> The reason is that they are not really a part of the language, they violate the spirit of the language

This is a good way of expressing my own frustration with bolting strong typing on languages that were never designed to have it. I hate that TypeScript has won out over JavaScript because of this - it’s ugly, clumsy, and boilerplatey - and I’d be even more disappointed to see the same thing happen to the likes of Python and Ruby.

My background is in strongly typed languages - first C++, then Java, and C# - so I don’t hate them or anything, but nowadays I’ve come to prefer languages that are more sparing and expressive with their syntax.

[tickettotranai]

I mean, you can just... Not annotate something if creating the relevant type is a pain. Static analysis \= type hints, and even then...

Besides, there must be some behavior you expect from this object. You could make a type that reflects this: IntIndexable or something, with an int index method and whatever else you need.

This feels like an extremely weak argument. Just think of it as self-enforcing documentation that also benefits auto-complete; what's not to love? Having an IntIndexable type seems like a great idea in your use case.

[spicyusername]

    And you better believe that every single admissible type

This is exactly why I hate using Python.

[thrance]

For this very specific example, isn't there something like "Indexable[int]"?

[gostsamo]

why not a protocol with getitem with an int arg?

[lucasyvas]

This is like saying you don’t like nails because you don’t understand how to use a hammer though. Developers are not understanding how to use the hints properly which is causing you a personal headache. The hints aren’t bad, the programmers are untrained - the acknowledgement of this is the first step into a saner world.