[jofer]

One of my biggest gripes around typing in python actually revolves around things like numpy arrays and other scientific data structures. Typing in python is great if you're only using builtins or things that the typing system was designed for. But it wasn't designed with scientific data structures particularly in mind. Being able to denote dtype (e.g. uint8 array vs int array) is certainly helpful, but only one aspect.

There's not a good way to say "Expects a 3D array-like" (i.e. something convertible into an array with at least 3 dimensions). Similarly, things like "At least 2 dimensional" or similar just aren't expressible in the type system and potentially could be. You wind up relying on docstrings. Personally, I think typing in docstrings is great. At least for me, IDE (vim) hinting/autocompletion/etc all work already with standard docstrings and strictly typed interpreters are a completely moot point for most scientific computing. What happens in practice is that you have the real info in the docstring and a type "stub" for typing. However, at the point that all of the relevant information about the expected type is going to have to be the docstring, is the additional typing really adding anything?

In short, I'd love to see the ability to indicate expected dimensionality or dimensionality of operation in typing of numpy arrays.

But with that said, I worry that typing for these use cases adds relatively little functionality at the significant expense of readability.

[hamasho]

I also had a very hard time to annotate types in python few years ago. A lot of popular python libraries like pandas, SQLAlchemy, django, and requests, are so flexible it's almost impossible to infer types automatically without parsing the entire code base. I tried several libraries for typing, often created by other people and not maintained well, but after painful experience it was clear our development was much faster without them while the type safety was not improved much at all.

[HPsquared]

Have you looked at nptyping? Type hints for ndarray.

https://github.com/ramonhagenaars/nptyping/blob/master/USERD...

[kccqzy]

What is this project actually for?

Its FAQ states:

> Can MyPy do the instance checking? Because of the dynamic nature of numpy and pandas, this is currently not possible. The checking done by MyPy is limited to detecting whether or not a numpy or pandas type is provided when that is hinted. There are no static checks on shapes, structures or types.

So this is equivalent to not using this library and making all such types np.ndarray/np.dtype etc then.

So we expend effort to coming up with a type system for numpy, and tools cannot statically check types? What good are types if they aren't checked? Just a more concise documentation for humans?

[jofer]

That one's new to me. Thanks! (With that said, I worry that 3rd party libs are a bad place for types for numpy.)

[dwohnitmok]

This isn't static, but jaxtyping gives you at least runtime checks and also a standardized form of documenting those types. https://github.com/patrick-kidger/jaxtyping

[jofer]

It actually doesn't, as far as I know :) It does get close, though. I should give it a deeper look than I have previously, though.

"array-like" has real meaning in the python world and lots of things operate in that world. A very common need in libraries is indicating that things expect something that's either a numpy array or a subclass of one or something that's _convertible_ into a numpy array. That last part is key. E.g. nested lists. Or something with the __array__ interface.

In addition to dimensionality that part doesn't translate well.

And regardless, if the type representation is not standardized across multiple libraries (i.e. in core numpy), there's little value to it.

[davnn]

I am using runtime type and shape checking and wrote a tiny library to merge both into a single typecheck decorator [1]. It‘s not perfect, but I haven‘t found a better approach yet.

[1] https://github.com/davnn/safecheck

[stared]

Some time ago I created a demo of named dimensions for Pytorch, https://github.com/stared/pytorch-named-dims

In the same line, I would love to have more Pandas-Pydantic interoperability at the type level.

[relatall]

wisdom in the community. people who can excel at things don’t go to big tech companies because they won’t be appreciated , they will get pipped if they can’t wear any of those chains shackles well

[efavdb]

Would a custom decorator work for you?

[jofer]

Unless I'm missing something entirely, what would that add? You still can't express the core information you need in the type system.