Looks like you found the not-so-secret repository we're using to prepare for a broader announcement :)
Please be aware this is pre-alpha software. The current version is 0.0.0a6 and the releases so far are all in service of validating our release process. We're excited to get this in people's hands, but want to set the expectation that we still have a lot of work left to do before this is production ready.
For pre-alpha software it's working fantastic for my project. I thought I type annotated it well, but Ty had quite a lot of feedback for me. Great job and I can't wait until this is released.
If you can say - are there any thoughts about implementing plugins / extension capabilities to keep type checking working even with libraries that aren't otherwise typecheckable?
(where "not otherwise typecheckable" means types that can't be expressed with stubs - e.g., Django, dataclasses pre-PEP-681, pytest fixtures, etc.)
What might, possibly, redeem Python in my eyes as a potential language for making production applications (something that today, it is most certainly not) would be if the type checker worked across the broader ecosystem of common Python packages.
For example, as my recent struggles showed, SQLAlchemy breaks `pyright` in all kinds of ways. Compared with how other 'dynamic' ORMs like Prisma interact with types, it's just a disaster and makes type checking applications that use it almost pointless.
Cool! Out of curiosity, what's the bedrock that's used to determine what the fundamental python AST objects are? I'm wondering what the "single source of truth" is, if you will.
Is this all based off a spec that python provides? If so, what does that look like?
Or do you "recode" the python language in rust, then use rust features to parse the python files?
Regardless of how it's done - This is a really fascinating project, and I'm really glad you guys are doing it!
Curious if this means it'll be released as a separate binary than ruff? I personally feel like having it within ruff is much nicer for ensuring that we have a consistent set of dependencies that play nicely with each other. Though I guess because a type checker doesn't mutate the files maybe that's not a real concern (vs formatting/linting with --fix).
I gave away the “ty” project name on pypi to Astral a week or so ago. I wanted to use it for a joke a few years ago but this is a much better use for a two letter project name. They agreed to make a donation to the PSF to demonstrate their gratefulness.
The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.
The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.
I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
> The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.
Or in this case, writing it in Rust...
mypy is written in Python. People have forgotten that Python is really, really slow for CPU-intensive operations. Python's performance may not matter when you're writing web service code and the bottlenecks are database I/O and network calls, but for a tool that's loading up files, parsing into an AST, etc, it's no surprise that Rust/C/even Go would be an order of magnitude or two faster than Python.
uv and ruff have been fantastic for me. ty is definitely not production ready (I see several bizarre issues on a test codebase, such as claiming `datetime.UTC` doesn't exist) but I trust that Astral will match the "crazy reality" of real Python (which I agree, is very crazy).
In defense of mypy et al, Typescript had some of the greatest minds of our generation working for a decade+ on properly typing every insane form found in every random Javascript file. Microsoft has funded a team of great developers to hammer away at every obscure edge case imaginable. No other python checker can compare to the resources that TS had.
>The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.
Nah, that's just part of the parade of excuses that comes out any time existing software solutions get smoked by a newcomer in performance, or when existing software gets more slow and bloated.
I also mentioned this up-thread, but https://pypi.org/project/django-types/ is compatible with pyright without plugins, so it should theoretically work with ty. It's not quite as good as the mypy-django plugin but it still catches a lot.
> prior to astral appearing, python's tooling has been beyond terrible, compared to say, Java's
I would concur with you if you said Go, Rust, Ruby, or even heck, PHP, but Java is probably the only language that I know that is in a situation even as bad as Python or even worse (at least for me definitely worse, because at least I understand Python tooling enough even when using it only for hobby projects, while I still don't understand Java tooling enough even after working professionally with JVM languages for 7+ years).
Java is the only language that I know except Python that has multiple project/package managers (Maven, Gradle, probably even more). It also has no concept of lock files in at least Maven/Gradle, and while resolution dependency in Maven/Gradle is supposed to be deterministic, from my experience it is anything but: just a few weeks ago we had a deployment that failed but worked locally/CI because of dependency resolution somehow pulled different versions of the same library.
Fighting dependency hell because different dependencies pull different version constraints is a pain (all Java/JVM projects that I ever worked had some manually pinned dependencies to either fix security issues or to fix broken dependency resolution), and don't even get me in the concept of Uber JARs (that we had to use in previous job because it was the only way to ensure that the dependency tree would be solved correctly; yes maybe it was by incompetence of the team that maintained our shared libraries, but the fact that we even got at that situation is unacceptable).
Oh, and also Gradle is "so fun": it is a DSL that has zero discovery (I had IntelliJ IDEA Ultimate and I could still not get it to auto-complete 60% of the time), so I would just blindly try to discover what where the inputs of the functions. The documentation didn't help because the DSL was so dynamic and every project would use it slightly different, so it was really difficult to discover a way to make it work for that specific project (the examples that I would find would be enough different from my current project that 90% of time it wouldn't work without changing something). Python at least has `pyproject.toml` nowadays, and the documentation from PyPA is good enough that you can understand what you want to do after reading it for 10 minutes.
Even after all the praise, I'd say they're underrated.
Modular.ai raised $100 million to solve tangentially similar problems with python. Astral has already had a much larger impact, while providing better integration with less than 10% of that money.
Very excited to have a new fully featured Python language server working in both vscode and vscode forks (e.g. Windsurf, Cursor).
Pylance is borked on these forked distributions, so having a new solid alternative here that doesn't involve adopting yet another forked Pyright implementation (BasedPyright, Cursor Pyright, Windsurf Pyright, ...) sounds great to me.
Not the most fun question, but as I see Astral taking over the python ecosystem, I can't help but wonder: how do y'all plan to make money? It seems like you've taken VC funding, so monetization is inevitable.
I'm curious to see what Astral will cook up! I assume they'll probably eventually create some sort of paid devtool service.
With that being said, the worst case scenario is that they go caput, but that still leaves the python community with a set of incredible new rust-based tools. So definitely a net win for the python community either way!
Pyrefly is a rewrite of Pyre, a Python typechecker which has been around for 4-5 years. Pyre is the strictest type checker I've used, compared to mypy, but its kind of a pain to set up.
Yeah.. Have either of the ty / pyrefly teams reached out to the other? I feel like the community does not really need two fast python type checkers.
(However, vc-backed astral probably need control over theirs to keep monetization options open, and Facebook probably need control over theirs so it can be targeted at Facebook's internal cool-but-non-standard python habits... Sigh.
Why do we have nice things? Money. Why can't we have nice things? Also money.)
in python eco system you have linters like ruff which do hardly any type checking and type checkers like mypy which do try to approach complete type checking, but still are absurdly full of holes
2. speed
any of the "established" type checkers either are supper slow (e.g. mypy) so you only run it like once before commit instead of "life" or do fail to properly type check so many things that if you have a requirement for "adequate static code analysis" they reliably fail that requirement (which might result in a legal liability, but even if not is supper bad for reliable code and long term maintenance)
also probably priorities are switched with 1st speed then closing holes as the later part is really hard due to how a mess python typing is (but in many code bases most code won't run into this holes so it's okay, well except if you idk. use pyalchemy as "ORM" subclassing base model (just don't terrible idea)).
Existing python typecheckers are bad, slow and their communities are fragmented. If the community would agree on a single good and fast type checker, everyone will benefit.
I am literally checking HackerNews while I wait for mypy to finish running, so I am excited to hear a faster type checker is on the way! Hope the error messages are also helpful.
We're definitely thinking hard about the ergonomics of our error messages! We're drawing inspiration from rustc and miette for the diagnostic model, and are aiming for a quality bar on par with rustc for their content.
I've been looking forward to this since the original announcement (and before, really).
On the modest codebase I tried it on (14k LOC across 126 files), it runs in 149ms compared to 1.66s in pyright (both run via uvx <tool>). I couldn't get it to play nicely with a poetry project, but it works fine (obviously) in a uv project.
Definitely some false-positives, as expected. Interestingly, it seems to hate the `dict()` initializer (e.g. `dict(foo="bar")`).
The timing of the recent batch of propaganda makes it hard to believe it's not coordinated. I wouldn't suggest paid actors but maybe just an attempt to counter some fairly visible and negative recent takes. The amount of "I love Rust but" comments make it hard to take the commentary seriously too.
Probably not a top priority but it would be really really cool if this thing had solid t-string support from the jump, to the extent that it’s feasible without actually executing code
Perhaps a silly question. Will ty be usable for getting semantical completions / suggestions. Similar to using pyright to get completions based on what's being written.
We are planning on shipping an LSP front end, and the goal is for that to include code completions. Though to set expectations, they will probably not be that sophisticated on day one. There's a lot of interesting work that we could do here, but it will take time!
Awesome work. What is the business model for these astral tools? It’s a bit of a “waiting for the other shoe to drop” feeling after seeing the VC backing on the company page.
From what I’ve gathered (because I had similar concerns), the code is properly open source. In the very worst case, should there ever come a rug pull, it can be forked.
Recently I started a python project and I wanted to do it the "proper" way. mypy + pylint. But even on this small 15-20kloc program these tools are way to slow to do anything in realtime. It takes double digit seconds to have feedback. Way to long for an LSP. I'm honestly appalled the state of affairs is this bad. What the hell do people do with moderately or even large sized code bases?
Pyright + Pylance + Ruff has been rock solid for me on my 100Kloc codebase for more than a year now. I use the VS Code extensions, and Pyright and Ruff are integrated into my pre-commit.
Fingers crossed this is/becomes extensible. Pyright and MyPy both suffer from lack of extensibility IMO (Pyright doesn't consider the use case and MyPy plugins come across as an afterthought with limited capabilities). There are many things that can be built on the back of type-checked AST.
Charlie already said in a podcast (https://www.youtube.com/watch?v=XVwpL_cAvrw) that they are not looking to make it extensible. That it's considered a feature that type checking works interchangeably across tools and projects.
Ruff's linting and formatting is more likely to get plugin/extension support at some point in the future.
We do plan to provide an LSP server and VS Code plugin, which will support GTD etc. Though as several others have pointed out (e.g. https://news.ycombinator.com/item?id=43919354), it's still very early days for ty, so we don't have concrete release announcements for that yet.
Looks good but it has the same issues that i have with mypy. Packages that don't include the type hints blow-up my process. In mypy i've come to terms with strategically ignoring packages or finding a package of type hints.
Mypy is runs cleanly on my project but I get >800 errors with TY, mostly things like:
lint:unresolved-import: Cannot resolve imported module `pydantic`
--> vartia/usr_id.py:4:6
|
2 | from typing import Optional, Any
3 | from enum import Enum
4 | from pydantic import BaseModel, ConfigDict
The current version can handle importing pydantic without error just fine, but it probably can't find your virtualenv, so it doesn't know what third-party dependencies you have installed. Ty will discover your venv if it is in `.venv` in the project directory; otherwise you can help it out with the `--python` CLI flag.
This will be similar to Typescript I assume? If so I can’t wait to use it!! I cant count how many times I’ve searched for “TS like in Python” since I’ve started working on Python codebase. TS is so awesome that I use it 100% on new projects. Ruff is also very good, but with this, large code base Python will be a breeze to work with
You can already use static type annotations in Python and check them with Pyright. This will just make it faster.
Also currently the Python IDE support (autocompletion, refactoring, etc.) in VSCode is provided by Pylance which is closed source, so this would provide an open source alternative to that.
zanie|9 months ago
Looks like you found the not-so-secret repository we're using to prepare for a broader announcement :)
Please be aware this is pre-alpha software. The current version is 0.0.0a6 and the releases so far are all in service of validating our release process. We're excited to get this in people's hands, but want to set the expectation that we still have a lot of work left to do before this is production ready.
Stay tuned for more for news in the near future!
(... I work at Astral)
BewareTheYiga|9 months ago
12_throw_away|9 months ago
(where "not otherwise typecheckable" means types that can't be expressed with stubs - e.g., Django, dataclasses pre-PEP-681, pytest fixtures, etc.)
davedx|9 months ago
For example, as my recent struggles showed, SQLAlchemy breaks `pyright` in all kinds of ways. Compared with how other 'dynamic' ORMs like Prisma interact with types, it's just a disaster and makes type checking applications that use it almost pointless.
How does Ty play with SQLAlchemy?
digdugdirk|9 months ago
Is this all based off a spec that python provides? If so, what does that look like?
Or do you "recode" the python language in rust, then use rust features to parse the python files?
Regardless of how it's done - This is a really fascinating project, and I'm really glad you guys are doing it!
theLiminator|9 months ago
opem|9 months ago
ZiiS|9 months ago
ngoldbaum|9 months ago
_carljm|9 months ago
Celeo|9 months ago
swyx|9 months ago
smitty1e|9 months ago
unknown|9 months ago
[deleted]
rrszynka|9 months ago
aleksanb|9 months ago
The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.
I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
mjr00|9 months ago
Or in this case, writing it in Rust...
mypy is written in Python. People have forgotten that Python is really, really slow for CPU-intensive operations. Python's performance may not matter when you're writing web service code and the bottlenecks are database I/O and network calls, but for a tool that's loading up files, parsing into an AST, etc, it's no surprise that Rust/C/even Go would be an order of magnitude or two faster than Python.
uv and ruff have been fantastic for me. ty is definitely not production ready (I see several bizarre issues on a test codebase, such as claiming `datetime.UTC` doesn't exist) but I trust that Astral will match the "crazy reality" of real Python (which I agree, is very crazy).
johnfn|9 months ago
TheTaytay|9 months ago
(And uv and ruff have basically proved that at this point)
amelius|9 months ago
And not directly related, but I wish more python modules did proper checks with Valgrind before shipping.
HideousKojima|9 months ago
Nah, that's just part of the parade of excuses that comes out any time existing software solutions get smoked by a newcomer in performance, or when existing software gets more slow and bloated.
Here's one of many examples:
https://m.youtube.com/watch?v=GC-0tCy4P1U&pp=0gcJCdgAo7VqN5t...
collinmanderson|9 months ago
davedx|9 months ago
tmvphil|9 months ago
- mypy (warm cache) 18s
- ty: 0.5s (and found 3500 errors)
They've done it again.
_carljm|9 months ago
This is an early preview of a pre-alpha tool, so I would expect a good chunk of those 3500 errors to be wrong at at this point :) Bug reports welcome!
mil22|9 months ago
https://github.com/microsoft/pyright
js2|9 months ago
> This project is still in development and is not ready for production use.
rybosome|9 months ago
Indeed they have. Similar improvement in performance on my side.
It is so fast that I thought it must have failed and not actually checked my whole project.
Handprint4469|9 months ago
simonw|9 months ago
Adding "time uvx ty check" shows it took:
wdroz|9 months ago
scosman|9 months ago
0.56s user 0.14s system 302% cpu 0.231 total
Damn that's fast. And zero issues in this project with pyright so curious that these are...
blibble|9 months ago
astral have now replaced the awful pip with the fantastic uv
various awful linters with with the fantastic ruff
and now hopefully replacing the terrible type checkers (e.g. mypy) with a good one!
I hope they have the pypi backend on their list too, my kingdom for Maven Central in python!
kokada|9 months ago
I would concur with you if you said Go, Rust, Ruby, or even heck, PHP, but Java is probably the only language that I know that is in a situation even as bad as Python or even worse (at least for me definitely worse, because at least I understand Python tooling enough even when using it only for hobby projects, while I still don't understand Java tooling enough even after working professionally with JVM languages for 7+ years).
Java is the only language that I know except Python that has multiple project/package managers (Maven, Gradle, probably even more). It also has no concept of lock files in at least Maven/Gradle, and while resolution dependency in Maven/Gradle is supposed to be deterministic, from my experience it is anything but: just a few weeks ago we had a deployment that failed but worked locally/CI because of dependency resolution somehow pulled different versions of the same library.
Fighting dependency hell because different dependencies pull different version constraints is a pain (all Java/JVM projects that I ever worked had some manually pinned dependencies to either fix security issues or to fix broken dependency resolution), and don't even get me in the concept of Uber JARs (that we had to use in previous job because it was the only way to ensure that the dependency tree would be solved correctly; yes maybe it was by incompetence of the team that maintained our shared libraries, but the fact that we even got at that situation is unacceptable).
Oh, and also Gradle is "so fun": it is a DSL that has zero discovery (I had IntelliJ IDEA Ultimate and I could still not get it to auto-complete 60% of the time), so I would just blindly try to discover what where the inputs of the functions. The documentation didn't help because the DSL was so dynamic and every project would use it slightly different, so it was really difficult to discover a way to make it work for that specific project (the examples that I would find would be enough different from my current project that 90% of time it wouldn't work without changing something). Python at least has `pyproject.toml` nowadays, and the documentation from PyPA is good enough that you can understand what you want to do after reading it for 10 minutes.
screye|9 months ago
Modular.ai raised $100 million to solve tangentially similar problems with python. Astral has already had a much larger impact, while providing better integration with less than 10% of that money.
sakesun|9 months ago
danlamanna|9 months ago
IIRC they have floated the idea of private registries as a commercial offering in the past.
zahlman|9 months ago
rexledesma|9 months ago
Pylance is borked on these forked distributions, so having a new solid alternative here that doesn't involve adopting yet another forked Pyright implementation (BasedPyright, Cursor Pyright, Windsurf Pyright, ...) sounds great to me.
maxloh|9 months ago
> basedpyright re-implements many features exclusive to pylance - microsoft's closed-source extension that can't be used outside of vscode.
hamandcheese|9 months ago
zanie|9 months ago
renmillar|9 months ago
krupan|9 months ago
dcreager|9 months ago
__MatrixMan__|9 months ago
joshdavham|9 months ago
With that being said, the worst case scenario is that they go caput, but that still leaves the python community with a set of incredible new rust-based tools. So definitely a net win for the python community either way!
never_inline|9 months ago
nickagliano|9 months ago
Not making any sort of ethical statement, just interesting that rust keeps eating the python and JS tooling worlds.
masklinn|9 months ago
Astral announced they were building a typechecker back in January: https://x.com/charliermarsh/status/1884651482009477368
emptysea|9 months ago
https://github.com/facebook/pyrefly/blob/a8626110da034f8e513...
tasn|9 months ago
ipsum2|9 months ago
kmacdough|9 months ago
zem|9 months ago
akdor1154|9 months ago
(However, vc-backed astral probably need control over theirs to keep monetization options open, and Facebook probably need control over theirs so it can be targeted at Facebook's internal cool-but-non-standard python habits... Sigh.
Why do we have nice things? Money. Why can't we have nice things? Also money.)
cristea|9 months ago
tayo42|9 months ago
dathinab|9 months ago
in python eco system you have linters like ruff which do hardly any type checking and type checkers like mypy which do try to approach complete type checking, but still are absurdly full of holes
2. speed
any of the "established" type checkers either are supper slow (e.g. mypy) so you only run it like once before commit instead of "life" or do fail to properly type check so many things that if you have a requirement for "adequate static code analysis" they reliably fail that requirement (which might result in a legal liability, but even if not is supper bad for reliable code and long term maintenance)
also probably priorities are switched with 1st speed then closing holes as the later part is really hard due to how a mess python typing is (but in many code bases most code won't run into this holes so it's okay, well except if you idk. use pyalchemy as "ORM" subclassing base model (just don't terrible idea)).
mvieira38|9 months ago
simonw|9 months ago
silverwind|9 months ago
pamelafox|9 months ago
dcreager|9 months ago
tiltowait|9 months ago
On the modest codebase I tried it on (14k LOC across 126 files), it runs in 149ms compared to 1.66s in pyright (both run via uvx <tool>). I couldn't get it to play nicely with a poetry project, but it works fine (obviously) in a uv project.
Definitely some false-positives, as expected. Interestingly, it seems to hate the `dict()` initializer (e.g. `dict(foo="bar")`).
SOLAR_FIELDS|9 months ago
throwaway63467|9 months ago
joshdavham|9 months ago
unknown|9 months ago
[deleted]
urbandw311er|9 months ago
andenacitelli|9 months ago
rc00|9 months ago
pizza|9 months ago
Sarios|9 months ago
dcreager|9 months ago
no_time|9 months ago
lemontheme|9 months ago
rowanG077|9 months ago
mil22|9 months ago
Hasnep|9 months ago
catlover76|9 months ago
[deleted]
kodablah|9 months ago
notatallshaw|9 months ago
Ruff's linting and formatting is more likely to get plugin/extension support at some point in the future.
andenacitelli|9 months ago
Affric|9 months ago
When it was released it might have been one of the easiest to use languages.
The focus on tooling and making the tooling fast has been sharp. Seeing people recommend using non-astral tooling seems nuts at this point.
cyounkins|9 months ago
preciousoo|9 months ago
simlevesque|9 months ago
joshdavham|9 months ago
The-Ludwig|9 months ago
drcongo|9 months ago
codydkdc|9 months ago
f311a|9 months ago
dcreager|9 months ago
briandw|9 months ago
lint:unresolved-import: Cannot resolve imported module `pydantic` --> vartia/usr_id.py:4:6 | 2 | from typing import Optional, Any 3 | from enum import Enum 4 | from pydantic import BaseModel, ConfigDict
looking forward to the release version.
_carljm|9 months ago
ljouhet|9 months ago
For now, I have some false negative warnings :
'global' variables are flagged as undefined `int:unresolved-reference: Name ... used when not defined` (yeah, it's bad, I know)
f(*args) flagged as missing arguments `lint:missing-argument: No arguments provided for required parameters ...`
pt_PT_guy|9 months ago
robertwt7|9 months ago
IshKebab|9 months ago
Also currently the Python IDE support (autocompletion, refactoring, etc.) in VSCode is provided by Pylance which is closed source, so this would provide an open source alternative to that.
TeeMassive|9 months ago
joejoo|9 months ago
sestep|9 months ago
_carljm|9 months ago
dhruv3006|9 months ago
ivanbelenky|9 months ago
selfselfgo|9 months ago
[deleted]
canterburry|9 months ago
[bring on the downvotes]
Hasnep|9 months ago
codr7|9 months ago
Not every situation calls for type safe languages, you're projecting a preference.