Python 3.12

[dandiep]

Ooh, seems there is a new syntax for declaring the types of kwargs [1]:

  from typing import TypedDict, Unpack
  
  class Movie(TypedDict):
    name: str
    year: int

  def foo(*kwargs: Unpack[Movie]): ...

Maybe now I'll be able to actually figure out what data to send libraries without actually reading their source code.

1. https://docs.python.org/3.12/whatsnew/3.12.html#pep-692-usin...

[appplication]

> Maybe now I'll be able to actually figure out what data to send libraries without actually reading their source code.

One could hope, but any library abusing kwargs in all their methods is showing they’re willing to go through the absolute minimum to make their code usable, let alone readable and self-documenting.

[winter_blue]

> be able to actually figure out what data to send libraries without actually reading their source code

Just reading this sent a chill down my spine. I have horrible memories of having to read the code to figure out what something was doing (in JavaScript, Python, Ruby, etc), due to the disaster of an anti-feature called dynamic typing having been used.

[ledauphin]

what's especially interesting about this is that it could create a new "meta" for static typing in Python.

One significant issue with static typing in Python is how much boilerplate is required to use types when also doing the sorts of things that dynamic Python is really good at - for instance proxying functions. If you want to do that now and preserve the types, you need to re-declare the types of everything in the wrapper.

Now, if the underlying function already made use of Unpack, you could "reuse" that type in your own wrapper with low boilerplate and less chance of things diverging in hard-to-refactor ways.

[pmeira]

Some packages already use those. For previous Python versions, those are available in typing_extensions: https://typing-extensions.readthedocs.io/en/latest/

[vorticalbox]

some add what it takes in the DOC string, but even then most don't actually state all the options.

[hyperbovine]

Surely you mean `**kwargs`

[unstruktured]

Unfortunately the typed kwargs are NOT composable :(.

[dagmx]

I wonder if Unpack works on a function? I assume it’s any callable.

[markrages]

Modern Python is sure ugly.

[IshKebab]

Finally! I've been waiting for this for years.

Now I just have to wait 5 more years until 3.12 is sufficiently old that work lets me use it.

Bets on user-upgradable Python on Linux by 2030?

[georgehotelling]

I'm just happy for itertools.batched for chunking iterables: https://docs.python.org/3.12/library/itertools.html#itertool...

[philshem]

Yes. I’ve written explicit code that needed this 100s of times.

[kastden]

This is the greatest addition since f-strings!

[ehsankia]

99% of my more_itertools imports are exactly for this.

there's 1-2 other stuff from more_itertools that I think should make it to itertools. I'd actually like to see statistics from huge monorepos/opensource about usage stats of various more_itertools functions.

[kzrdude]

Great call-out! (Mistakes elided..)

[abyesilyurt]

Checkout more-itertools for more variants: https://pypi.org/project/more-itertools/

[zem]

yeah! that's been in the ruby stdlib practically from day one, no idea why python was so resistant to it.

[smacke]

It's not in the highlights, but one of the things that excites me most is this: https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-i...

> PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details.

Low-overhead instrumentation opens up a whole bunch of interesting interactive use cases (i.e. Jupyter etc.), and as the author of one library that relies heavily on instrumentation (https://github.com/ipyflow/ipyflow), I'm very keen to explore the possibilities here.

[maegul]

Great insight! I wouldn’t have made that connection back to reactive runtimes. Excited to see what you can do with ipyflow!

[bloopernova]

What's new: https://docs.python.org/dev/whatsnew/3.12.html

Summary, sorry for poor formatting, I'm not sure HN can do a list of any kind?

New features

More flexible f-string parsing, allowing many things previously disallowed (PEP 701).

Support for the buffer protocol in Python code (PEP 688).

A new debugging/profiling API (PEP 669).

Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).

Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.

Support for the Linux perf profiler to report Python function names in traces.

Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement.

Type annotations

New type annotation syntax for generic classes (PEP 695).

New override decorator for methods (PEP 698).

Deprecations

The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.

In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).

The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.

A number of other old, broken and deprecated functions, classes and methods have been removed.

Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)

The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)

[H8crilA]

Isolated subinterpreters (PEP 684): Just how isolated are those? Is it simply a more complicated version of multiprocessing, with all the same drawbacks (communication via pipes/socket/some-other-stream)?

[satvikpendem]

> I'm not sure HN can do a list of any kind?

It can, just use `-` characters for bullets, like in Markdown. They won't render as Unicode bullet points but dashes are fine enough.

[thanatropism]

What are the advantage of Cython over something like C++ with pybind11 or whatever the equivalent in Rustland?

[liquidpele]

Tbh the f-string stuff sounds pretty in-pythonic to me… like it would make it much harder to read and abuse.

[godelski]

They keep getting improved error messaging and this is one of my favorite features. But I'd love if we could get some real rich text. Idk if anyone else uses rich, but it has infected all my programs now. Not just to print with colors, but because it makes debugging so much easier. Not just print(f"{var=}") but the handler[0,1]. Color is so important to these types of things and so is formatting. Plus, the progress bars are nice and have almost completely replaced tqdm for me[2]. They're just easier and prettier.

[0] https://rich.readthedocs.io/en/stable/logging.html

[1] Try this example: https://github.com/Textualize/rich/blob/master/examples/exce...

[2] Side note: does anyone know how to get these properly working when using DDP with pytorch? I get flickering when using this and I think it is actually down to a pytorch issue and how they're handling their loggers and flushing the screen. I know pytorch doesn't want to depend on rich, but hey, pip uses rich so why shouldn't everyone?

[brenns10]

I love and use rich too, but gosh I hope that libraries don't start depending on it just because pip does.

It has a lot of dependencies of its own, and dependency creep is real. I know pytorch isn't exactly lightweight in terms of dependencies. But I prefer using libraries that make an effort do only pull in absolutely necessary dependencies.

[bluish29]

I think the support for isolated sub-interpreters with separate Global Interpreter Locks is the most interesting new feature in python. It is doubtful not the best way to offer some sort of concurrency but still a step closer to maybe one day get rid of GILs.

[ptx]

Since it currently lacks any way to transfer objects between interpreters other than pickling, does it offer any advantage over the multiprocessing module?

[qsort]

Too bad there's no Python API available yet :(

It's scheduled to be delivered next year if I'm not mistaken.

The new syntax for generic types is also a very nice QOL improvement, you can now just do:

  class MyList[T]:
      ...

[yxhuvud]

It may not be a step towards that. Ruby has guilds which is a very similar idea and they are explicitly not working towards removing them altogether at this point. Matz did a full keynote on the latest Euruku defending not working towards removing the whole of it. See https://www.youtube.com/watch?v=5WmhTMcnO7U&t=1244s for the full talk if you are interested.

I find the convergent evolution of features in these two languages pretty funny, as it is very clear that they don't really look at implementation details of the other language even if they quite often land on ideas that are pretty close in practice.

[zozbot234]

Well, that's where the whole "for Workgroups" part comes in I suppose. See my sibling comment.

[nickdrozd]

Running my project's computation-intensive test suite takes 83 seconds under 3.11 and 67 seconds under 3.12. That's a pretty good improvement!

[zozbot234]

Wait, what happened to Python 3.11 for Workgroups? Something's not right here.

[formerly_proven]

Breaking changes:

PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils, if you still require it in Python 3.12 and beyond.

gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment.

The asynchat, asyncore, and imp modules have been removed, along with several unittest.TestCase method aliases.

[reaperman]

What are the best ways to utilize PEP 669? I’m always looking for two things: a good “time-travel” debugging experience (PyCrunch-Trace seems to be broken for me) and for a way to generate and view large traces (Firefox Performance Profiler can’t open traces that lasted 2-5 minutes because they're too large)

PEP 669: Low impact monitoring for CPython

PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details.

[Alosra]

Everything aside, I enjoyed the "And now for something completely different" part.

[smallerfish]

The new generic typing seems intuitive - https://docs.python.org/3.12/whatsnew/3.12.html#pep-695-type.... I've never run into its gross predecessor: https://peps.python.org/pep-0484/#generics. Nice change.

[bloopernova]

oh cool, multi-line f-strings. With inline comments!

    f"This is the playlist: {", ".join([
        'Take me back to Eden',  # My, my, those eyes like fire
        'Alkaline',              # Not acid nor alkaline
        'Ascensionism'           # Take to the broken skies at last
    ])}"

[thatcherthorn]

As now f-strings can contain any valid Python expression inside expression components, it is now possible to nest f-strings arbitrarily:

>>> f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}"

'2'

Is anyone aware of the change to the interpreter that allows for this?

[pytlicek]

:) Oh, this is neat! Python 3.12.0 has brought some cool features to the table. I'm particularly keen on the flexible f-string parsing and the performance improvements. The new type annotation syntax for generic classes is a nice touch too. It's always good to see Python getting refined with each release. Looking forward to exploring these new additions...

[roger_]

The new `type` statement with lazy evaluation looks interesting.

Wonder if it could be used to add lazy evaluation to other libraries, like numpy?

[justinmayer]

Python 3.12 is now available via asdf [1]:

    asdf install python 3.12.0

[1]: https://justinmayer.com/posts/homebrew-python-is-not-for-you...

[gorgoiler]

What is the rationale for types being part of the syntax but without shipping a built in type checker? Are we being conservative and waiting to see where the four type checkers — mypy, pyrite, pyre, pytype, and others? — lead us before rolling one into the standard distribution?

[heavyset_go]

Read the PEP for Type Hints, the rationale is laid out there.

Python will probably never have a built-in type checker beyond Python's existing strong typing at runtime.

[est]

> PEP 684, a unique per-interpreter GIL

We are on the track to remove GIL!

[benrutter]

I dunno man, seems like there's more GILs than ever before now you can have one per interpreter. . .

[skrause]

This change is completely separate from nogil, not it’s first step.

[paganel]

What's with the politicising at the end? I agree that both the EU (from where I'm from) and the US have chosen to take a very dark and authoritarian view on immigration, but I think that that discussion doesn't belong in a Python release announcement.