[groodt]

Python packaging gets a lot of criticism. It's a meme. The thing is, it's actually improved dramatically over the years and continues to improve.

The problems it solves are very complex if one looks a little below the surface. It is solving different problems to the ecosystems that it's often compared to: golang, rust, java, js.

[atoav]

This is true, still: Everyday things that should be straightforward, or pythonic if you will, are way to convoluted and complicated.

As a python dev with experience since 2.6 I agree it has gotten better, but it is also rotten at it's core. The problems python has to solve there are hard, but this is why it should be priority number one to solve them elegantly and in a fashion so that 99% of python users never have to worry about them.

Right now packaging and dependency managment are my number one concern when I think about rexommending the language to beginners. The things I needed to learn just to figure out what is a good way of developing something with dependencies and deploying it on another machine is just way too much. When I was going through this there was no official "this is how it is done" guide.

Now there is poetry for example. Poetry is good. But there are still hard edges when you need to deploy to a system without poetry on it for example.

[JAlexoid]

Well... The setuptools package was around for way too long.

I never managed to get data files packaged into a dist elegantly... annoys me to this day.

[short_sells_poo]

I wonder to what degree these complex problems are self inflicted due to more than a decade of flipflopping between a myriad packaging and environment management solution. In such an environment, you are bound to have so many different approaches between important projects that trying to bring them all under one umbrella becomes next to impossible. What is a python package is relatively well defined, but how you build one is completely up to you.

Edit: I've just read another comment which I think pointed out the most pertinent fact - that python has often served as mere glue code for stuff written in lower level languages. This then results in an explosion of complexity as a python package not only has to package the python part, but has to be able to build and package the lower level stuff.

[wiredfool]

I'd have to look at numbers to be sure, but I think that the number of popular packages that include compiled libraries is dramatically higher than it was 10 years ago, and that's about when wheels were taking over. The Data/Science stack has really exploded in that time, and it's very heavily skewed towards packaging c/fortran/rust libraries. Previously there was PIL, some database drivers, and numpy was just getting started. I think more of the big packages were pure python. The earlier egg/easy_install/msi/exe installers have all faded away, and now it's really just wheels and sdists.

[hot_gril]

How is it very different from NodeJS? Cause I find npm way easier to deal with than Python packaging, and it's also dealing with native code. I used Python heavily for 6 years and still have no idea how the packages work as a user trying to install libs; I used to just thrash around till it works. I don't use it anymore at my new job.

The one thing I understand is npm installs everything locally by default (unless you -g), and in Python it's hard to stay local even if you use a venv.

[pxc]

> How is [Python] very different from NodeJS?

Unlike Node, Python is essentially older than modern package management. When Python developers first decided to tackle distributing their code, `apt-get` did not yet exist.

Early approaches which stuck around way too long let any package do more or less anything at install time, and didn't bother with static metadata (can't figure out what your deps are except by attempting an install!). Subsequent solutions have struggled to build consensus in an already vast and mature ecosystem. Backwards compatibility means compatibility with bad conventions.

[uranusjr]

Fundamentally the problems are the same, but the communities have very different priorities. The vast majority of Node is still web dev and the community focuses on dependencies that are deployed as JavaScript files (including transpiled code); this allows NPM to mostly ignore issues related to linking to native binaries and put more focus on those user-facing features you see and like. Python on the other hand has a lot of stakes on interacting with native binaries (scientific computing, automation, etc.) and with systems older than Node’s existence. This consumes up a ton of the community’s resource, and every new UX improvement also needs to take account of those use cases. If you’re ever tasked with keeping projects with native dependencies working over a moderate period of time in both languages, you’d gain a lot of respect on how well Python packaging works.