I just enter the directory and the environment is created/'activated'.
In all the years I never had any need to use the real activate method. Just running Python and scripts directory from the bin/ in the virtualenv is enough to run 'inside the env'.
When I get the choice I use Nix and ignore as much of the "python" packaging infrastructure as possible - it's all a vague and unreliable toy compared to a real package manager. It all particularly falls apart when it comes to non-python dependencies.
`pip-tools` has served me well, but an annoying gripe that pipenv/Poetry resolves is that you can't install git links in non-editable mode, i.e.
-e git+https://github.com/...
Works, but
git+https://github.com/...
Doesn't.
Also can you pin your dependencies to a specific hash? Last I checked you cannot. A major advantage of pipenv/Poetry is that the lockfile protects you against compromise of the package manager; if someone swaps in a new binary for yourdep==1.2.3, the lockfile will fail to validate the hash and you'll get an error.
Using --user here, no per-project virtualenv, used to have a virtualenv in ~/.env, where i keep everything up to date and have many -e installs from ~/src. For production deployments: --user in containers.
This means all my projects have to work with the same versions of dependencies.
This means all my dependencies have to work with the same versions of dependencies.
All my dependencies have to be up to date: all versions must be latest, or next to-be-released.
When don't support new versions of other dependencies: I deploy a fork and open a PR.
At such, I'm not spending time trying to make obsolete versions work, rather, I'm spending time making new versions work together.
My perception is that this strategy offers better ROI than all others which I have tried, not only for me, but for the whole ecosystem. That's also how I made my Python 2 to 3 transition, and have been using 100% Python 3 for a while (once you've ported one big software, porting others and their smaller dependencies is easy).
I'm extremely happy about this strategy and my life has just been better since I started working this way, and highly recommend it.
For my own (countless) packages: I use setupmeta which simplifies auto-updates (I think it succeeds what pbr tried to achieve).
There are a few highly dismissive comments here. While the python community has (clearly) been getting by with requirements.txt/setup.cfg/setup.py, the project/package management story is far less usable than more recently developed systems like npm and pub.
With poetry+pyproject.toml, I have one file that is flat, simple, and readable. @sdispater has done incredible work on this project, and I hope they get more resources for development.
Your comment does not explain why you think that the package management story is better in npm, it's merely like saying "node-gyp is a pile of debt" and not going into details, but I can surely tell I have no idea why node_modules ends up so fat and why my npm install eats a lot of bandwidth, time and disk space ... is npm trying to workaround incompatibilities by installing different versions of a same package in the same node_modules ? I hope not, because that would be just perfect to accumulate debt in the whole ecosystem. Not to mention that I was just handed a frontend source code where building the source requires node-gyp which requires g++ and python2 :)
Quick question since you seem to know npm very well, is there a better solution than this to automate npm package publishing nowadays ?
sed -i "s/GIT_TAG/${CI_COMMIT_REF_NAME/v/}/" package.json
It took me a while to learn to love pipenv. It also bothers me how many developers "blame pipenv" for any problem they can't easily explain.
Anyway, has Poetry caught up to the dependency resolution provided by the pipenv lock command yet? Last time I tried it (~6 months ago), it couldn't produce working environments for my requirements.
That's funny, Poetry's README [1] claims exactly the opposite (that it resolves dependencies better than pipenv). And looking at the git history, that section of the README is 2 years old, so that claim was made when you experimented.
Was the problem with dependency resolution you had similar to what Poetry describes in it README? Or are there other problems the README fails to mention?
The description they make got me interested, but up to now all alternatives I tried to build moderately complex python projects failed to deliver on their promises, so definitely interested in what you have to say.
I do really like the workflow pipenv provides. But last time I checked, the mayor downside of pipenv is it doesn't and won't support multiple Python versions [0]. For libraries, this can be a dealbreaker. For applications, especially ones you're deploying to a controlled environment, this isn't an issue.
Been meaning to look into poetry. Not a huge fan of TOML, but hopefully all the tooling supports pyproject.toml now (I know black does, but not sure about flake8, isort, pylint, pytest, coverage). I know there's still some hacks required for tox + poetry though.
My experience was the opposite. pipenv always told me something couldn't be installed because of version mismatches and poetry has never given me any trouble.
Does pipenv create cross-platform lock files? That has been my primary requirement when looking at solutions, more than the usability issue of Python using a bunch of tools and files for the whole development process without a coherent story.
iirc Poetry does have cross-platform lock files and a coherent story. I'm just blocked on some PRs due to the original author getting busy and not yet having delegated to lieutenants.
In the past Pipenv has failed to resolve dependencies for my project before, where Poetry (run last week) succeeded. Not a fair comparison as I've not tried Pipenv recently, but that at least lets you know that Poetry is now better than where pipenv was a few months ago.
The big win seems to be the lock file - like Cargo in Rust, or yarn in the JS world. It's really, really hard to lock down dependencies reliably in Python, especially when you are talking dependencies of the primary packages you are installing (and their dependencies, ...).
One solution at the moment is to run 'pip freeze' and put that in as your requirements file, but that very much feels like an 'and now I have a different problem' solution.
For dependency management needs, 99% of what you need can be done easily with vanilla virtualenv, vanilla pip, vanilla setup.py, and a Makefile. A small shell wrapper that sources bin/activate and runs your Python app is all you need to allow anyone to run your code without additional steps.
The biggest problem with Python packages isn't a dependency manager, it's that Python developers don't reuse and extend existing projects. They just keep churning out new similar projects with different features. All of the most commonly used Python packages are basically one-offs that use a bunch of other one-offs. This creates a cycle of reinventing the wheel and friction in ramping up on existing technology, due to all the extra integration work and split development, and later lift-and-shift into completely different dependencies that implement the same thing. PyPI is awash with duplicate abandoned one-offs to the point that just trying to just find a useful module to do what you want can take a long time.
I can't find a good comparison to Conda - is the main distinction simply that Poetry uses the official repos?
For people working outside of scientific Python: conda is a package and env manager maintained by a private company that's become the go-to because it's really good at handling binary dependencies.
Worked a little with and it looks really nice and promising, but my biggest concern is number of issues on github and that vast majority of commits is done by one person.
From what I've gathered, the project grew faster than the original dev had time to build up other people to also monitor PRs and has since gotten very busy, slowing down the review process. I know I have several straightforward outstanding PRs. I hope this is able to be resolved so we can have a sustainable community.
I like how transparent poetry is about what's happening when you run it, and how well presented that information is. I've come to loathe pipenv's progress bar. Running it in verbose mode isn't much better.
I can't be too mad at pipenv, but all in all poetry is a better experience.
Python is the worst thing to happen to SWE (specifically SWE, not ML, Education, etc).
Python makes huge sacrifices in readability and efficiency for a hypothetical win in writeability. It's also fundamentally at odds with the multicore world we're living in an will continue to live in, an issue that many people have tried to fix and failed at.
I can't count the number of times I've had to spend my entire night reading through an older Python service without type annotations, desperately trying to understand the types of inputs and outputs. It's just a mess, and the bias of programmers to believe that they are writing readable code (when in reality it's only readable to them) exacerbates the use of names that provide little context to solve the problem.
Python is awful, and it's an uphill battle to fix it. Asyncio can solve some performance issues, but negates the benefit of Python's simplicity by destroying the sequential mental model that Programmers can easily grok. It also requires a complete rewrite of libraries and a split between non-asyncio land and asyncio land.
Type annotations can make Python more readable and thus more maintainable, but gradual typing still leaves lots of unanswered questions when you're interfacing with untyped code.
Python is really not good unless you are using it to prototype or build something on your own. It's led to a world of slow, buggy software that is impossible to rewrite. It's downsides are easy to measure, but its benefits are difficult to quantify.
How do you start a new Python project in 2019? I mean, there's so much stuff. Poetry, pip, virtualenv... oh, my. With PHP, I just composer init and that's it.
Neat, definitely something I would want to give a try.
I see though that it only supports pure python for building packages, does that mean that it doesn't build if you are dependent on compiled libraries?
Is there also a plan to add some of the functionality of bundling tools such as web-pack into this build phase? like automated css optimization, image compression... Could be handy for some django/flask projects.
They can't support reading the poetry data because it's not generic. Nor are most practical implementations of tools using pyproject.toml because they all use custom tool-namespaced fields in pyproject.toml: it's far from a standardized format yet, despite the push for using it from the people that made it.
I still wish poetry would support setup.cfg in parallel. It has been working for 2 years, is compatible with pip, setuptools and the whole legacy stack, and most fields are standards and documented.
I was excited by this project, and pipenv, but unfortunately I haven't had consistent results cross platform with either. I have ended up sticking with venv because of this.
[+] [-] SmirkingRevenge|6 years ago|reply
- vanilla virtualenv: I don't even bother with the wrapper most of the time.
- vanilla setup.py/setup.cfg: Its just really not that bad. Forget about project.toml or whatever the next big thing is.
- pip-tools: ditching pipenv and using this for pinning app requirements, has made my life so much simpler.
[+] [-] aequitas|6 years ago|reply
In all the years I never had any need to use the real activate method. Just running Python and scripts directory from the bin/ in the virtualenv is enough to run 'inside the env'.
[+] [-] ris|6 years ago|reply
[+] [-] theptip|6 years ago|reply
Also can you pin your dependencies to a specific hash? Last I checked you cannot. A major advantage of pipenv/Poetry is that the lockfile protects you against compromise of the package manager; if someone swaps in a new binary for yourdep==1.2.3, the lockfile will fail to validate the hash and you'll get an error.
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] aiven|6 years ago|reply
[+] [-] marcosdumay|6 years ago|reply
The one large problem of those are forgetting about activating a venv, so I get I'll spend some time learning how to use direnv from aequitas advice.
[+] [-] 1337shadow|6 years ago|reply
Using --user here, no per-project virtualenv, used to have a virtualenv in ~/.env, where i keep everything up to date and have many -e installs from ~/src. For production deployments: --user in containers.
This means all my projects have to work with the same versions of dependencies.
This means all my dependencies have to work with the same versions of dependencies.
All my dependencies have to be up to date: all versions must be latest, or next to-be-released.
When don't support new versions of other dependencies: I deploy a fork and open a PR.
At such, I'm not spending time trying to make obsolete versions work, rather, I'm spending time making new versions work together.
My perception is that this strategy offers better ROI than all others which I have tried, not only for me, but for the whole ecosystem. That's also how I made my Python 2 to 3 transition, and have been using 100% Python 3 for a while (once you've ported one big software, porting others and their smaller dependencies is easy).
I'm extremely happy about this strategy and my life has just been better since I started working this way, and highly recommend it.
For my own (countless) packages: I use setupmeta which simplifies auto-updates (I think it succeeds what pbr tried to achieve).
[+] [-] dec0dedab0de|6 years ago|reply
This means all my dependencies have to work with the same versions of dependencies.
Then why not just install everything globally?
[+] [-] micimize|6 years ago|reply
With poetry+pyproject.toml, I have one file that is flat, simple, and readable. @sdispater has done incredible work on this project, and I hope they get more resources for development.
[+] [-] 1337shadow|6 years ago|reply
Quick question since you seem to know npm very well, is there a better solution than this to automate npm package publishing nowadays ?
sed -i "s/GIT_TAG/${CI_COMMIT_REF_NAME/v/}/" package.json
(I don't have this issue with Python's setupmeta)
[+] [-] korijn|6 years ago|reply
Anyway, has Poetry caught up to the dependency resolution provided by the pipenv lock command yet? Last time I tried it (~6 months ago), it couldn't produce working environments for my requirements.
BTW, for anyone who wants to educate themselves, read the issue description here and follow the links: https://github.com/pypa/pip/issues/988
[+] [-] tduberne|6 years ago|reply
Was the problem with dependency resolution you had similar to what Poetry describes in it README? Or are there other problems the README fails to mention?
The description they make got me interested, but up to now all alternatives I tried to build moderately complex python projects failed to deliver on their promises, so definitely interested in what you have to say.
[1] https://github.com/sdispater/poetry/blob/master/README.md#de...
[+] [-] guitarbill|6 years ago|reply
Been meaning to look into poetry. Not a huge fan of TOML, but hopefully all the tooling supports pyproject.toml now (I know black does, but not sure about flake8, isort, pylint, pytest, coverage). I know there's still some hacks required for tox + poetry though.
[0] https://github.com/pypa/pipenv/issues/1050
[+] [-] ben509|6 years ago|reply
I've tried similar configurations with poetry and it doesn't seem to have a problem with them.
[1]: https://github.com/pypa/pipenv/issues/2596
[+] [-] flyingdeertown|6 years ago|reply
[+] [-] epage|6 years ago|reply
iirc Poetry does have cross-platform lock files and a coherent story. I'm just blocked on some PRs due to the original author getting busy and not yet having delegated to lieutenants.
[+] [-] theptip|6 years ago|reply
[+] [-] angrygoat|6 years ago|reply
One solution at the moment is to run 'pip freeze' and put that in as your requirements file, but that very much feels like an 'and now I have a different problem' solution.
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] 0xbadcafebee|6 years ago|reply
The biggest problem with Python packages isn't a dependency manager, it's that Python developers don't reuse and extend existing projects. They just keep churning out new similar projects with different features. All of the most commonly used Python packages are basically one-offs that use a bunch of other one-offs. This creates a cycle of reinventing the wheel and friction in ramping up on existing technology, due to all the extra integration work and split development, and later lift-and-shift into completely different dependencies that implement the same thing. PyPI is awash with duplicate abandoned one-offs to the point that just trying to just find a useful module to do what you want can take a long time.
[+] [-] andyljones|6 years ago|reply
For people working outside of scientific Python: conda is a package and env manager maintained by a private company that's become the go-to because it's really good at handling binary dependencies.
[+] [-] HunOL|6 years ago|reply
[+] [-] epage|6 years ago|reply
[+] [-] micimize|6 years ago|reply
That said, "ejecting" a plain requirements.txt is possible on the alpha branch via `poetry export`
[+] [-] blurbleblurble|6 years ago|reply
I can't be too mad at pipenv, but all in all poetry is a better experience.
[+] [-] dissent|6 years ago|reply
[+] [-] mjs2600|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] ilovecaching|6 years ago|reply
Python makes huge sacrifices in readability and efficiency for a hypothetical win in writeability. It's also fundamentally at odds with the multicore world we're living in an will continue to live in, an issue that many people have tried to fix and failed at.
I can't count the number of times I've had to spend my entire night reading through an older Python service without type annotations, desperately trying to understand the types of inputs and outputs. It's just a mess, and the bias of programmers to believe that they are writing readable code (when in reality it's only readable to them) exacerbates the use of names that provide little context to solve the problem.
Python is awful, and it's an uphill battle to fix it. Asyncio can solve some performance issues, but negates the benefit of Python's simplicity by destroying the sequential mental model that Programmers can easily grok. It also requires a complete rewrite of libraries and a split between non-asyncio land and asyncio land.
Type annotations can make Python more readable and thus more maintainable, but gradual typing still leaves lots of unanswered questions when you're interfacing with untyped code.
Python is really not good unless you are using it to prototype or build something on your own. It's led to a world of slow, buggy software that is impossible to rewrite. It's downsides are easy to measure, but its benefits are difficult to quantify.
[+] [-] j88439h84|6 years ago|reply
[+] [-] bbmario|6 years ago|reply
[+] [-] zys5945|6 years ago|reply
[+] [-] sametmax|6 years ago|reply
[+] [-] julienkervizic|6 years ago|reply
I see though that it only supports pure python for building packages, does that mean that it doesn't build if you are dependent on compiled libraries?
Is there also a plan to add some of the functionality of bundling tools such as web-pack into this build phase? like automated css optimization, image compression... Could be handy for some django/flask projects.
[+] [-] j88439h84|6 years ago|reply
I hope the rest of the ecosystem can catch up quickly.
Tox and pip and and pex need full support for PEP 517/518.
[+] [-] sametmax|6 years ago|reply
I still wish poetry would support setup.cfg in parallel. It has been working for 2 years, is compatible with pip, setuptools and the whole legacy stack, and most fields are standards and documented.
[+] [-] oweiler|6 years ago|reply
[+] [-] markandrewj|6 years ago|reply
[+] [-] pbreit|6 years ago|reply
[+] [-] Areading314|6 years ago|reply
[+] [-] LaundroMat|6 years ago|reply
I like Poetry a lot; I hope platforms like Heroku will start supporting it soon.