As I mentioned the last time this came up, I recently converted a medium-sized (about 30,000 lines) system from Python 2 to Python 3. The problem is not the language changes. Those are minor, and most of the new forms worked in Python 2.7.9. It's the state of the external modules. Important old Python 2 modules have been discontinued, and their replacements are not used enough or tested well enough to use reliably. I kept finding bugs that should have been found in the first months of heavy use of a module.[1] It took an extra month of work to find and work around all the external module bugs. These aren't obscure modules; they're ones most large server-side applications would need.
Porting was at least possible. Two years ago, I took a look at doing this, and there weren't enough modules ready to even attempt it. Now there are. There is progress. Slow progress.
The download statistics tell the story: "We can see that in the past year Python 3.x has grown from roughly 2% of the total downloads from PyPI to roughly 5-6%."[2] That's why the external modules don't work reliably on Python 3. 95% of the package use is on Python 2. Python 2 use is probably higher than that, since those are download statistics; PyPI doesn't see packages downloaded long ago and still running.
There's extensive denial about this among Python fans. That's a big part of the problem. Another problem is that PyPI is just a link farm - unlike CPAN, it doesn't centralize code hosting, bug tracking, support forums, and quality control. This allows Python fans to be unaware of how buggy the packages are.
Number one reason why I haven't switched yet?... print. Admittedly is is the dumbest possible reason, it's irrational; I would never admit it in a face-to-face conversation; I would deflect to "something something third-party library...", but deep-down inside, if I'm really honest with myself, it's print. I want asyncio, I want better unicode support, but all I think about are SyntaxError's in what I __strongly__ consider to be valid python.
My stubbornness is rooted in the simplicity of print. It is so simple and stupid, why change it? Never has a production app relied heavily on print, it's just a dumb way to check shit while developing. I mean sweet jesus if it ain't broke right? I know I'm wrong in thinking this kind of thing happened everywhere in the 2->3 increment, but as I said earlier, I'm under no illusion of rationality. Python 3 broke hello world, and that just skeeves me out.
Eventually I'll write new stuff in 3, probably sooner rather than later especially with asyncio et all, but for now I'll stick with 2.
I'm seeing an awful lot of green there. Most of the things that are red (which might influence someone to use Python 2 on a greenfield project) have alternatives that are green.
5000 packages converted, 55,000 to go. If you can't port until somebody else does, then projects that depend on you can't convert. Guido talks about this, and says more than "looks green to me".
Some packages are ported but that does not mean people use Python 3. If you look at the download numbers it's pretty clear that they are not indicate actual use: https://caremad.io/2015/04/a-year-of-pypi-downloads/
the situation for greenfield projects is much better than it has been in past years. I would feel just fine about, for example, starting a new django web app in Python 3.
however, the vast overwhelming vast huge vast majority of software in the world is legacy systems. software lifecycles are long and the cost of migrating from Python 2 to Python 3 is very high for those systems.
Bloomberg's Open API is not on there and I think pretty much the only thing keeping me on 2.7. Don't know the ecosystem as well as others but imagine there will be more enthusiasm for 3.x in finance once that's done.
I remember, when Py3 first came out, everything was incompatible -- unnecessary incompatibilities like the u" notation for Unicode string literals that was dropped. Unnecessary incompatibilities in the C-extension-module implementation layer. And so on. The list of incompatibilities was just huge.
Later several of them where dropped, like the string literal trouble ... But than the trouble was already done. Many extension modules where not lifted to the new version, since the overhead was to big.
I think, many more projects would have adopted Py3, if more extension modules would support it.
The huge library of extension modules was always the strength of Python. Now we have many projects still running on Py2, because Py3 did ignore this strength.
I think, when it would have cared more for compatibility from the beginning, much of the damage done would have been avoided.
It raises some good points. It makes me think of other projects I've been a part of where the working, existing system was starved for resources in favor a new system which didn't exactly dominate in performance or capability.
I've been programming for almost 20 years but am new to Python. I'm currently working on an API for a product and testing it against 2.7 and 3.4. It's not the end of the world, and if it works for 3.4 it pretty much works in 2.7. Granted, it's not the world's most complex or deep piece of python, but is there a real problem in making 2.X code run on 3.X? Is it really that big of a change?
It's easy to get your own code to run on 3.x. The challenge is making sure all your C library dependencies run on it. And Python is nothing without its vast library ecosystem.
For me with the headaches with Unicode and international content, python 3 is great and well worth the switch. However, I know most don't have that big of a win to switch and there is really no business reason to switch.
If they'd solved the multithreading problem in Python3 so that you can get true multithreading that would have been a tremendously motivating force to get people to switch. But that didn't happen and it's really undercut a lot of the pressure.
The pydata subcommunity, the fastest growing and probably already the biggest one doesn't care about asyncio. A good multithreading solution on the other hand would have been decisive.
I used this analogy to describe the state of Python 2/3 to a friend who knows nothing about programming. Like many Python programmers, he too would continue using the old road.
Wow, that's remarkably unconvincing. I'll think of this the next time I hear of some complex ethical dilemma solved by asking a small child.
I really wish we as a community just set a date and ported. As it is right now there's almost zero chance that the project I'm working on will get ported to Python 3. I'm not sure what it'll take, but I really wish we were passed this stage.
This is the "problem" with not having things controlled by some central figure. Following the road example, the Dept of Transportation would simply open the new road, and close the old. You wouldn't have a chance to continue traveling down the old. This happens (to a large extent) with Java. Oracle opens the new road, then stops fixing potholes on the old one. You're not "forced" to move over, but eventually you stop wanting to fix your suspension.
That said, who is central enough to put an "end" date on Python and "force" people to move over?
Imagine you have a company that does something not related to software. You have some important business tools that are implemented in Python. What is your incentive to spent money and take the risk of new bugs porting to Python 3? It doesn't make business sense.
That's why there is still lots of COBOL in the world and why Python 2.x is going to live for many years, far past the 5 year predicted end of life. It will be far cheaper to make new releases of Python 2.x than to port the millions of lines of existing code.
Python 3 is nice, take a look at it. It probably will be easier porting than you think. However, don't feel like you are forced into moving to it. Python 2.x will continue to be around.
>I really wish we as a community just set a date and ported.
The thing is there is no "community". There's a core team, smaller teams around some projects with their own ideas, and a huge number of individuals and companies using Python, each with their own needs, use cases and timelines...
I understand the desire, but there is no "we" in that sense. Those of us using python are a "community" in that we have shared interests, but not in the sense that we have shared risks and rewards. Each person/team has to deal with the constraints of their own specific situation. So, the bottom line is the tool either gains organic adoption or it doesn't. I really don't think having a king who can decree a switch is the answer. The answer would have been not forcing the choice in the beginning. Now the answer is probably going to be that more people look at alternative ecosystems that haven't suffered from this sort of schism.
The metaphor doesn't appear to make sense - at first, the new road is only "slightly wider and speed limit is fractionally higher" but then later on it appears that the road needs "enough new lanes" for all the cars on the old road.
I thought the premise of building the new road was because the old road's ground wasn't good enough, not that there was too much traffic and the road needed more lanes.
Anyway, the whole thing hinges on the authors assertion that the new road isn't that much better, so it's really up to the informed developer/user to decide whether or not they take the new road.
I tried to stay away from a which is better battle. I think the numbers speak for themselves as to which road everyday users of Python are using (scripts to scrape cat pictures don't count ;).
One of the nice things about Python 2 is that there will be no breaking changes introduced by an episode of CADT syndrome in the language or library design.
I'll take the Python 3 road when the big stable distributions (Ubuntu LTS, RHEL & CentOS) start providing onramps to it by default, and include the packages we depend on in their default repositories.
I just can't get myself interested in rolling my own packages for the dozens of libraries I depend on, especially since they already "just work" with Python 2.
If you mean "adoption of Python 3" in the sense of new Python programmers choosing 3 over 2, then yes, there has been progress towards learning Python 3. It's pushed in all of the online "beginning Python" guides I've looked at as well. From everything I've read about it, learning 3 is the way to go, since if you learn 3 you can always code in 2 if necessary. When the time comes that you'll need to know 3, you'll already know it.
If you mean adoption by businesses and entities using it in their infrastructure, then no, the author is correct: We're not seeing much progress there.
I realise that 3 has the nice async stuff, I get a little miffed that certain improvements won't make it into a 2.7.x (or even 2.8).
Performance improvements for one.
It seems like the same attitude as the GNOME developers .. (disclaimer I use both GNOME and python, I must be a mashochist).
Why hasn't someone written a good 2 to 2/3 converter which also updates old c libraries? Wouldn't that speed up conversion considerably? It shouldn't be too hard to do it for the c libraries as well, or am I missing something?
Python 3's asyncio / co-routines is the reason to write any new project in Python 3.
The API is tiny, and the result is beautiful.
If you haven't seriously tried using it to implement something that really benefits from such computer science innovation, then you have simply seen nothing of Python 3 yet. There are many types of logic that are vastly simplified with such a programming paradigm. Faster to code, and far simpler to debug. By orders of magnitude. One example you could try if you cannot envision one is to implement a high performance network protocol. I for instance implemented the SSH protocol from scratch (RFCs) in a matter of weeks. I though it would take months before I tried, especially since I had never programmed in Python before! As I said, the API is tiny. Learning it (Python and Asyncio at once) was so easy and ended up taking such an insignificant amount of time compared to the vast quantities of time and headache saved, that, well, I just had to write this post with all that extra time and energy!
Check it out! You will find that it is beyond worth just addressing whatever fears or problems you have with making the switch.
Im sure most people will start to use Python 3 for one of those large new features, but to me its all the small QoL changes for the standard library which I constantly see when searching for tricky problems and finding them in SO.
asyncio seems to me like it should be the biggest killer app for Python 3 - a universal replacement for the sprawling, confusing, and opposing async options out there for Python 2.x (Twisted, Tornado, gevent...).
I'm hoping that aiohttp, or a similar library, further matures and we start seeing apps built on it. Developing asynchronous HTTP code in Python 2.x is significantly harder than throwing together a Node app, where things are "asynchronous by default" (unlike, say, Flask or Django). Having a universal, built-in event loop removes a huge point of confusion and learning for async development in Python, and can hopefully bring interest from developers who would normally look to Node.
Is there a thorough getting started guide for asyncio floating someplace on the web? I tried to pick up asyncio a few weeks ago. The official documentation assumes a lot of background that I don't possess, despite being a long-time Python user. Each method and class seems to be documented, but conceptual and end-to-end examples were much less common. Thanks in advance!
I don't think so especially since it still uses bytes to output to a webserver, even go is way better for this kind of tiny library.
For bigger projects i would still go the java route especially since there is karyon from netflix.
[+] [-] Animats|11 years ago|reply
Porting was at least possible. Two years ago, I took a look at doing this, and there weren't enough modules ready to even attempt it. Now there are. There is progress. Slow progress.
The download statistics tell the story: "We can see that in the past year Python 3.x has grown from roughly 2% of the total downloads from PyPI to roughly 5-6%."[2] That's why the external modules don't work reliably on Python 3. 95% of the package use is on Python 2. Python 2 use is probably higher than that, since those are download statistics; PyPI doesn't see packages downloaded long ago and still running.
There's extensive denial about this among Python fans. That's a big part of the problem. Another problem is that PyPI is just a link farm - unlike CPAN, it doesn't centralize code hosting, bug tracking, support forums, and quality control. This allows Python fans to be unaware of how buggy the packages are.
[1] https://news.ycombinator.com/item?id=9378898, search for "quality control" [2] https://caremad.io/2015/04/a-year-of-pypi-downloads/
[+] [-] bcj|11 years ago|reply
[+] [-] jefb|11 years ago|reply
My stubbornness is rooted in the simplicity of print. It is so simple and stupid, why change it? Never has a production app relied heavily on print, it's just a dumb way to check shit while developing. I mean sweet jesus if it ain't broke right? I know I'm wrong in thinking this kind of thing happened everywhere in the 2->3 increment, but as I said earlier, I'm under no illusion of rationality. Python 3 broke hello world, and that just skeeves me out.
Eventually I'll write new stuff in 3, probably sooner rather than later especially with asyncio et all, but for now I'll stick with 2.
[+] [-] PhantomGremlin|11 years ago|reply
People say "print became a function", but that doesn't do it justice. Your quip is infinitely better.
[+] [-] reuben364|11 years ago|reply
[+] [-] stuaxo|11 years ago|reply
[+] [-] koenigdavidmj|11 years ago|reply
I'm seeing an awful lot of green there. Most of the things that are red (which might influence someone to use Python 2 on a greenfield project) have alternatives that are green.
[+] [-] RogerL|11 years ago|reply
5000 packages converted, 55,000 to go. If you can't port until somebody else does, then projects that depend on you can't convert. Guido talks about this, and says more than "looks green to me".
[+] [-] the_mitsuhiko|11 years ago|reply
[+] [-] metaphorm|11 years ago|reply
however, the vast overwhelming vast huge vast majority of software in the world is legacy systems. software lifecycles are long and the cost of migrating from Python 2 to Python 3 is very high for those systems.
[+] [-] teddyh|11 years ago|reply
[+] [-] whatok|11 years ago|reply
[+] [-] baldfat|11 years ago|reply
[+] [-] PythonicAlpha|11 years ago|reply
I remember, when Py3 first came out, everything was incompatible -- unnecessary incompatibilities like the u" notation for Unicode string literals that was dropped. Unnecessary incompatibilities in the C-extension-module implementation layer. And so on. The list of incompatibilities was just huge.
Later several of them where dropped, like the string literal trouble ... But than the trouble was already done. Many extension modules where not lifted to the new version, since the overhead was to big.
I think, many more projects would have adopted Py3, if more extension modules would support it.
The huge library of extension modules was always the strength of Python. Now we have many projects still running on Py2, because Py3 did ignore this strength.
I think, when it would have cared more for compatibility from the beginning, much of the damage done would have been avoided.
[+] [-] fullwedgewhale|11 years ago|reply
I've been programming for almost 20 years but am new to Python. I'm currently working on an API for a product and testing it against 2.7 and 3.4. It's not the end of the world, and if it works for 3.4 it pretty much works in 2.7. Granted, it's not the world's most complex or deep piece of python, but is there a real problem in making 2.X code run on 3.X? Is it really that big of a change?
[+] [-] et1337|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] cyberpanther|11 years ago|reply
[+] [-] pyre|11 years ago|reply
[+] [-] msandford|11 years ago|reply
[+] [-] c22|11 years ago|reply
[+] [-] elyase|11 years ago|reply
[+] [-] sp332|11 years ago|reply
[+] [-] coldtea|11 years ago|reply
[+] [-] jessaustin|11 years ago|reply
Wow, that's remarkably unconvincing. I'll think of this the next time I hear of some complex ethical dilemma solved by asking a small child.
[+] [-] wowzer|11 years ago|reply
[+] [-] wspeirs|11 years ago|reply
That said, who is central enough to put an "end" date on Python and "force" people to move over?
[+] [-] nas|11 years ago|reply
That's why there is still lots of COBOL in the world and why Python 2.x is going to live for many years, far past the 5 year predicted end of life. It will be far cheaper to make new releases of Python 2.x than to port the millions of lines of existing code.
Python 3 is nice, take a look at it. It probably will be easier porting than you think. However, don't feel like you are forced into moving to it. Python 2.x will continue to be around.
[+] [-] gregor7777|11 years ago|reply
[+] [-] coldtea|11 years ago|reply
The thing is there is no "community". There's a core team, smaller teams around some projects with their own ideas, and a huge number of individuals and companies using Python, each with their own needs, use cases and timelines...
[+] [-] markbnj|11 years ago|reply
[+] [-] greggyb|11 years ago|reply
[+] [-] jamesfe|11 years ago|reply
I thought the premise of building the new road was because the old road's ground wasn't good enough, not that there was too much traffic and the road needed more lanes.
Anyway, the whole thing hinges on the authors assertion that the new road isn't that much better, so it's really up to the informed developer/user to decide whether or not they take the new road.
[+] [-] idank|11 years ago|reply
[+] [-] andrewstuart|11 years ago|reply
Another person asserting that Python 3 is going nowhere, based on their opinion and nothing more, no science, no facts, no figures.
Without any reasonable justification for this opinion then I call this complete FUD and to be entirely disregarded.
[+] [-] pnathan|11 years ago|reply
[+] [-] falcolas|11 years ago|reply
I just can't get myself interested in rolling my own packages for the dozens of libraries I depend on, especially since they already "just work" with Python 2.
[+] [-] ak217|11 years ago|reply
What planet does the author live on?
[+] [-] morganvachon|11 years ago|reply
If you mean adoption by businesses and entities using it in their infrastructure, then no, the author is correct: We're not seeing much progress there.
[+] [-] coldtea|11 years ago|reply
[+] [-] stuaxo|11 years ago|reply
It seems like the same attitude as the GNOME developers .. (disclaimer I use both GNOME and python, I must be a mashochist).
[+] [-] Klockan|11 years ago|reply
[+] [-] thuffy|11 years ago|reply
The API is tiny, and the result is beautiful.
If you haven't seriously tried using it to implement something that really benefits from such computer science innovation, then you have simply seen nothing of Python 3 yet. There are many types of logic that are vastly simplified with such a programming paradigm. Faster to code, and far simpler to debug. By orders of magnitude. One example you could try if you cannot envision one is to implement a high performance network protocol. I for instance implemented the SSH protocol from scratch (RFCs) in a matter of weeks. I though it would take months before I tried, especially since I had never programmed in Python before! As I said, the API is tiny. Learning it (Python and Asyncio at once) was so easy and ended up taking such an insignificant amount of time compared to the vast quantities of time and headache saved, that, well, I just had to write this post with all that extra time and energy!
Check it out! You will find that it is beyond worth just addressing whatever fears or problems you have with making the switch.
[+] [-] belorn|11 years ago|reply
[+] [-] avolcano|11 years ago|reply
I'm hoping that aiohttp, or a similar library, further matures and we start seeing apps built on it. Developing asynchronous HTTP code in Python 2.x is significantly harder than throwing together a Node app, where things are "asynchronous by default" (unlike, say, Flask or Django). Having a universal, built-in event loop removes a huge point of confusion and learning for async development in Python, and can hopefully bring interest from developers who would normally look to Node.
[+] [-] Scramblejams|11 years ago|reply
[+] [-] vtbassmatt|11 years ago|reply
[+] [-] themckman|11 years ago|reply
[+] [-] merb|11 years ago|reply