top | item 13144713

Why I'm Making Python 2.8

241 points| cpeterso | 9 years ago |naftaliharris.com | reply

379 comments

order
[+] lloeki|9 years ago|reply
> And the majority of Python code written does not run under any of the 3.x interpreters. This makes it harder for its users to be productive.

What a load of bollocks. For new projects this only matters if libraries aren't ported, which they are for the most part. For old projects, either you're in a situation where you can spend time porting your code to Python 3, or you don't; but as TFA mentioned pep-404, the writing has been officially on the wall ever since 2011 so at that point you have to admit you did choose to incur tech debt and do nothing about it, so the claimed loss of productivity is on you.

> Unlike 2.7 code, Python 2.8 wouldn't be able to guarantee exact 3.x compatibility, since there are some python scripts that will run under both Python 2.7 and Python 3.x but produce different output, and Python 2.8 chooses the 2.7 behavior in these cases.

What a terrible, terrible situation. Now you'll have "python" code that will neither run on 2.7 nor run compliantly on 3.x. As for the latter, please explain how that will alleviate anything on the following point, since behaviour at runtime will be subtly different:

> adding these remaining Python 3 features would greatly simplify running code targeting Python 3, and allow people to use Python 2.8 to run a mix of Python 2 and 3 code.

I don't know what recourse the PSF has but maybe they should even go all in and defend the "Python" name so as to prevent confusion and stop a potential community fracture. Just call it anything else but "Python 2.8" is not Python.

[+] ceronman|9 years ago|reply
> I don't know what recourse the PSF has but maybe they should even go all in and defend the "Python" name so as to prevent confusion and stop a potential community fracture. Just call it anything else but "Python 2.8" is not Python.

This! A thousand times! I love open source and free software. I absolutely love the fact that you can fork the code and adapt it to your needs. If you find others who like it great! But please, don't use the Python name! It will create more confusion than help. This fork with a different name is 100% fair in my opinion.

[+] VanL|9 years ago|reply
(Replying to the top-ranked comment so that as many people as possible see it)

While I wish Naftali well in his efforts - I have a private Python-derived language myself! - this is not "Python 2.8." For trademark purposes, "Python" is only what is released or endorsed by the PSF.

We have already reached out to Naftali and asked him to change the name of his project and update this blog post accordingly.

Obviously, though, this is someone who cares a lot about Python, so let's be sure not to rain down on him with a lot of scorn; I admire that he was willing to sit down and 'scratch his own itch.'

Source: I am the General Counsel of the PSF.

[+] coldtea|9 years ago|reply
>What a load of bollocks. For new projects this only matters if libraries aren't ported, which they are for the most part. For old projects, either you're in a situation where you can spend time porting your code to Python 3, or you don't; but as TFA mentioned pep-404, the writing has been officially on the wall ever since 2011 so at that point you have to admit you did choose to incur tech debt and do nothing about it, so the claimed loss of productivity is on you.

I call BS (to counter your "bollocks").

Whether the "writing was on the wall" or not, doesn't change the fact that people had to actively port their old code if they wanted it to run on 3.

Sometimes that code could run into the tens of thousands (or even millions for large companies) of lines.

And why would they do it (and at a great cost and time effort)? For the marginal improvements Python 3 brings?

The "writing has been on the wall" is not an excuse, it's mostly blackmail ("port or else you wont run on 3, and we'll stop the 2.x line"). And most people didn't (and shouldn't) fall for that.

[+] ChuckMcM|9 years ago|reply
And this captures the good and the bad of open source all so succinctly.

Here we have a person (the author) who has rejected the path that an open source project has taken, and invested the time and energy to move the source along a path they prefer.

In this particular case, there is a natural constituency of people who share that desire but are unable or unwilling to put in the effort to push the source down the path.

When there is critical mass, that group forks off and begins to bring other people along to the alternate path.

At that point the people who endorsed the change in direction come out in force to yell at these people who aren't doing what they are supposed to and threaten them and implore a higher power to emasculate their effort.

Sometimes that works, sometimes it doesn't. But it always results in massive amounts of confusion when someone new comes to the community and sees these two different paths for the same thing and can't really figure out why they are different.

Further because there is no mechanism for "righting" the ship as it were, the diverging paths lead to a lot of wasted time and effort on everyone's part. This happens to be a Python fork but its happened to window systems, video codecs, graphics libraries, data bases, hell even C compilers.

The nice thing about a Cathedral is that the Pope keeps the Cardinals toeing the one and only line.

Reminds me a lot of the Perl 5 / Perl 6 debates.

[+] lima|9 years ago|reply
> I don't know what recourse the PSF has

They own the Python trademark, so they can make him stop using it.

In fact, I really hope that they do. This project does no good.

[+] ianamartin|9 years ago|reply
The Python Software Foundation has an obligation to defend the trade name. If it fails to exercise it, then it can be denied in the future.

I don't expect litigation because that's not how the community rolls. I expect a polite message from gvr to the author asking to change the name.

If that fails, the PSF has no choice. It's use it or lose it.

[+] smoyer|9 years ago|reply
I came here to say exactly this ... you're better off upgrading your systems as you can. There's one point missing above - do you really want to use a version of Python that's maintained by one person and of unknown quality? You're better off staying on 2.7 if you can't afford to upgrade.
[+] epberry|9 years ago|reply
Yes totally agree. In the end this won't matter because the momentum of the ecosystem is so great at this point but it just baffles the mind that someone would think this is a good idea, especially with the "can't run on 2.7 or 3.x" situation.
[+] naftaliharris|9 years ago|reply
> the writing has been officially on the wall ever since 2011 so at that point you have to admit you did choose to incur tech debt and do nothing about it, so the claimed loss of productivity is on you.

When Python 3 was released, it offered Python users a trade: In exchange for a productivity loss (porting your Python 2 code), you'd get a productivity gain (new features in Python 3 and removed cruft). Some projects and companies thought this was a good trade, and have upgraded over the years, and many have not, and haven't. The interpreter I've been working on tries to improve on the terms of that deal for people who have not switched to Python 3.

> What a terrible, terrible situation. Now you'll have "python" code that will neither run on 2.7 nor run compliantly on 3.x.

That's the point, yes. Obviously any interpreter that's backwards compatible with 2.7 but includes new features from 3.x is going to let people write code that doesn't run under 2.7 or 3.x. But what does it matter if your code doesn't run under interpreters that you aren't using and don't intend to use?

> Just call it anything else

I'll change the name.

[+] staticassertion|9 years ago|reply
> What a load of bollocks. For new projects this only matters if libraries aren't ported, which they are for the most part.

Except when they aren't. And then what?

I've run into this multiple times. Sometimes there's a branch of the project for 3 that's underway, and I sit and wait. Other times it means dropping the project or committing to reimplementing a library.

[+] hartator|9 years ago|reply
I think after 5 years we should came to the realization that Python 3.x is not the future of Python. You can't blame people to try to find solution.
[+] greyman|9 years ago|reply
> What a terrible, terrible situation. Now you'll have "python" code that will neither run on 2.7 nor run compliantly on 3.x.

I don't say you are wrong, but I am afraid this situation looks "terrible" only to people who do care about 3. If someone doesn't care about it and thinks that he can survive with never porting to 3 or start using it, for those people the situation isn't that terrible...from that perspective, his 2.7 language evolved to next step, and he know that new features can be used if he upgrades from 2.7 to 2.8.

[+] dismantlethesun|9 years ago|reply
> What a load of bollocks. [Snip] For old projects, either you're in a situation where you can spend time porting your code to Python 3

The majority of Python code is old projects, just like every other established language. This might change in the future as I now see people starting new projects in Python 3, but if your company is older than 5 years old, then there is a good chance that you started with Python 2 simply because at the time of creating your codebase a whole lot of libraries weren't ported to Python 3.

[+] susan_hall|9 years ago|reply
I currently work for a client who has decided to shift away from PHP and towards Python. They had a monolithic PHP app with perhaps 250,000 lines of code. Now we are developing a series of Python apps in the microservices style. We've decided to develop everything as Python 2.7. We are not looking at Python 3.x. There are a few reasons. Some libraries that we want are in Python 2.7. And Amazon only supports 2.7. And we are not wild about Python 3.x's attempt to imitate a classical object oriented style.

We would look very closely at a Python 2.8, if it existed.

[+] dagss|9 years ago|reply
Software typically evolves and grows and changes over decades. There very often isn't any point where you can go "for our next project we choose X". Each new project is a feature using 90% of some existing huge codebase.

"Starting fresh" is something many (most?) companies simply never does, over decades. (And if they attempt it is often an all out disaster..)

2011 is fairly recent in this context, and many popular libraries were not available on Py3 until much more recent than that, even if you have the rare luxury of starting fresh.

[+] jl6|9 years ago|reply
Name suggestion: Pythoff.
[+] e98cuenc|9 years ago|reply
That has been official since 2011, but even today you may need some lib only available in Python 2.x and thus may need to start a new project in 2.7. I started several python projects since 2011 knowing that it was a dead end, but my hands were tied. At the time there was not even a working wsgi spec, and no web frameworks for 3.x, and the first ones has an awful performance (2.7 is bad enough).

Only recently 3.x has become a viable alternative. I for one welcome this 2.8 fork.

[+] upofadown|9 years ago|reply
well Python 3 itself was a version hijack so it might be a little late to complain about others doing the same thing.
[+] yladiz|9 years ago|reply
As others have said, maybe this project fixes some actual problems and backports some features from 3, but this isn't "Python". Beyond the fact that Python is a trademark of the Python Software Foundation, Python is more than the language, it's the community and the tools (as with every programming language). So while there are some vocal people that really dislike Python 3 (either in part or wholly), my understanding is that with the planned phase out of Python 2 and Python 2 only receiving bug fixes at this point, much of the industry is transitioning to Python 3 (either currently doing so or planning to) and so it seems relatively fruitless to attempt to build upon Python 2. I personally think the effort put into this would be much better spent making tooling around Python 2 to 3 transformations.

I also think it's pretty irresponsible of the author to call this Python 2.8, because it may cause confusion to developers unfamiliar with the history and come from a tutorial that is still in Python 2 (it does show up on the first page of Google for me). It's also especially irresponsible and hubristic to attempt to make a language that is seemingly compatible with both Python 2 and 3, because 1) I trust that if it was possible Guido and the other developers would have made it, and 2) it can cause significant confusion when code doesn't work when it hits an edge case, and then the whole tooling around it can't be guaranteed to work. The last thing I'd want in my programming language is unaccounted for ambiguity.

[+] naftaliharris|9 years ago|reply
> It's also especially irresponsible and hubristic to attempt to make a language that is seemingly compatible with both Python 2 and 3, because 1) I trust that if it was possible Guido and the other developers would have made it

It is possible actually, that's kind of the point! The interpreter I've been working on passes the 2.7 unit tests (i.e. those in Lib/test/), and as well as unit tests for the new features that have been backported from Python 3.

Even if you don't believe me, it's interesting to note that, e.g., while Python 3.0 was being developed, function annotations and keyword-only arguments coexisted with tuple unpacking. I built the code and ran it myself, in fact: https://twitter.com/naftaliharris/status/784421498291310592. Tuple unpacking was actually removed later, introducing the backwards incompatibility after the new functionality had been added. Timeline:

Oct 2006, keyword-only arguments.

Dec 2006, function annotations.

Mar 2007, removing tuple unpacking.

There was also a promising backport of keyword only arguments to CPython 2.6 (!) that was never merged, (http://bugs.python.org/issue1745), due to lack of follow-through.

[+] dismantlethesun|9 years ago|reply
I don't know about that. It forked a Python compiler, and is fully interoperable with 100% of Python 2 code, and much of Python 3 code. It's even compatible with Python C extensions.

Why isn't it a valid Python compiler?

To me, the whole morass about trying to end-of-life Python 2 is a bit silly. People have gotten emotional about the situation.

On one side, people like Zed Shaw are calling the Python maintainers 'evil' and claiming conspiracy.

On the other side, people are calling companies using Python 2, 'lazy' and claim they're a threat to the ecosystem.

Yet elsewhere, C is still being written in all of its various year-specific formats, and people end up using 'old' versions simply because they join pre-existing projects or need to totally interface with something that's written in an 'old' version.

Python is an extablished language, it's likely that 10 years from now there will still be Python 2 codebases going strong.

[+] greyman|9 years ago|reply
> much of the industry is transitioning to Python 3 (either currently doing so or planning to)

Except it isn't.

[+] yoz-y|9 years ago|reply
I make software that people can write plugins for in Python. After months, years of struggle we finally dropped support for Python 2 because our small team could not bear the overhead of maintaining two bindings. We work a lot with researchers in signal processing domain and we have hard time as it is to get people to use Python 3. Please, do not put obsolete software on life support.
[+] AnonymousPlanet|9 years ago|reply
If you have several large software products rolled out and churning away at hundreds of customer sites, moving from Python 2.x all the way to 2.7 alone is a slow and tedious process of tests and deliberations. And we're still not talking about going all the way to 3.x which breaks things in even more new and exciting ways.

So scoff all you want, but Python 2.x isn't going away that soon.

[+] jprzybyl|9 years ago|reply
Obsolete is a funny word to use. In this case, it would mean that Python 2 is in good working order, but is no longer wanted. That's bound for a flame war, because:

- There is a community that wants it (largely enterprise).

- The Python team does not want it.

A less controversial word is deprecated - the Python team is discouraging use of Python 2, but not prohibiting it's use or development. That's fair, and if you read this page:

https://wiki.python.org/moin/Python2orPython3

they are not very opinionated about it, largely saying "Use 3, unless you can't, then use 2 and start trying to migrate, unless you can't, then just use 2."

I will say, not to give somebody a bad day but, 2.8 seems like a bad idea. Currently python's development has still largely been a straight line, which is good for transitioning, but 2.8 would cause a fork. It would give a lot of people a short-term win for a long-term lose. Better not to tempt people.

[+] coldtea|9 years ago|reply
>Please, do not put obsolete software on life support.

Regarding adoption, it's 3.0 that's obsolete, and 2.7 that's vibrant. Even for new code (they conveniently only count totally greenfield projects, but most new code is written in fact to work with established 2.x codebases under Python 2, not as a totally greenfield project).

[+] wfunction|9 years ago|reply
> Please, do not put obsolete software on life support.

Downvoting because Python 2 is anything but obsolete. People still love using it.

[+] scriptproof|9 years ago|reply
There is nothing bad in making a newer version of an older language. But I agree they must use a more distinctive name.
[+] SeanDav|9 years ago|reply
> "Please, do not put obsolete software on life support."

The vast majority of current development out there is in python 2.x. A small minority use Python 3.x. How on earth does this make Python 2.x obsolete?

This is like saying Perl 5 is obsolete, just because Perl 6 is out and completely ignore the realities of the real world use of the products.

[+] toggafgnadx|9 years ago|reply
The worst thing about open source is that people can do stupid stuff with your software.

If you're going to create this abomination, at least do us all a favour and DON'T call it Python. Call it Retardython or something. I don't want to imagine people coming into the official support channels and claiming they are using "Python 2.8", then other people lecturing them about what that software really is, etc. Sounds like a horrible waste of time. (Source: I spend many hours a week helping fellow Python users.)

[+] gigatexal|9 years ago|reply
Python programmers and companies with python code should spend the time and effort to move to python 3 instead of spending that time and effort to backport stuff to python 2 because python 2 is deprecated and the future is python 3. Python 3 I think people and businesses with python 2 code would be better off moving their code bases to python 3 instead of doing things like this.
[+] jzwinck|9 years ago|reply
Companies with Python code are probably better off keeping their working, tested code than switching to an incompatible interpreter and set of libraries which among other things will print "b'Hello',b'World'" into their mission critical CSV files.

Yes, the built in csv module really does that in Python 3.

[+] wfunction|9 years ago|reply
> Python programmers and companies with python code should spend the time and effort to move to python 3 instead of spending that time and effort to backport stuff to python 2 because python 2 is deprecated and the future is python 3.

I hate Python 3's removal of the (lambda (key, value): blah) tuple unpacking syntax, and the forcing of parentheses for print statements. They might seem minor but they aren't for me. So I'm not at all eager to move to version 3 and don't really see any benefit. Not sure if those who aren't migrating feel the same way, but I wouldn't be surprised if some of them do.

(Edit to address comment below: There are more issues I have with Python 3. It allows more bugs to slip through, for instance. I actually particularly like a comment I just wrote, so I'll link to it here: https://news.ycombinator.com/item?id=13145299 Do note that this was added after the reply below.)

[+] mathgenius|9 years ago|reply
People don't seem to be considering the possibility that a stagnant python 2.7 may actually be a reason to like that version of the language. I must admit it is nice to not have those oh-so-keen python developers messing with my favorite language.

However, I recently had a gig working in py3. Apart from screwing up every single print statement for a long time, it was entirely drama-free, and actually pretty great. There really is a difference between living and dead languages I think. 2.7 is the latin of python.

[+] naftaliharris|9 years ago|reply
Author here. Imagine my surprise when I got back from a day of sightseeing (I'm on vacation in Spain) and saw that this had blown up. I had intended to "release" this project after New Years, after I'd gotten back and a week or two after 3.6 is released [1], and didn't expect this to get picked up since the project has been on Github for over a year (although inactive for much of that time) and since my blog usually doesn't get much traffic.

A lot of people here have strong opinions about the name "Python 2.8". I don't mind changing it, and intend to do so, (https://github.com/naftaliharris/python2.8/issues/47). I picked it initially since when talking with friends about this project it conveyed pretty darn immediately what the project is and does. I'd be very keen to hear people's suggestions for alternate names!

For those of you with 2.7 codebases or projects, I'd be extremely interested in hearing about whether you were able to get this interpreter to run your code. Personally, the biggest challenges I've had so far are with dependencies that check for `sys.version_info[:2] == (2, 7)` as opposed to something like `sys.version_info[0] < 3`. But I'd be very interested in other people's experiences, particularly with larger codebases.

[1] A minor and somewhat pedantic point: The interpreter I've been working on includes PEP 515 (underscores in numeric literals), which is new in 3.6. I didn't think it was right for me to "take credit" for this new feature before it was even out in Python 3.6. Obviously, the real credit for this feature existing (in 3.6 or in any interpreter) goes to the CPython core devs, and especially Georg Brandl.

[+] cesarb|9 years ago|reply
Reading a thread from the last time something like that was proposed (although the only change in that proposal was that 2.8 would, on Windows, use the C runtime from a more recent MSVC compiler), I found a relevant argument (https://mail.python.org/pipermail/python-dev/2013-November/1...) against this "Python 2.8" idea:

What if more than one person did that? (paraphrasing Raymond Chen's "What if two programs did this?")

There can be only one official Python 2.8, but since it will never exist (see PEP 404), there can be many unofficial mutually incompatible "Python" 2.8 implementations. Therefore, calling it "Python 2.8" is a bad idea.

[+] dagss|9 years ago|reply
I see many people vigorously defending Py3 but I wonder how many of these have a paying-the-bills kind of job. Where you would look at the cost of porting a large project to Py3 and get an answer like half a million USD (easily). Do you go "of course we do that, that money is easily recouped with the added programmer productivity of Py3"? No chance.

So the question is do you want to basically light that money on fire, or just keep your perfectly fine Py2.7 code running and maintained another few years.

More discussions of the monetary value of programming languages please. What is "correct" or "right" isn't all that interesting to many, for good reasons.

Kudos to this project and hope it can set us on a saner migration path to Py3. (Should totally change the name though.)

[+] dom0|9 years ago|reply
> I've been working on Python 2.8 (not an official Python release)

It is very dishonest to call it Python 2.8 then.

[+] eej71|9 years ago|reply
It's too bad the python 3.x fans can't see this as feedback about how difficult it for users of 2.x to upgrade to the latest and greatest. Many 2.x folks have sprawling code bases and complex operational needs. The 3.x advocates seems to consistently ignore that.

Shoot the 2.8 messenger all you want for choosing to call it Python 2.8, but don't dismiss the issue that drives thoughtful people to get value out of this strategy.

[+] mathgenius|9 years ago|reply
Christian Tismer tried this a couple of years ago [1].

I guess the intention was a bit different: he wanted to have stackless features in python. It's not clear to me the reason he decided to back down, whether because of licensing issues or just because the other python developers didn't like it.

[1] http://www.stackless.com/pipermail/stackless/2014-January/00...

[+] silent1mezzo|9 years ago|reply
What another load of crap. Call it something else but this isn't Python. I'd never use this because it's not official. Who knows if or how long it'd be supported for or if any backdoors would/could be introduced.

I've scheduled time this year for my teams project to update to Python 3. It's expensive in the short term but in the long term we get continued support and new features which is a huge win.

[+] nzjrs|9 years ago|reply
I can't fault a single thing in his justification. This should have been the approach to modernising python all along.
[+] maxander|9 years ago|reply
So when you see a project written in Python 2.8, you'll know that it contains lots of legacy code under active development by developers who are gung-ho about fancy language features, but that this team hasn't had the time/resources/gumption to do a code overhaul anytime in the past six years.

And also that you yourself, and any users of your own project, will need to run it with this guy's own homebrewed version of Python in order to know it will behave correctly.

Useful?

[+] grx|9 years ago|reply
I don't think this is a good idea simply because it reduces the need to upgrade your library from 2.x to 3. It's a clear cut and a good chance to weed out unmaintained libraries
[+] MaxfordAndSons|9 years ago|reply
Slightly tangential, but: I've noticed an interesting parallel between the 2.7 / 3.x partisan divide, and the US political partisan divide. In both cases, as partisan passions have increased without relief, there's both A) an increasing unwillingness to agree on basic facts about reality essential to the debate, and B) increasing presumptions of bad faith on the part of their opponents.

Examples of A in this discussion:

- Disagreements about the degree of library support for 3.x

- '' the ease/value of porting from 2 to 3

- '' the rate of industry adoption of 3 for new projects

- '' the degree to which people are driven away from Python entirely because of the version situation

Examples of B in this discussion:

- Claims of paternalism on the part of GVR/ the PSF in pushing 3

- Claims of unreasonable/emotional attachment to 2 by partisan devs

- Claims of willful distortion of facts by both sides (see A)

- Claims that the writing is on the wall for 2, because usage of 3 is supposedly accelerating

- Claims that the writing is on the wall for 3, because it's supposedly taken too long to drive not enough adoption

It seems outrageous to suggest that the differences between 2 and 3 are anywhere near as significant as the differences between, say political conservatism and liberalism, and yet the level of partisanship seems nearly the same. How did it come to be like this?

[+] dekhn|9 years ago|reply
Thanks for doing this! I took a look at the arguments underlying the proposed switch to Python 3 and came to the conclusion that they did not meet the necessary threshold of effort to switch. Instead, I want to see every feature in Python 3 backported into 27. I don't think the unicode change can be accomodated, but 99% of the rest of the new features in 3 can.
[+] js8|9 years ago|reply
IMHO, Jython 3 would be more useful than Python 2.8, but it's your project.

We have a project at work and the only reason we need to support 2.7 is because we want to run under Jython.

Anyway, I would like to hear other people's use cases for Python 2.7.

[+] rini17|9 years ago|reply
The discussion is a nice overview how py3k crowd just steamrolls over any discussion that there are indeed inherent difficulties with forcing lossy conversion of imperfect outside input.

See example in the fine manual: https://docs.python.org/3.7/library/urllib.request.html#exam...

You are only told "we know python.org uses utf-8 so just decode it as utf-8." No further discussion, no pointers are provided how to correctly fetch an URL with text content into a string. Even small convenience function that at least tries to look on Content-Type: header would help here!

I am well aware that "in py2k it just worked" was mostly an illusion. But honestly, is the situation above an improvement?