Goto in Python

[david-given]

Hey, awesome! And it produces a true goto as well, working on the bytecode level. I wonder if there's a way to remove the extra nops?

This is really, really useful for doing language-to-language translation. Without the ability to arbitrarily branch from basic block to basic block, it's possible for the basic block graph produced by a program in one language to be simply inexpressible in your target language. You end up having to fake it with a while loop and switch statement, which has horrible performance implications.

(Emscripten has some exotic algorithms to try and express the source program's basic block graph using the target language's structured programming constructs. This is vitally important to make goto-less Javascript perform well, but it can't work in all cases, and some C functions will force Emscripten to fall back to loop-and-switch. And, of course, these functions are typically the ones which have been carefully hand-optimised for speed...)

[wallunit]

> I wonder if there's a way to remove the extra nops?

Yes, technically it would be possible to remove the code instead injecting NOPs. But then I'd have to adjust the jump targets. However, I don't think these NOPs are too bad. Note that goto jumps directly after the NOP ramp of the label, and the NOPs of the goto itself are never reached. The only scenario where NOPs are actually seen by the interpreter is when the natural code path visits a label.

EDIT: Instead re-assembling the bytecode and adjusting jumps, I went to use JUMP_FORWARD(3) instead 7 NOPs now. Note that there are still 4 NOPs left to fill the gap, these however are never executed as they are skipped by the preceding jump instruction. https://github.com/snoack/python-goto/commit/2b0f5e5069cbb88...

[masklinn]

> I wonder if there's a way to remove the extra nops?

Sure you just have to recompute all the jump destinations and patch the jumps themselves.

[mrsirduke]

The technique used to implement this is rather interesting, and the implementation itself[1] is very understandable.

I'm sure neither should be used in a production project.

[1]: https://github.com/snoack/python-goto/blob/master/goto.py

[simgidacav]

https://www.xkcd.com/292/

Sorry, I had to.

[wallunit]

Awesome, I added it to the README!

[zarex]

Dammit, you beat me to it :)

[tveita]

This is cute but of course horribly unsafe since it doesn't respect control statements.

Some examples you'd expect to work that will crash the interpreter:

  @with_goto
  def fun1():
   while True:
    for x in [1]:
     goto .next
    label .next
  
  @with_goto
  def fun2():
   goto .next
   for x in [1]:
    label .next
  
  @with_goto
  def fun3():
   while True:
    try:
     goto .next
    finally:
     print('skipped')
    label .next

[kazinator]

Note how Lisp's TAGBODY/GO doesn't have this problem.

  (loop
    (tagbody
      (loop for x in '(1 2 3)
            do (go next))
    next
      (print 'out)))

(OUT is printed repeatedly.)

How it works is that tagbody contains a bunch of labels mixed with forms. The tagbody establishes an exit point for GO expressions used in these forms. When a (go label) occurs, it works by abandoning the current sub-form being evaluated, and initiating a control transfer to the exit point, which then transfers control to the appropriate label.

The upshot is that whatever form is interrupted will cleanly unwind, as necessary:

  (tagbody
    (unwind-protect (progn 
                      (go out)
                      (progn 'never-printed))
      (print 'bye-bye))
   out)

Note that you can only have labels and GO in certain forms like TAGBODY and PROG, not just willy nilly in any construct. The labels of a given TAGBODY are all on the same level of nesting: immediate children of the TAGBODY form. SO this isn't possible:

  (tagbody
     (go impossible)
     (let ((x 42))
       impossible x))

Here, impossible is not considered a label associated with the tagbody, so the (go impossible) is branching to a nonexistent label. If it were allowed, of course it would create the problem of what becomes of the initialization of x.

Thus, compilers only have to reason about GO within specific constructs, and rely on those GO's to only be performing abandonment followed by a simple lateral move.

[wallunit]

Fixed: https://github.com/snoack/python-goto/commit/a46dbe57e9cfa4f...

Jumps out of loops and other blocks work now. However, there are some limitations:

1. As I only have 4 bytes left to inject POP_BLOCK instructions, you can't exit more than 4 blocks with a goto. But this is handled and results into a SyntaxError rather than crashing Python.

2. Jumps into loops still doesn't work. However, it's handled now, and causes a SyntaxError as well.

3. When jumping out of a try- or with-block the finalizer is skipped.

[RussianCow]

How would you expect the second one to work?

[polemic]

Incidentally, from 2009: http://code.activestate.com/recipes/576944-the-goto-decorato...

And presented at KiwiPycon 2014: https://www.youtube.com/watch?v=DdU8I09BGsU

[moretti]

Here's another Python 3 compatible implementation: https://github.com/cdjc/goto

[ssanderson11235]

If you're interested in this sort of thing, you might also check out https://github.com/llllllllll/codetransformer, which is a general-purpose library for doing these sorts of shenanigans in an almost-sane way. (Disclaimer: I'm one of the library authors).

As an example, I've got a PR open right now that lets you do things like:

    @mutable_locals
    def f():
        out = []
        x = 1
        out.append(x)
        locals().update({'x': 2, 'y': 3})
        out.append(x)
        out.append(y)
        return out

    assert f() == [1, 2, 3]

which works using a combination of ctypes hackery and replacing all LOAD_FAST instructions with appropriately-resolved LOAD_NAME instructions.

[veddox]

47 years after Dijkstra's "Go To Statement Considered Harmful"[1], the Structured Programming Wars flare up again on Hacker News :D

Neat hack, though.

[1] http://www.u.arizona.edu/~rubinson/copyright_violations/Go_T...

[jsbg]

Someone wrote an implementation of COMEFROM (a joke response to the Dijkstra article) along with GOTO in Python for an April fools' day joke as well.

https://en.wikipedia.org/wiki/COMEFROM

[Veedrac]

I was aware of the April Fools version.

The idea that it needs optimizing, no doubt because someone's using it in a hot code, is hilarious. That's either the best satire or the worst truth I've heard all day.

[sklogic]

I am totally going to use this approach in a production code. Because I am using generated code a lot.

[ainiriand]

WHY?! Good experiment anyway, well done.

[ceronman]

Because it's a fun exercise to understand the internals of CPython. That's what hacking is all about.

[aap_]

Because goto can be useful?

[WalterBright]

goto is sometimes the right tool for the job.

[Intermernet]

Agreed. The Linux kernel being a good example.

See http://blog.regehr.org/archives/894 , http://thread.gmane.org/gmane.linux.kernel/84823/focus=85436 and https://github.com/torvalds/linux/search?q=goto for some discussion and examples.

From Mr Torvalds (as diplomatic as ever...):

>I think goto's are fine, and they are often more readable than large amounts of indentation. That's _especially_ true if the code flow isn't actually naturally indented (in this case it is, so I don't think using goto is in any way _clearer_ than not, but in general goto's can be quite good for readability).

>Of course, in stupid languages like Pascal, where labels cannot be descriptive, goto's can be bad. But that's not the fault of the goto, that's the braindamage of the language designer.

[Retra]

Theorem: "X is sometimes the right tool for the job" is true for all X.

Now could we maybe explain what jobs we're talking about and why it's the right tool for them?

[parados]

For example for writing Python C extensions using 'Pythonic' C: https://github.com/paulross/PythonExtensionPatterns/blob/mas...

[csl]

Nice. But does it handle extended jumps?

I know this is a problem for at least the Byteplay module.