WingNews logo WingNews GitHub [2]
top | item 7595400

A collection of not so obvious Python stuff

88 points| adamnemecek | 12 years ago |nbviewer.ipython.org

15 comments

order
[+] andolanra|12 years ago|reply
I'm being a bit pedantic, but the last snippet under About lambda and closures-in-a-loop pitfall is arguably misleading when it says, "This, however, does not apply to generators…" Which is true in the example given, but not true in general. Generators don't actually provide a new environment on each evaluation, they just lazily construct their contents, which assuages the problem in certain situations. This snippet

    for l in (lambda: n for n in range(3)):
        print(l()) # prints 0, 1, 2
will alternately request an argument from the generator, return a lambda, call it, and only then request the next argument from the generator, which means at the time of the funtion call, n will not have been incremented up to its final value. We can see the previous closures-in-a-loop behavior return if we keep the closures around to execute again, at which point the loop environment contains the final value of n:

    funcs = []
    for x in (lambda: n for n in range(3)):
        funcs.append(x)
        print(x()) # prints 0, 1, 2
    for y in funcs:
        print(y()) # prints 2, 2, 2
So generators don't 'fix' the problem by constructing distinct environments for each loop iteration, but by deferring evaluation of their body until necessary.
[+] MereInterest|12 years ago|reply
Yeah, that part stood out to me as well. The only way I've found to get different values for each of the functions is to use a different closure for each functions.

    funcs = [(lambda i: (lambda :i))(n) for n in range(3)]
    for f in funcs:
        print(f())
[+] acjohnson55|12 years ago|reply
What I love about Python is that these hangups are mostly relatively obscure, due to the puritanical adherence to principle of least surprise by Guido and the other language maintainers. No language is completely without quirks though, especially one as old as Python.
[+] jsmeaton|12 years ago|reply
This bit bothered me slightly:

>But what if we put a mutable object into the immutable tuple? Well, modification works, but we also get a TypeError at the same time.

    tup = ([],)
    tup[0] += [1]
Of course that's an error. It's equivalent to:

    temp = tup[0]
    temp += [1]
    tup[0] = temp
But the given explanation is way too obtuse. This shouldn't be a mystery at all:

> If we try to extend the list via += "then the statement executes STORE_SUBSCR, which calls the C function PyObject_SetItem, which checks if the object supports item assignment. In our case the object is a tuple, so PyObject_SetItem throws the TypeError. Mystery solved."

[+] rix0r|12 years ago|reply
It's not ENTIRELY obvious. For example, tup[0].extend([1]) would work fine, and the behaviour of __iadd__() seems like it should be the same as extend(). In fact, the reference[1] has the following thing to say:

> For instance, to execute the statement x += y, where x is an instance of a class that has an __iadd__() method, x.__iadd__(y) is called.

However, this is patently not true, apparently:

    tup = ([],)
    tup[0].__iadd__([1])  # Works
    print tup

    tup = ([],)
    tup[0] += [1]  # TypeError
    print tup
[1] https://docs.python.org/2/reference/datamodel.html#emulating...
[+] zwegner|12 years ago|reply
The "consuming generator" problem is a good point, but their workaround misses a better solution. In Python 3, range() returns a range object, which has a __contains__ method, so you can use "in" just fine without consuming any generators. It's also much faster (it would be constant time/memory except for the fact that integers can have arbitrary size).

    l = range(5)
    print('2 in l,', 2 in l)
    print('3 in l,', 3 in l)
    print('1 in l,', 1 in l)
prints...

    2 in l, True
    3 in l, True
    1 in l, True
Also, the lambda/closure example is something that really irritates me about Python. Both dynamic and lexical scoping have this problem: they always reevaluate the variable inside the closure whenever it's called, the only difference is which parent scope the value is pulled from (the parent scope or the calling scope). It's not that the "last lambda is being reused", as the article says, but rather, there are five different lambdas, but they each refer to the same variable, which is set to 4 after the list comprehension. This sucks for metaprogramming, where you'd like to create a bunch of copies of a function with different parameterizations. There's two workarounds that I know of for this for dynamic scoping: creating another nested scope, or using keyword arguments. I'm not sure what you'd do for lexical scoping (but who uses that anyways?).

    for x in [(lambda i: lambda: i)(i) for i in range(5)]:
        print(x())
...or...

    for x in [lambda i=i: i for i in range(5)]:
        print(x())
In my purely-functional Python-like programming language, Mutagen, closures are static, so that they capture the value of variables from the enclosing scope at the point that they're defined. So this works as you'd expect, printing 0 and 1 (no list comprehensions yet, and lambdas are full functions, and of course lists are immutable so no +=):

    lambdas = []
    for x in [0, 1]:
        lambdas = lambdas + [lambda: return x;]
    for l in lambdas:
        print(l())
[+] emidln|12 years ago|reply
The first workaround is actually the same as what you do for a map in 2.x (that always works):

    for x in map(lambda i: lambda: i, range(5)):
        print(x())
[+] zwegner|12 years ago|reply
Oops, just realized I mixed up dynamic/lexical scoping, and it's too late to edit. Python uses lexical scoping.
[+] d64f396930663ee|12 years ago|reply
Even after reading this list, I'm comfortable forgetting everything (with maybe one exception) because these surprises really only come up when you use horrible coding practices. But it's nice to know it really takes some serious effort to find these kinds of things in Python.