Python built-ins worth learning

[Waterluvian]

Sets are my #1 favourite Python built-in. I used to have these tedious imperative functions for doing change detection on collections. Then I learned sets and it turned into obvious code like:

added = b - a

removed = a - b

repeated = a & b

Etc.

Of course this is set theory and not Python specific. But still. Learn sets!

[ehsankia]

Pycon 2019 about how to use Sets more in your every day code: https://www.youtube.com/watch?v=tGAngdU_8D8

[joatmon-snoo]

frozenset is a really nice one for enforcing immutabolity :)

[MattConfluence]

A recent(-ish) addition to Python 3 I also think is worth mentioning is the statistics module: https://docs.python.org/3/library/statistics.html

Every codebase eventually needs an averaging function, now you don't have to re-implement it yourself every time. Plus it's a nice toolbox of a few different utilities that you would otherwise need to install numpy/scipy/etc. to get.

[lsorber]

Thanks, hadn't seen this before!

I wonder why they didn't have just one median function and control its behaviour with keyword arguments instead of creating 4 different median functions.

[psalminen]

Didn't know about this, thanks!

[joshuamorton]

I disagree with only one thing in this article. Frozenset is super useful. You should prefer frozenset over set by default imo. Immutable objects are safer to work with, often easier to work with (frozenset is hashable, you can key a dict with it), and sets come up all the time in a lot of work. You can get by with a sorted list, but sets are nicer and faster.

[th]

Great point. I do agree that those who use sets heavily should probably know about frozenset. :)

I've never very rarely seen these used in production code though. I'm also not as strongly biased toward them because set objects in my own code are often pretty short-lived so their mutability rarely matters.

[masklinn]

> I disagree with only one thing in this article. Frozenset is super useful.

Many of the "don't" callables are useful and I'd say I've used about half (`format` is especially nice as a more flexible version of `str`).

But Python's tendency to mutability and the `set` interface makes frozenset verbose to use & have higher overhead than sets[0]. Usually code is terser and just as clear using sets without mutating them. Which is a shame as of course you lose the safety of enforcing immutability (at runtime but still).

Then again, I guess if you're using a type checker it matters less: it shouldn't let you mutate a `set` if you've typed it as `Set` (rather than `MutableSet`).

[0] IIRC there's only one operation which is specifically optimised for frozensets: copy

[mplanchard]

I completely agree. Working on complex enterprise Python applications, a pretty significant suite of bugs comes from long-lived, mutable objects. We’ve generally had good luck with preferring immutability wherever possible, both using builtins (tuples rather than lists when possible, frozenset) and custom classes. I just wish frozendict was part of the stdlib!

[kitotik]

Ha! That was the thing I was most excited to learn about. I’ve never come across it in any codebases I’ve worked on, but will definitely begin training myself to use it by default over a normal set.

[hprotagonist]

Relatedly, I highly recommend Dave Beazley's "Built-In Superheroes" talk: https://www.youtube.com/watch?v=j6VSAsKAj98

I rewatch it every 6 months or so and usually learn a new trick or two.

[JoBrad]

Any idea what tool he's using for the "slides"?

[kabacha]

I always tend to recommend intermediate/advanced programmers to take a look at `functools` and `collections` packages - they are filled with amazing things that you haven't even though you need!

My favourites are `functools.partial`, `collections.Counter` and `collections.defaultdict` that are used very often by people who are aware of their existance.

[ehsankia]

I use defaultdict so damn often, it honestly should even be a builtin in my opinion. You also forgot `itertools`, which imo is even more useful than `functools`. I use `chain` and `groupby` quite often. There's also `collections.deque` which is a quick linked list implementation if you need either a stack or a queue.

[abhishekjha]

I have been using defaultdict and Counter a lot lately, not much partial though. Also sets are another nice thing about python.

[radarsat1]

Also `itertools.product` is very useful to avoid nested for-loops.

[gloflo]

If you are reading this, the orange bar at the bottom 5% of my screen is hugely distracting. Please allow us to close it.

[th]

Thanks for noting this. I rarely look at my site on mobile and hadn't noticed how big that widget was. I just spent a few minutes making it less visually obtrusive. I may figure out more ways to improve it later.

[ASalazarMX]

I think it's because it overlaps a little of the content. I guess it makes some of us want to tidy it away.

[striking]

http://archive.is/lwklL has it closed.

[radarsat1]

I notice that "globals" is on the "you won't use this much" pile. I am wondering, when using Python interactively from Emacs, I often do this kind of thing:

    if 'myvar' not in globals():
        myvar = ...

That way I can just execute the whole buffer instead of having to carefully be picky-choosy about what I send to the interpreter. It's especially helpful if the line involves loading a large file, calculating something that takes a while, or downloading something. Is there a better way to handle this in an interactive context? Like an automatic globals cache of some kind?

(Note, it's helpful to be able to dirty the variable when necessary by simply,

    del myvar

)

[santiagobasulto]

Amazing post Trey, as usual. This is a very comprehensive list. I can't think of any other important function missing. My students will love this :)

[th]

Thanks Santiago! :)

[rurp]

This article covers a lot of good material and I like the breakdown of categories from essential > maybe you'll look this up some day.

My one minor suggestion would be to cover getattr and related methods sooner. I stumbled across that one pretty early on as I was learning programming and it was a serious "Ah ha!" moment. Dynamic lookups with a sensible default value are useful in countless contexts.

[Splendor]

> Many Python users rarely create classes. Creating classes isn’t an essential part of Python, though many types of programming require it.

Huh. Okay.

[pfranz]

It totally depends on the kind of work you're doing. My first few years with Python I never defined a class. I (and imagine most people) were working on code get from a to b. There's a lot of that kind of work to be had. Only later when I started writing modules and libraries did I need to.

[analog31]

I spent many years with Visual Basic before learning Python. Some of the earlier versions of VB were object based, meaning that you could use pre-written objects (standard and third party), but you had to buy a special kit to create your own objects.

This was quite useful for those of us who were bewildered by OOP.

When I learned Python, I was comfortable enough with using objects, that I had no problem creating classes.

So I think it's fair to say that creating classes is something that beginning Python programmers can put off.

Since I introduced Python to my workplace, I get to watch some fairly neophyte programmers develop their skills. These are typically engineers doing scientific programming, not commercial software developers. There's a point where I get to say: "You could put that stuff in a class." And later on, "You're creating too many classes." ;-)

[emddudley]

It's really true. You can do a lot with data objects (dicts, lists) and stateless functions.

[civility]

Which part are you reacting to? I can't speak for "many" users, but all three parts you quoted apply to my usage of Python.

[jackbrookes]

Data scientists for example.

[rat87]

while python is object oriented in the sense that everything is an object it doesn't insist of doing everything via message passing. It's quite happy with basic data types and simple functions (plus decorators and context managers)

[coldtea]

OK what? Both statements are objectively true.

[ggm]

what was fascinating to me was how I fitted into the MUST SHOULD MAY MAYBE-NOT DONT matrix (in pseudo-normative form)

I used all but one of the MUSTS.

I used a few of the SHOULD

I used a similar few of the MAY

I only used one of the MAYBE-NOT (pow, and I am doing base arithmetic but not crypto)

I don't think I use any of the DONT

I'm interested in moving up on the SHOULD and MAY

[ausbah]

I am honestly a bit surprised divmod and complex are so high, I have only used them a couple of times myself - but I think they're almost always mentioned in "Intro to Python" books so most people should at least be familiar with them right?

[scarface74]

I’ve been writing Python for a little over a year mostly as glue code and scripting around AWS. I always had a sneaking suspicion that there was more I needed to learn and that I wasn’t doing things the “Pythonic” way.

This article confirmed my suspicions. Great post.

[chombier]

For a few moments I thought of all these years during which I could have just typed "breakpoint" instead of "import pdb; pdb.set_trace()".

Luckily it's been added only recently (3.7) :)

[edanm]

Wow I had the same reaction.

And that's despite the fact that I read an article a few weeks ago about this new feature! If you haven't read it, I highly recommend it, because there's more to the breakpoint feature than mentioned here (basically, you can set an env variable to choose which callable is activated on breakpoint, meaning you can turn on and off debugging on production systems, or even set prod systems to open a web port for debugging): https://hackernoon.com/python-3-7s-new-builtin-breakpoint-a-...

[amelius]

I miss the "map" function. Sadly it was removed in the transition from Python2 to Python3. In Python3, "map" is a class.

[laurentoget]

i cannot remember the last time i used str() instead of a string format.

[purplezooey]

Wish Python would get 'study' from Perl.

[tyingq]

It sorta does, now that study() is a no-op in Perl: https://perldoc.perl.org/functions/study.html

[mixmastamyk]

What is the use case?

[tigrezno]

Stop using python, let it die for god's sake!