Toolz: A functional standard library for Python

[ianbutler]

I had a bit of a derp here at first, I was like what, python already has a functional (as in working) standard lib, then I read the docs and was like ohhhhh that functional.

I think python made some half measures in regards to functional programming, not a bad thing since python tends to blend the different styles decently but at least for me it would be nice to have a good library to extend the functional side a little more, hopefully this scratches the itch.

[jonmoore]

Toolz, like seemingly everything Matt Rocklin is a major contributor to, is something of a model library: cleanly designed and coded, with strong documentation.

Although Python is not going to match a full Lisp, Haskell or ML in all their strengths, using a functional style can be useful and expressive. The toolz docs give some relevant background at https://toolz.readthedocs.io/en/latest/heritage.html .

At a language level, Peter Norvig gave a lengthy comparison of Python and Lisp at https://norvig.com/python-lisp.html in 2000.

[vorticalbox]

I've been writing functional code for a year or so now at work with the help of ramda in nodejs, there is also a port for python.

My colleagues don't like becuase it's so different to what they are used to but I am putting out code faster and with less bugs so I'm not going to stop.

[zelphirkalt]

Why use a library for it? Can it not be done with just Python? And if using the library is much different from normal Python, does it mitigate Python's problems with functional programming? (For example one expression only lambdas and no TCO.)

I also do use some functional concepts in my Python work, but do not use a library for it. Only procedures or functions. No additional dependencies.

[archarios]

Hell yeah me too! Some of my coworkers like it, but there are a few out there who basically don't want to learn something new.. I have been using Ramda professionally for about two years now and I love the heck out of it. My org is using more and more TypeScript which makes using Ramda a little harder but it still works pretty well in this context too. I have been working on a test data inventory project with Ramda recently and it has made the implementation so much nicer than it would have been otherwise. I started using pipeP (by defining it with pipeWith) and omg it's so great for dealing with a mix of async and non async calls.

[agumonkey]

I've dealt with python noob codebase that had many deep side effects over multiple (~2000loc total) files. The overall logic fit into a single page, bugproof and easy to augment using FP idioms.

Yet people want to stick with a "style".

Crazy.

[dean177]

> My colleagues don't like becuase it's so different to what they are used

Is it worth being faster if it makes everyone else slower?

[seertaak]

Functional programming in Python is severely hampererd by Python's lambda keyword, which - simply put - sucks.

[samasblack]

I like the way the Functional Programming HOWTO in the Python docs (https://docs.python.org/3/howto/functional.html#small-functi...) puts it:

  Fredrik Lundh once suggested the following set of rules for refactoring uses of lambda:

  1. Write a lambda function.
  2. Write a comment explaining what the heck that lambda does.
  3. Study the comment for a while, and think of a name that captures the essence of the comment.
  4. Convert the lambda to a def statement, using that name.
  5. Remove the comment.

[heavyset_go]

Yeah, Python would really benefit from multi-line anonymous functions.

There was a limitation in Python where spacing and indentation gets ignored between parentheses, which makes it impossible to pass a multi-line lambda as an argument to a method or function. However, given the new parser, that limitation might be able to be mitigated.

[dragonwriter]

Functional programming in Python is severely hampered by the fact that Python is statement-oriented rather than expression-oriented.

The lambda limitations are a product of this plus the particular whitespace rules Python has.

[raymondh]

Not really.

Python's lambda does pretty much what you would expect from Scheme. It creates a callable the binds arguments to parameters in a lexically scoped namespace and then evaluates the body of the lambda in that namespace. And with an open parenthesis, you can write multiline lambdas and indent it however you want.

All the usual wizardry is possible:

  >>> (
      lambda n: (lambda fact: fact(n, fact))(
          lambda n, inner: 1 if n == 0 else (n * inner(n - 1, inner))
      )
  )(5)
  120 

Also, Python has rough equivalents to some special forms in Scheme:

  (if testexp posexp negexp)  ⟶  (posexp if testexp else negexp)
  (cond (p1 e1) (p2 e2) (else e3)  ⟶  (e1 if p1 else e2 if p2 else e2)
  (begin e1 e2 e2)  ⟶  [e1, e2, e3][-1]
  (and e1 e2 e3)  ⟶  (e1 and e2 and e3)
  (or e1 e2 e2)  ⟶  (e1 or e2 or e3)

Some statements do have a functional form but aren't well known:

  class keyword  ⟶ type(name, bases, namespace)
  import keyword ⟶ __import__(name, globals, locals, fromlist)

You have map(), filter(), partial(), and reduce(). The operator module provides function equivalents for most operators. Also, the itertools were directly based on their equivalents in functional languages or array manipulation languages.

That said, Python does lack some essential tooling that you would really miss:

  - There is no way to create new special forms.
  - Some important special forms are missing: let, let*, and letrec
  - The language spec precludes tail call optimization.
  - Some Python statements lack functional equivalents: try/except, with-statement, and assert-statement

[cabalamat]

Given that Python allows unicode characters in source, I'm surprised it doesn't allow lambda to be replaced by the λ character.

[AlexCoventry]

Also by the lack of persistent data structures with structural sharing. That forces you to take an expensive copy whenever you want a modified version of some data, or bash it in place.

[gameswithgo]

I imagine it is problematic that Python is already very slow, and then to pile up idioms with performance downsides on top of that is kind of rough.

[bj0]

This is one of the biggest reasons I really like (the python superset) Coconut: https://github.com/evhub/coconut

[0x2c8]

Can you elaborate on this? What is wrong with Python's `lambda`?

[vmchale]

Also the recursion limit.

[lysium]

I prefer functional style whenever I can. I noticed that this style slows Python programs down. It seems function calls are rather expensive. Does anybody have a similar experience?

[heavyset_go]

PEP 590[1] makes calling callables less expensive. It was finalized for Python 3.9.

[1] https://www.python.org/dev/peps/pep-0590/

[AlexCoventry]

It's been a while since I knew those kinds of details, but it certainly been the case in the past.

https://stackoverflow.com/questions/22893139/why-is-a-functi...

[thisiszilff]

Yes, I suspect it is a pitfall of a multi-paradigm language that cannot assume so much about code in order to optimize. Opinionated functional languages (ie clojure, haskell) can have a lot more guarantees about what is going on in order to optimize all those function calls, lexical bindings, etc.

[vmchale]

They have to check a recursion limit, for one.

[mujina93]

For a type safe approach to functional programming in python, try returns (https://returns.readthedocs.io/en/latest/).

[pansa2]

It always amazes me the lengths people will go to to avoid learning & using a different language.

Perhaps if you want statically-typed functional programming, you shouldn’t be using Python? It’s not the best choice for that - in fact, it’s almost the worst choice.

[rcv]

Looks like a very nice more functional-focused alternative to pydash[1] (which is a Python port of lodash.js, which in turn is a superset of another library called underscore.js - whew).

[1] https://pydash.readthedocs.io/en/latest/index.html

[jgalt212]

I've always found logging and debugging to be just so darn tedious in functional realms.

e.g.

how to do easily you log/instrument f(g(h(x)))?

[ElevenPhonons]

Perhaps a custom compose function can help with these use cases? This series has a few examples of composing computation in Python that might be useful.

https://github.com/mpkocher/Functional-Programming-Technique...

And there's a general list of resources here.

https://github.com/sfermigier/awesome-functional-python

[rcv]

Not sure how easy convenient this is in practice, but the docs mention using the "do" function [1]

[1] https://toolz.readthedocs.io/en/latest/api.html#toolz.functo...