top | item 2906964

Functional Programming Is Hard, That's Why It's Good

177 points| trptcolin | 14 years ago |dave.fayr.am | reply

102 comments

order
[+] tumult|14 years ago|reply
FP is becoming the new OOP. People who don't understand its original meaning are misinterpreting it and incorrectly expounding its usefulness. Newcomers are not grasping how it fits into the bigger picture. Be cautious.
[+] dustingetz|14 years ago|reply
You know, I've learned something today!

When some people talk about functional programming, they talk about monads and iterating with recursion and lambda calculus.

Other people talking about functional programming mean using data-oriented abstractions, where functions operate on data in well-defined ways, to avoid state and interlocked dependencies[3]. I've been calling the latter a "functional style", but clearly there is ambiguity and people get turned off and say things like "we're using an OO language why would i want FP"

in Avdi Grimm's talk "Confident Code"[2] he calls this style "narrative structure", compared to "haphazard structure"[4]. I like that; "functional" envokes all sorts of emotion about being hard to understand and unnecessary. "Narrative" or "confident" or "data-oriented" are more articulate, and don't evoke flame wars.

Avdi's example (code screenshot) is the most articulate example of this I've yet seen. https://lh4.googleusercontent.com/-Cs-muD17ato/Tk6b1Rb9MwI/A...

And it's in ruby. Nothing hard here, but it meets a functional style per John D Cook's definition[1]: "OO makes code understandable by encapsulating moving parts. FP makes code understandable by minimizing moving parts." "functional in the small and OO in the large" seems a good path. The Scala community seems to get this, and seems to me Scala is mainstreaming much quicker than any of the pure functional languages.

[1] http://www.johndcook.com/blog/2010/11/03/object-oriented-vs-... [2] http://avdi.org/talks/confident-code-2010-01-12/confident-co... [3] http://www.dustingetz.com/how-to-become-an-expert-swegr [4] http://www.dustingetz.com/confident-code-vs-timid-code

[+] joe_the_user|14 years ago|reply
I think you miss difference between OO and FP.

It is paradoxical. Fifteen years ago, OO was this giant, over-sold piece of junk that was going to magically make random code piece. But, but it does/did have merits for ginormous software operations employing many, many mediocre programmers. It contains a simple metaphor and simple device for turning totally bad code into slightly less bad code, for allowing completely unrelated stuff to relate together in a half-assed way. OO isn't about giving programmers more power but allowing the power one programmer to interface with that of another (even perhaps another inferior programmer).

FP is about power. You could anhilate whole galaxies with the power of a third order functional if your mind was strong enough to wrap itself around such a thing - from another link currently up: "Lisp's purpose in the programming language galaxy is to assist our most gifted fellow humans in thinking previously impossible thoughts, remember?" Essentially, neither Lisp nor FP nor Logic Programming exist to collect the multitude of junk that enterprises have and make it more sensible.

==> I am speaking "jocularly" here. I actually think collecting the junk of enterprises is a worthy activity. Large scale programming in a typical enterprise (not Google) requires you to make allowances for a multitude of human foibles and this can expand you as a human being and not just a genius.

Just consider, there are geniuses who just produce "amazing stuff" for others to run after trying to understand, there are geniuses who explain to other geniuses and there are geniuses who explain to normal people. All have their place.

[+] dustingetz|14 years ago|reply
Care to elaborate?
[+] mwexler|14 years ago|reply
Ok, I've been a procedural programmer for years. I struggle even with OO. Where's my easy bridge to Functional? Where's the killer "here's the trick, the secret, the leap"?

Because I have to say, for all the "functional will save you" mantras, I keep finding a procedural approach gets the problem solved. Are my problems too simple? Not scale issues? Perhaps. And clearly, I am not trained in anything other than intro Lisp and Hadoop, so I haven't had the deep dive indoctrination others appear to have had.

But as each new computing metaphor comes, we find ways to make it easy for folks trained in older metaphors to come over. Other than Scala, I've found few bridges that are trying to help procedural and OO lang folks adopt functional. It's no-one's fault, I guess, other than new folks are trained in it, and older folks aren't.

But I'll keep looking for that bridge, that shining one thing that will make me go "aahhha, I see" and not "one of these books will explain how passing this function through this multi-nested other function is better than just making a loop".

Because after all these articles, I know functional is great. I just feel bad that I haven't been able to make it great for me... yet.

[+] jberryman|14 years ago|reply
I don't know if FP will appeal to you or make your life easier, but why not spend a couple weeks learning Haskell and see? Try "learn you a Haskell" or the Real World Haskell book (both excellent and free online).

I'm a fucking music major and figured it out pretty well. I'm sure you'll have no problems if it's something you're actually interested in learning about in your free time.

[+] dustingetz|14 years ago|reply
one of the cardinal rules of OO is: "each unit (method, object, whatever) should do exactly one thing"[3]. this rule is universally ignored in all the big OO codebases I've seen[1]. applying this rule with discipline, forces you to have a better understanding of what your code is actually doing. I think, that applying this rule even in OO languages forces you towards "functional-style in the small, object-oriented in the large", and the only difference is your code will have fewer classes/methods, and more high-level data-structures with operations like list.filter on them.

[1] i get it, refactoring takes discipline and time, and desire to practice and learn[2], and a team of like minded people, on top of sensible business constraints. [2] http://www.dustingetz.com/nostrademons-75h-work-week-harmful... [3] http://en.wikipedia.org/wiki/Single_responsibility_principle

edit: added source for single-responsibility principle

[+] dasil003|14 years ago|reply
The thing that got me really interested in learning Haskell was the realization that something like 90% of the nasty bugs that I fixed in production over the years would be have been impossible in a pure functional language.

The thing that slows me down is the prospect of actually getting paid to write Haskell, and my doubts about it's ultimate suitability for iterate-quickly, fail-fast software world.

But reading this article reinvigorates me. Pure functional programming may be much harder, but its rigor allows for more powerful abstractions. Grokking those abstractions is no doubt useful regardless of what language you're using.

[+] thesz|14 years ago|reply
>doubts about it's ultimate suitability for iterate-quickly, fail-fast software world.

Lest you doubt: http://thesz.mskhug.ru/svn/hiersort/doc/hhm-eng.pdf

We did a cycle-accurate prototype of MIPS CPU with some twists. In Haskell.

For Haskell to shine in prototyping you have to apply it to some critical and new task, where type system works with you, preventing errors. The novelty is crucial, I think. You will have to explore the solution space with the help of some sort of theorem prover (type system).

[+] richcollins|14 years ago|reply
They might have been impossible but how long would it have taken to write the functional code vs write the code with side effects and fix the bugs?
[+] rednum|14 years ago|reply
It is worth noting that there are a few other paradigms that can change the way you think of programming - recently there was a post providing a good overview of languages that will make you learn some new abstractions, functional programming being one of them: http://blog.fogus.me/2011/08/14/perlis-languages/
[+] vegai|14 years ago|reply
I have done a few real, production projects with Haskell, and I've come to realize that a lot of interest in it is hype. The paradigm is neat, the language is relatively well designed (considering that it's an academic committee language) and the implementation is passable.

Programmers sadly forget that the greatest challenge of software creation very often happens outside of your text editor: it happens in the domain, in architecture, specification and communication. If you succeed in those, your project will succeed, no matter which language* you chose. If you fail in them, no amount of strictness in your programming language will save you.

* unless of course it's Java

[+] Dn_Ab|14 years ago|reply
I see that some people are saying that functional programming is more powerful than OOP. I disagree with that.

I do not think one is less than the other. Actually there is a mathematical argument why one is not less than the other. To me functional programming is "what does what I'm trying to describe do?". In object oriented programming it's "what are the properties of the thing I'm trying to describe?". In functional programming I describe the interactions directly and in OOP the interactions come about from how I have described the objects. Done properly, they are both about interactions, it is just a difference of what you focus on.

It has been shown that classes of OOP can be modelled for the most part as co-algebras. This means that objects are a mathematical dual to algebraic types of functional programming. The reason why functional programming is important is that some things are easier to express in a dual space. This means that many things that are hard in OOP are trivial in functional languages. But the reverse is also true. This is why it is important to not drop one for the other.

The real advantage IMO in functional programming is that these languages tend to be developed with stronger mathematical foundations. Hence programming in them tends to encourage people to be principled and rigorous (at least in theory). It is very likely that if you study the subject you will naturally come to be interested in why monads are only one particular type of functor, that polymorphic functions are well described as natural transformations or try to wrap your head around mechanically generating dynamic algorithms in terms of hylomorphisms. The more mathematical nature also makes it less magical and rickety to the self taught programmer (such as myself). That I think is the real advantage. But there is nothing inherent in OOP that stops it from also being built from more rigorous foundations. Such things will come to matter more with increasing concerns in security.

I think functional programmers sleep on the power of coalgebras. In fact his example of google map reduce is completely ignorant of the fact that the real hero in MapReduce tm is unfold not reduce.

I have a pet theory that the fact that algorithms are written in dual styles is why experienced OOP people find functional programming so hard. They literally have to reverse their style of thinking. This takes a lot of energy. Just because it takes place in your head doesn't make it any less physical than trying to roll a boulder uphill or get a wagon wheel out of a rut.

http://www.cs.ru.nl/E.Poll/papers/durham97.pdf

[+] dustingetz|14 years ago|reply
i think the root problem here is out-of-band -- expertly written OO models are fine. the problem is that i've never encountered an expertly written OO system, which is probably because OO allows you to get lazy, where functional patterns require thought and understanding. i'm gradually starting to suspect that an expertly written OO system actually kinda looks like a functional system, except with practical compromises with respect to functional purity.
[+] jberryman|14 years ago|reply
> The more mathematical nature also makes it less magical and rickety to the self taught programmer (such as myself). That I think is the real advantage. But there is nothing inherent in OOP that stops it from also being built from more rigorous foundations.

I feel that OO the paradigm is rickety, a marrying of ad hoc semantics and syntactic sugar, and that it is inherently limited.

[+] Eliezer|14 years ago|reply
And here I was expecting the surprise moral to be, "Functional code looks better than average, just because only smarter-than-average programmers can manage to work in FP languages."
[+] richcollins|14 years ago|reply
Functional Programming Is Hard, That's Why It's Good

This is a major issue in software development. Developers like a challenge so they are often drawn to complexity. Unfortunately, this interest in complexity often carries over into their work.

[+] kenjackson|14 years ago|reply
I'd really love to hear PG defend this line: "But with Lisp our development cycle was so fast that we could sometimes duplicate a new feature within a day or two of a competitor announcing it in a press release. By the time journalists covering the press release got round to calling us, we would have the new feature too."

I'd really love to see the code for something that one would do in a day or two in Lisp, but would be much longer in another language. I don't suspect there exists a wide class of such things -- I suspect the features tend to look like embedding Lisp compilers.

[+] anghyflawn|14 years ago|reply
Also speculating, but I would hazard a guess that a lot of that was due to the fact that the library ecosystem was much smaller then, which means people would be implementing things that you don't have to think too hard about now.
[+] richcollins|14 years ago|reply
It's not because they were using lisp. It's because everyone else was using C.
[+] dustingetz|14 years ago|reply
i speculate the edge compounds as, say, a competitors java codebase's complexity increases faster than a lisp codebase. exponentiate this over time and the edge is obvious.
[+] Apocryphon|14 years ago|reply
How about learning functional JavaScript?
[+] brohee|14 years ago|reply
"Pointers are a very powerful and fundamental abstraction"

An abstraction of what? Can't get much closer to the metal than with pointers. Frankly I'm not sure someone writing this have much insight worth wasting time on.

[+] gregburek|14 years ago|reply
I have an electrical engineering background and all this FP VS IP sounds like Async Logic VS State Machines. How far off base is that description?
[+] kwithl|14 years ago|reply
In the first paragraph a dialect of Lisp is considered a functional programming language. So I don't read any more. 133 points for this?
[+] Paddy3118|14 years ago|reply
The "One True Language Paradigm"....

... Is functional programming!

(while it lasts that is).

Keeps the money circulating.

[+] quizbiz|14 years ago|reply
Can anyone recommend any resources for getting started with Scheme?
[+] rimmjob|14 years ago|reply
the little schemer series was very good. you can safely skip the first 5 chapters if you're comfortable with recursion. learn scheme in fixnum days is good too for learning the language right away.
[+] nirvana|14 years ago|reply
I love Erlang so much I haven't yet gotten around to learning another functional language. (still growing with erlang).

I learned it reading Armstrong's book from Pragmatic Programmers. It was a joy. I'm also reading the OReilly book, and it seems good as well, though I can't tell how it would be if I didn't know the language already. As for the titles-- one is Programming Erlang, and the other is Erlang Programming!

[+] stmartin|14 years ago|reply
I think you should learn Brainf*ck. It has awesome mental/intellectual benefits to you as a Erlang lover. No, but really.... ! And if you don't you will suffer! Muahaha!
[+] stmartin|14 years ago|reply
This is the lie that every functional programmer has perpetuated for the last 50 years since the dawn of Lisp - that somehow, automagically, productivity or effectiveness of programmers increase with the use of functional languages.

It's the biggest lie in the programming world, and it was designed to make the uber-geeks of this world, those people that you walk around the block just to avoid saying hello to because you know they are deeply and profoundly anti-social, feel good about themselves, to feel special, as though they have the "upper hand" that other normal people who deliberately choose other languages over purely functional ones.

There is value in learning how to think recursively, but a lot of programming, especially systems programming doesn't need to have its for loops transformed into recursive functions, or its addition operators into primitive recursive functions.

I am sick and tired of people expounding on the alleged efficacy and efficiency of functional languages and I wish they would just STFU once and for all and go back to their porn intermezzos between their functional programming spurts.

[+] dasil003|14 years ago|reply
You're making the same mistake that the Java loyalists did when Rails came out: judging something based on your distaste for the advocacy around it without any real understanding. I'm not going to pretend to know your motivations here, but often times people throw up this defense mechanism to protect their own professional knowledge; but here's the thing, that does nothing but stunt your own growth.

Functional programming is about the removal of side effects, aka mutable state. Removing side effects makes programming harder because ultimately the whole point of computer programming is to create side effects. Isolating those side effects and writing most of your code in a functional (mathematical definition: always the same output for a given input) way is challenging, but it also promises an order of magnitude more possibility of correctness. The mathematical rigor of a pure functional language like Haskell allows deeper reasoning to be done, and thus more powerful abstractions to be introduced.

In practice functional programming is not the fastest way to solve many problems, but it is an excellent way to push forward the state of computer science as a whole. Think about the relationship between physics and math. We would not have the deep practical knowledge of the physical universe that we have today without the complex math that allowed physicists to reason about things well outside the realm of experimentability.

[+] BDFL_Xenu|14 years ago|reply
Two things: this is one of the trolliest posts I have seen on HN on a while, save for my own (now deleted) counter-troll. The fact that it keeps getting upvoted is astounding. Seriously, associating a certain technique with uber geeks/neckbeards/nerds is actually a valid argument for discounting it?

It's true that FP is presented too often as a remedy for the common ailment of programming of not producing software that you can understand quickly enough. Also known as 'productivity'. But just because it's over-enthusiastically promoted it doesn't mean there's no truth to the claims. It's mostly that

  a. you can compose your software nicely from parameterizable operations over common structures (map, reduce, filter, take, etc.)

  b. mutable state can make programs messy so you should avoid it
But of course you don't care. At all. After all, if you did, you wouldn't be quickly hand waiving the whole issue with a fleeting reference to 'recursion', as if it was the dominant instrument in functional programming. You learn _very_ early that writing recursive functions is just more work than throwing together a bunch of functions over a sequence or some such if needed.
[+] danenania|14 years ago|reply
"it was designed to make the uber-geeks of this world, those people that you walk around the block just to avoid saying hello to because you know they are deeply and profoundly anti-social, feel good about themselves, to feel special, as though they have the "upper hand" that other normal people who deliberately choose other languages over purely functional ones."

What is the point of this vitriol? Makes you seem fairly anti-social yourself.

[+] KirinDave|14 years ago|reply
I think you're misunderstanding the aim of my essay. It's not to say that FP is uniformly superior to OOP. It's that there is value (and difficulty) in learning the functional abstractions. The same is true of most programming paradigms, but we live in a world where FP is one of the less understood and increasingly relevant tools.

Do you really think this is so controversial?

[+] tete|14 years ago|reply
No offense but the same could be said about Object Oriented Programming. To me functional languages make a lot more sense, because of the "No side effects" stuff.

Reminds me of a quote from Richard Stallman: "Adding OOP to Emacs is not clearly an improvement; I used OOP when working on the Lisp Machine window systems, and I disagree with the usual view that it is a superior way to program."

But I also don't consider functional programming harder. I actually do have more problems with (C++/Java/Python style) OO, even though I like it when it is a real language concept, like in Smalltalk and others.

But I am weird. I consider Assembly to be easier (to learn) than C(++) and Perl easier (to read) than most other scripting languages.

I think it is a lot about the way you think. A language has to compensate the things that are hard for you so you can concentrate on the easy stuff. It's a bit like having different tutorials or teachers. There are people that think similar to you while there are other who think very different. It's a common problem in schools. If the teacher thinks in a different way the pupils will have a hard time.

[+] klez|14 years ago|reply
I'm not a FP-fan boy (yet), but it seems you have a very limited view of FP. It's not just recursion...
[+] joemoon|14 years ago|reply
Wow, this is really dramatic.