top | item 21282647

Haskell in Production

200 points| allenleee | 6 years ago |felixmulder.com | reply

237 comments

order
[+] Mathnerd314|6 years ago|reply
All the interesting code is in the next post, http://felixmulder.com/writing/2019/10/05/Designing-testable....

I'm not sure why they use a generic monad rather than ST, they don't need continuations for this.

The Reader monad with a big record is standard Haskell, it's basically what GHC uses: https://github.com/ghc/ghc/blob/1219f8e8a3d1b58263bea7682232...

data-has is less standard, it only seems to have been used seriously by one project, which has a bug on file to stop using it: https://github.com/myfreeweb/magicbane/issues/20

But overall it's interesting, we'll see where the series goes.

[+] microtherion|6 years ago|reply
It's always good to see examples of how a language is actually used, rather than just assertions of its superiority.

That said, if Haskell programmers truly consider this a superior and readable way to write code would tend to reinforce my preconception of them as an insular community elevating arcane jargon to a virtue (not unlike the APL programmers of yore).

[+] gip|6 years ago|reply
> Haskell is great for business and great in production

I disagree. It's a beautiful language but it lacks a lot of features to make it useable at scale. And in my experience Haskell engineers are extremely smart but the environment/culture they create makes it difficult to foster team spirit.

I've been in 2 companies in the last 4 years who initially used Haskell in production. One has transitioned to Go and the other is rewriting the codebase in Rust.

[+] ljm|6 years ago|reply
I’ve met some pretty damn solid engineers who started on Haskell and, even at a junior level in other languages, produce an elegant solution far more easily than a senior engineer in that language. You probably wouldn’t put the code in production verbatim but you can very easily see what’s going on and it isn’t haunted by spectre of early abstraction, which IMO is the biggest flaw of OOP at scale.

People think you want to see clean architecture in the form of lots of indirection and state and then you don’t see any real programming chops, just organisation or filing.

Not at all related to Haskell specifically, but damn, it does cut through the boilerplate. Although I can’t comment on any sort of Haskell at scale and I imagine they have the same issues when it comes to excessive point-free/applicative style.

I mean, from my naive perspective it’s easy to make classes out of everything, and to hold state and put side-effects everywhere, but you don’t want to deal with the trouble of a monad until you need it. So you have an automatic inclination towards cleaner code when you start functional and move on.

[+] whateveracct|6 years ago|reply
My experience is upper management will scapegoat Haskell the moment they get the chance. Outside-hired leaders who doubt it from the start. It's a weird unpopular language - an easy target.

If you just get over it & use Haskell, things will be fine. You'll get huge gains thanks to a lot of Haskell features & libraries, and you might have to write some more fundamental libraries yourself. Haskell makes that pleasant anyways. Worst-case, using Haskell could end up being a wash vs other languages.

Rewrites aren't always indicative of failures of language or the engineers writing it. They're also a useful mechanism for solidifying control for a VPE-type at a nascent but quickly-growing startup. Especially if said VPE-type wants to push out personnel that were there before them.

[+] danharaj|6 years ago|reply
Hey you. I'm pretty sure I know who you are :) I hope everything's going well!

I definitely agree that there are many examples of ineffective Haskell culture. As a Haskell contractor I get to see how a lot of different shops do things. One thing I can't really tease apart is that most startups have a lot of engineering culture problems, and picking Haskell doesn't inoculate you from that effect.

The main cultural issue I see specific to Haskell shops is that frequently rabbit holes aren't discouraged. By that, I mean there is too much of a tinkerer mentality where engineers are allowed to or even encouraged to experiment with untested ideas instead of known effective solutions. That is a habit I had to deprogram myself from too. It's very fun! In your spare time.

I hope we get to work together again! But I'm usually only called in if there's trouble, so maybe I shouldn't :^)

[+] timbuckley|6 years ago|reply
> It's a beautiful language but it lacks a lot of features to make it useable at scale.

Can you expand on this? What features is it missing?

[+] verttii|6 years ago|reply
> it lacks a lot of features to make it useable at scale.

Can you elaborate on this?

[+] maximente|6 years ago|reply
in my (limited) experience, Haskell projects (and to a slightly lesser extent, functional programming projects) work best when thoroughly planned out in thorough whiteboard/spec sessions and are then implemented by a couple of gurus responsible for the code, who work almost exclusively in functional languages in their day to day. there seems to be a need for way more thought-per-line-of-code in Haskell/FP projects.

many engineers (and businesses!), culturally, come from the opposite angle (legions of generally pluggable engineers on java/python/golang plowing through tickets/features). that's fine, but it isn't super amenable to the Haskell world, because IME non-imperative setups require different mode of thinking about design, so harder to drop in and crush a ticket. i wish i could explain this more, but i'm struggling to articulate with examples.

i think the thorough types really help you get up to speed on data structures, but unless you are in a functional mindset pretty much exclusively, your (my?) brain spends a lot of time un-imperative-ing itself. also, laziness can be a factor, so need some expertise there.

it also helps to have conventions e.g. old school java (are we point free, or not? etc. etc.)

all in all i think it requires a lot of discipline that can easily break down, whereas some of the popular imperative languages you can still sort of plug along (=> punt the technical debt) despite that.

[+] runeks|6 years ago|reply
> […] it lacks a lot of features to make it useable at scale.

Can you give an example or two?

[+] mbrodersen|6 years ago|reply
> it lacks a lot of features to make it useable at scale

Facebook (and other companies) run Haskell in production, at scale. So I wonder what those "lots of missing features" you mention are? With "lots of them" I am sure you can mention at least 5+?

[+] seddona|6 years ago|reply
We are a YC company doing very well, all our back end code is written in Haskell. We have produced a lot of functionality with a relatively small team. I would say we are existence proof that Haskell is good for business.
[+] wolco|6 years ago|reply
What happens when you need to scale up or if one or more of the gurus quits? Do you think Haskell will still be primarily used when you grow?
[+] StreamBright|6 years ago|reply
For the problem you are solving and at your scale. Things start to change when you need to hire n+1 teams or engineers quickly.
[+] pictur|6 years ago|reply
Who determines that you are a very good company?
[+] KirinDave|6 years ago|reply
This is a great guide and sound advice. And hopefully in a year we'll see the community converge on Polysemy (or fused-effects for performance) to make this strategy even more natural.
[+] madhadron|6 years ago|reply
I've been out of the Haskell game for a while, so I hadn't seen Polysemy. Another thing on my reading list...
[+] MisterOctober|6 years ago|reply
Always nice to see Haskell at work. I haven't learned the language myself yet, but I absolutely love postgREST, which is implemented in Haskell.
[+] leshow|6 years ago|reply
I'm not sure why this post is getting so much hate, it's well written and I really enjoyed reading it. Thanks!
[+] whalabi|6 years ago|reply
How is such an obscure language - with associated difficulty in finding talented engineers - ever going to be a better choice than a more mainstream language, which probably has many of the same features?
[+] mbrodersen|6 years ago|reply
> with associated difficulty in finding talented engineers

This is a myth. There are now many experienced developers who currently can't use Haskell in their day job but would love to. If I decided to start another business, picking Haskell would give me access to a lot of top talent who would otherwise not be interested.

[+] whateveracct|6 years ago|reply
It's actually not that hard to find talented engineers, especially if you are a distributed/remote team. I've interviewed & given a thumbs up to a constant stream of Haskell candidates (either with Haskell experience or interest in learning).

That said, I've also seen a lot of Haskell engineers go through the interview pipeline & get thumbs up from all interviewers _except_ a single person in leadership who would later push to move away from Haskell for "hiring reasons" :eyeroll:

[+] dmitriid|6 years ago|reply
How long does it take the “new programmers who have not worked with Haskell before” to learn and understand all the ASCII-art in the example code? How $ differs from <$ differs from & differs from .

On top of trying to remember that <> is string concatenation and >>= and <- have something to do with monads.

[+] shantly|6 years ago|reply
I still can't understand why Perl is easily dismissed as "LOL line noise" but Haskell's apparently fine. At least dipping into Perl you can take on the line noise slowly so you have time to absorb each new technique, and you can write it just fine with hardly any of that if you want, and it won't hurt a thing—Haskell seems to dump all that on you up front, and it seems to require it, idiomatically. It's very off-putting.
[+] agentultra|6 years ago|reply
A couple of weeks in my case. It's not any more difficult than remembering that the -> is the cache-miss operator in C++.

You also don't have to learn the entire universe of Haskell before being productive in it. I still have no idea what profunctors, comonads, and other things are and I'm shipping production Haskell code.

[+] thesz|6 years ago|reply
Two weeks, Just as much as for any other new language.

After two weeks new member can introduce simple features. After a month - relatively complex ones. Full development speed is achieved after half a year and is roughly equal to all languages because all this time people learn problem(s) domain(s) and its (their) mapping to the code.

For example, if I ask you to optimize Kaldi's HCLG-based decoder, you won't get any interesting results even after two months EVEN if you known C++ well. Just to tip you with something: you can save off about 10% of execution time just by presorting HCLG graph by BFS - it is scale-free and there are nodes which decoder will visit more often than others. To do this, you have to look at the graph, you have to know about scale-free graphs and their properties. Which are way outside of programming language scope.

[+] verttii|6 years ago|reply
Operators are just functions. Learning the type system is the key. You can easily check any function's type signature in the docs once you understand how the type system works.
[+] kccqzy|6 years ago|reply
The exact same time it takes a programmer who has never seen Java or its close siblings like C++ to learn and understand all the keywords like "public" or "static" or "class" or "interface" or "extends" and how they work.

The only difference is that a random programmer you pick has probably already spent the time understanding Java and OOP, but not Haskell.

[+] slaymaker1907|6 years ago|reply
I really wish Java had associated types and static methods as parts of interfaces (i.e. constructors as part of the interface). Those two features make complex architectures much easier to design and use.
[+] chrisseaton|6 years ago|reply
Interfaces can have static methods.
[+] verttii|6 years ago|reply
This is an incredibly good series. Just what I've been looking for. I'm writing a Servant API myself and this really explains a lot of the fundamental concepts in plain English.
[+] dmix|6 years ago|reply
> We tried a lot of different patterns - readers, handlers, MTL, and tagless final.

Is there any books or long form articles digging into these different approaches? I know basic Haskell and heard of these different patterns a few times. I'm curious how it looks in a larger Haskell project and what the pros/cons are of each, and general popularity.

Edit: I see the next article in the series has more real code examples

[+] rpmisms|6 years ago|reply
The more I see it in use, the more I want to learn Haskell, but I can't think of a practical reason to do so.
[+] axilmar|6 years ago|reply
The amount of knowledge required just to have a clean design o the simplest of things, i.e. a few functions that add and delete users is very big; almost all of Haskell's important parts need to be known.

Is that good? for me, it's an obstacle. I can't expect all the developers that come and go to have a perfect understanding of monads in order to use/debug/extend such code.

And what did Haskell actually buys us here? my apologies for being skeptical, but it doesn't seem to buy us anything...in other languages, such things as creating a user, deleting a user, logging actions requires a lot less boilerplate and complexity.

[+] c3534l|6 years ago|reply
Everyone in the comments either doesn't like Haskell, because it's hard and different or niche or whatever, so they think Haskell is bad for production even though they don't code in it and haven't worked at a place that used it extensively.

The other group of people in the comments are people who love Haskell and are happy to have validation from the OP.

But I'm not really any wiser. I like Haskell, but I work in DevOps and the language simply isn't relevant there.

[+] mcmayer|6 years ago|reply
It's interesting to watch the wonderfully meandering train of thought(s) triggered by a success report of using a non-mainstream programming language. So let's meander some more...

The strong reactions are understandable. New programming languages are added to the zoo faster than ever before. And within each language there's a steady influx of new frameworks (angular1, angular2, react, vue, ...) each with a learning curve. It's a challenging environment, and it feels like it's getting exponentially harder as time progresses. One must resist the trap of following every silly new idea.

So why bother spending even an afternoon with Haskell? Spend an afternoon playing with Python and you'll be writing code the next morning. Spend a day with Haskell and you'll have more questions than answers.

The fundamental problem is, most likely, the new paradigm. Functional programming takes some time getting used to. Sure, the functional style is creeping into mainstream languages, so most of us have seen and used it already. But pure functional programming is another level. Broadly speaking, everything is a function, a function in the mathematical sense: Same input, same output. Everything is immutable, it's all about chaining pure functions. Crazy, right? Why would you subject yourself to such rigour?

I'll argue that pure functional programming has something to offer that is more relevant than ever. 1. Pure functions are easy to test. Same input, same output, that is a strong guarantee. It'll still be a challenge to thoroughly test every single function, but at least each test is meaningful because there are no interactions with anything outside the function. 2. Pure functions are easy to compose and parallelize. Single-thread performance is hardly ever a relevant benchmark anymore. Sure, C is faster, but good luck trying to parallelize a complex C program. (The worst thing I can imagine is having to debug messed-up, multi-threaded C++ code.) 3. There is structure in how to compose pure functions, and it's structure in the mathematical sense, not in the fuzzy Gang of Four pattern sense. Much of the 'arcane', 'academic' jibber jabber is exactly about these mathematical structures. The power of having a rigorous method for structuring programs is immense. It really is. The hurdle is not a high as it may seem (unless you have a low affinity to maths - im which case you probably shouldn't be programming computers anyway). 4. Pure functions are easy to refactor. (See below.)

I'll also argue that strong typing has a lot to offer and will become more mainstream. 1. Make the compiler your friend, not your enemy. Strong types glue the functions together in just the right way and prevent me from making silly mistakes. 2. Business logic can be encoded in a compact and transparent way. Changes to the business logic (and it always does change, doesn't it?) are also compact and transparent. 3. Refactoring is just ... wonderful. Strong types combined with pure functions, that is a magical combination. Change the type and let the compiler tell you what the implacations are. As there are no side effects code can be refactored with confidence. It is a vastly different experience from any OO language.

Now, finally, to Haskell: It can be characterized as a strongly typed, lazy-evaluated, garbage-collected, pure functional programming language. (Lazy evaluation hasn't been mentioned, yet. It's something that needs to be understood and considered in order to write efficient code.) In "language design space", something like Haskell must exist and will always exist. (I say 'always' because maths has something eternal about it and pure functional programming and strong typing have solid mathematical underpinnings.) And in this "language design space" Haskell dominates. Contrast this with the compiled, weakly typed, imperative, low-level languages like C, C++, Golang, Rust. I have a hard time picking amongst those, but I believe everyone should know at least one of them.

So is it worth learning Haskell? I tried to argue that it's not a fad and the concepts are fundamental and increasingly important. To me, personally, it looks like a an investment I can benefit from for decades to come.

As a business, do the advantages outweigh the costs and risks? We all know that corporate culture often stands in the way of change. One root cause is the asymmetric payoff: A thank you if it goes well, and job-loss if it goes badly. In such an environment inertia is the rational choice. - But if your company truly believes to be an innovator, or leader in the field, or seeks to gain an edge over the competition, then Haskell should be on the list of things to try.

[+] jonny383|6 years ago|reply
Advice for people using Haskell in "production": Don't. Save not only yourself the disappoint, but every other stakeholder around you. Use something tried and tested.
[+] mbrodersen|6 years ago|reply
Let me guess: you have never programmed in Haskell?
[+] MuffinFlavored|6 years ago|reply
> Haskell is great for business and great in production.

Hot take: no it isn't. It is extremely hard to learn, has an extremely confusing + needlessly complicated syntax and I question the payoff immensely. I question the well-being of anybody who subjects themselves to the pain and torture that is Haskell.

If I stood up in a corporate business boardroom meeting for tech analysis on a new project and said "I want to write it in Haskell", I'd get laughed + kicked out.

[+] foopdoopfoop|6 years ago|reply
Sounds like you've barely programmed in Haskell and don't know what you're talking about.

> It is extremely hard to learn

Haskell was the first language I learned. I didn't think this at all and I still don't. It doesn't strike me as any more difficult than learning Java or something.

You may think this because Haskell is a different paradigm than what you're used to, so while you may be able to get quickly started with Rust coming from a C++ background, Haskell will take more work because what you already know doesn't intersect with what you need to know quite as well. You may misconstrue this mismatch as Haskell being more difficult to learn.

> extremely confusing + needlessly complicated syntax

I think Haskell syntax is less confusing/complicated than many other language. Unlike a lot of other languages, Haskell doesn't have a lot of things baked-in. As an example, lot of oft-used functions are "user-space" functions that anyone could define themselves. Like `($)` or `otherwise`.

> If I stood up in a corporate business boardroom meeting for tech analysis on a new project and said "I want to write it in Haskell", I'd get laughed + kicked out.

This is such a stupid argument: what relevance is the opinion of boardroom meeting attendees on the suitability of Haskell for a project?

It sounds like you're hating on Haskell out of ignorance and some personal encounter with the language that went poorly. Instead of actually being informed.

[+] madhadron|6 years ago|reply
> Hot take: no it isn't.

Odd. I did some Haskell in prodution (hardware control and Unix daemons) and it was delightful.

> It is extremely hard to learn, has an extremely confusing + needlessly complicated syntax and I question the payoff immensely.

It is unfamiliar if you have only worked in the C family of languages. The syntax is quite similar to the rest of the ML family, which dates back to 1973. It's basically as old as C. I had the good fortunate of learning Standard ML around the same time as C, SQL, and Perl. Standard ML and SQL were by far the most straightforward to learn.

> If I stood up in a corporate business boardroom meeting for tech analysis on a new project and said "I want to write it in Haskell", I'd get laughed + kicked out.

Yes, you would, because why are you talking about programming languages in a boardroom? If you mean a more technical review committee, I wouldn't be so sure of that.

[+] thesz|6 years ago|reply
One of my colleagues was tasked (by me) to extend microcontroller model for new controllers, including new commands, different bus widths, etc. Models was written in Haskell and parametrized by different widths - program counter width, address space width, etc. As you can imagine, it was heavy on type-level code, has monads and hierarchy of type classes and had to produce C and Java code for different modeling tools.

Cooleague was not familiar with Haskell before, barely touched it. He also was recovering from hard divorce and, as he has said later, was not willing to work at all.

After a month he delivered changes needed, with tests and ready to use.

So my experiences (there are several others anecdotes) directly contradict what you say.

Now about RTS and language itself.

Haskell has threads greener than Erlang and/or Go. Go's thread stack size is 4K-8K, in Haskell it is about 1K-1.5K. It is possible in Haskell to have scheduler as a library http://hackage.haskell.org/package/meta-par for finer scheduling which is not possible with Erlang and Go.

So I question the well-being of anybody who subjects themselves to the pain and torture that is not Haskell.

[+] wellpast|6 years ago|reply
I will probably get stomped on for this but to me it's a giant elephant in the room.

When that one purist on the team proposes mandating having 100%, 80%, or any hard number of unit test coverage, most sane people tend to disagree and understand its wild impracticality and counter-productivity.

However - when one (ie, a Haskeller) says that the code must universally pass 100% strong static type verification coverage -- arguably SO MUCH worse cost/benefit than unit testing -- and you say "no that's impractical" it's suddenly a different game.

The elephant in the room is this. Haskell and strongly typed languages are highly impractical, academic pursuits. They do not at all respect the dynamism of the real world. You can certainly build production systems in any language including Haskell but by picking Haskell you are throwing sticks between your legs unnecessarily.