top | item 23418699

A History of Clojure [pdf]

627 points| puredanger | 5 years ago |clojure.org | reply

374 comments

order
[+] Naomarik|5 years ago|reply
Prior to Clojure I feel that I didn't really know how to do things well.

In the context of the domain I have the most experience with, web applications, I'd summarize the bulk of programming as defining data, data transformations and shoving data through APIs. Once you get the hang of it, Clojure makes it trivial to transform datastructures from A to B. This has enormously expanded the kinds of problems I can solve.

When Rich says "It was impossible to avoid the sinking feeling that I had been “doing it wrong” by using C++/Java/C# my whole career."

I feel somehow that this is the case for the majority of people but they don't realize it yet because their experience is the most popular trendy language or framework. I've seen many examples of libraries in different languages having enormous amounts of commits and hundreds issues for problems that are trivial to solve in Clojure.

I was in the same boat, constantly grabbing for frameworks and if one wasn't available that made my task simple, would struggle trying to bend the frameworks or language I was using to come up with a solution.

I'm not a language geek and I don't write code outside my work for fun. I want to spend the least amount of time possible in front of the computer and sleep knowing my web applications won't topple over. Clojure has fit me so well that I don't think I would have accomplished what I have in other languages.

[+] natdempk|5 years ago|reply
Could you elaborate on a problem that illustrates this property of Clojure? This sounds awesome, but I have a hard time understanding what you're getting at without knowledge of Clojure.
[+] rhlsthrm|5 years ago|reply
What stack do you use for web application development for clojure?
[+] ashtonkem|5 years ago|reply
I’m in the opposite boat. I went from Clojure to Java.

I got really tired of “finishing” my work, only to discover that it was riddled with bugs from all kinds of weird edge cases. My favorite is having to use (map vec <thing>) all over the place, because the seq abstraction leaks like a sieve.

To be fair, I also worked in a domain that was extremely poorly suited to Clojure. The “data first” approach of Clojure falls to pieces the moment you have a bunch of data that’s syntactically similar, but semantically different. This requires a lot of cross-method coordination, creating a huge mess and a risk of missed cases.

[+] _bxg1|5 years ago|reply
The fascinating thing to me about Clojure is that it's so vocally a "practical" language, and yet is a) a Lisp, and b) purely functional[1]. Usually those two things are associated with language enthusiasts, and not """pragmatic""" development. In the end it works, clearly, it's just very interesting to see that juxtaposition.

I think it says something interesting about the practical value of higher-brow programming concepts, but also about how important the packaging/marketing/ecosystem is to conveying that value and making it accessible to the masses.

[1] I know it's technically not 100% functional, but all of its messaging highlights that philosophy as a focal point and advantage.

[+] brabel|5 years ago|reply
Great story, really enjoyed reading. Rick Hickey is a great speaker (if you haven't watched his talks on youtube, do it! They are totally worth it), and certainly a great programmer, which is why I respect his opinions on dynamic typing, even if I stubbornly disagree (one day I might still dive into Clojure, but I am diving into the arguably opposite end of languages right now: Rust!!). I have even more respect for him now knowing that he actually worked on Clojure for years without any income!! This is a work of passion, and I admire him for that, and for being able to turn that passion into a company big enough to pay his bills and that of several of Clojure's early adopters.
[+] mumblemumble|5 years ago|reply
FWIW, I don't think it really does to treat his opinions on typing as a blanket, universal, prescriptive statement that one can or should agree or disagree with according to one's own blanket, universal, prescriptive position. He generally doesn't come out and say this explicitly, but, in his talks, he's always speaking to a certain audience, about a certain class of problems.

You couldn't come up with a better pair of languages than Clojure and Rust for illustrating this sort of thing. I like them both (as far as my extremely limited experience in each will allow, anyway). But, if a language like Clojure is a candidate for the thing I'm working on, then I would never have dreamed of picking a language like Rust. And vice versa.

[+] mapgrep|5 years ago|reply
>I have even more respect for him now knowing that he actually worked on Clojure for years without any income!!

Hickey mentioned in one of his talks that he actually burned through retirement savings (at least some?) while taking these years to do Clojure. I got the impression he had been thinking about it for a while prior and that it was in the neighborhood of two years. (The context was, he was thanking his wife for helping to make Clojure possible, in part by indulging in many converstions about very minute details of how Clojure would work.)

(I have no idea if this is covered in the paper as it's 46 pages, I've bookmarked it but not yet read it.)

[+] tom_b|5 years ago|reply
I remember being at the first Clojure/conj - I had just spent some time learning a bit of Common Lisp and stumbled onto the conference announcement somehow.

This was very fortunate for me - after attending the conj, I was able to use Clojure at work to sneak in some FP on the readily-available JVM platforms. I later used it to do some internal REST-API work. The code for that project has run for years without modification or error.

For whatever reason, I also discovered that thinking functionally with Clojure worked so much better and naturally for me than object-oriented design methods. While I have drifted away from Clojure over the last couple of years, I find that my problem solving mind is much better in other languages because of the time spent thinking functionally with Clojure.

[+] adamkl|5 years ago|reply
"The code for that project has run for years without modification or error."

To follow up on this point, if you ever need ammunition to sell the use of Clojure inside an enterprise level organization, just show your Chief Architect the graphs of code stability over time for Clojure (pg. 71:26) and ClojureScript (pg. 71:30).

After working with JavaScript (not trying to bash) for the last few years, the idea of stability has become top of mind for me.

[+] bangonkeyboard|5 years ago|reply
I was at a talk by Guy Steele where he mentioned in passing that Clojure was a Lisp which had done everything right. This was all the more impressive to him because Rich Hickey had until then been a relative outsider to the Lisp/Scheme community.
[+] jwr|5 years ago|reply
I think what Rich got right is the practical approach. Clojure is about compromises (like all engineering). It's not a theoretical exercise, it doesn't prove a point, it's about getting things done.

This also means that sometimes things do not seem as "beautiful" as one might like. Well, that's because they aren't. There is no way to sugarcoat certain things without making the language less practical, and Clojure seems to have been designed with no sugarcoating in mind.

Incidentally, I love this quote from Rich, an answer to people complaining that Clojure isn't easy to learn: "Instruments are made for people who can play them". I've grown to appreciate that over the years. You don't expect to be able to sit at a piano of a particular brand and just start playing it.

[+] globular-toast|5 years ago|reply
I just wish it didn't target the JVM. I think if it had a native runtime that loaded quickly like Python it would be more popular. I love Clojure and really wish I could use it, but I just don't do the kind of things that can justify running a JVM.
[+] rodeph|5 years ago|reply
High praise indeed! Can you provide a source for it? It would be nice to be able to cite it.
[+] zitterbewegung|5 years ago|reply
Yea Clojure is like the Python of lisp style languages. Good language design and it can interact with Existing Java libraries. (Python does the same thing but with C).
[+] adamkl|5 years ago|reply
A great quote from the introduction, on why Rich took a sabbatical to work on Clojure:

...to give myself the opportunity to work on whatever I found interesting, without regard to outcome, commercial viability or the opinions of others. One might say these are prerequisites for working on Lisps or functional languages.

[+] microcolonel|5 years ago|reply
The results are also astounding, because what he came back with produced extreme commercial viability, and is excellent in the opinions of others.

Having used Clojure professionally for a couple years now, I feel so blessed. It is really well cut out for a whole host of problems that are common but extremely labour-intensive in conventional languages.

Maybe one of these days I'll get time for my own sabbatical, and solve the two major tasks I see with Clojure: taking the performance from good to excellent; and hosting it in a robust systems language suitable for implementing well-trodden functions.

[+] omginternets|5 years ago|reply
A similar point was raised in his Hammock-Driven Development talk. It inspired me to follow suit during this COVID19 lockdown.

I've never been more intellectually fulfilled, nor happier.

[+] SeeTheTruth|5 years ago|reply
For working on the languages themselves for sure. But thankfully working with functional languages helps with outcomes and commercial viability.

If only languages like Clojure, F#, OCaml, Kotlin, and many more enjoyed more popularity and support.

[+] simongray|5 years ago|reply
> When I first was learning Common Lisp I was appalled at the tolerance on Usenet comp.lang.lisp of arrogance, hostility, intimidation and other anti-social behavior of the presumed ‘smartest guys in the room’. I was determined (a) that the Clojure community would not be hosted there and (b) that kind of behavior would not be tolerated.

Thank you, Rich Hickey!

The persistent data structures and functional core library are great, but many would not bother with Clojure if they had to deal with the toxicity that is present in the Common Lisp community.

[+] bjoli|5 years ago|reply
Comp.lang.lisp was awful, and the trend somewhat continues even though it has gotten better. Looking through newb threads on various lisp forums there are a few people that always seem to try their best to be mean or funny on someone else's expense.

It is not that hard: if someone writes awful code, point them in the right direction. Don't claim they have malfunctioning brains.

I stopped writing CL a few years ago even though I enjoyed it. For my personal stuff scheme is even more fun, and the communities are a lot more pleasant.

[+] rafaelferreira|5 years ago|reply
I've been working with Clojure for a few years now, and the thing I love most about the language is how it nudges developers to write straightforward code. It's maybe closer to a well-structured procedural style than to highly indirect OO or to deeply polymorphic category-theory-inspired typed functional programming. The emphasis on mostly-first-order pure functions composed of other functions, receiving and returning immutable values, makes it easy to untangle even the worst balls of mud, while the well designed set of data-structures (few) and functions operating on them (many) allow for easy and terse expression of everyday data manipulation chores.

If we keep the majority of the code pure and straightforward, we can deal with state and I/O with other constructs, like atoms and records, only where we need to (doing this systematically tends to lead me to some variation of the functional-core/imperative-shell pattern). STM, multimethods, macros and whatnot are cool and have their uses, but after a while I think the really cool thing is how little do we need to resort to fancy stuff and how much is accomplished with just functions and data.

[+] leethomas|5 years ago|reply
> functional-core/imperative-shell

Lol I’ve been learning Clojure for fun these last few days and I had to read this twice because the first time I thought I was looking at a function from a module.

> we can deal with state and I/O with other constructs, like atoms and records

Can you elaborate on what you mean here or link to specific articles? I’m talking about the representing I/O side effects as records part.

[+] elamje|5 years ago|reply
I was super excited to see this paper released. I've spent time with Python, Java, C#, Javascript and limited time on C and Rust. Clojure was the largest paradigm shifter by a large margin.

Having done concurrent programming in Java, and even implemented some concurrency primitives like semaphores, Clojure was incredible to use. For those that don't know, Clojure uses immutable data structures that mitigate concurrency issues. If you are curious how that's even practical, I highly recommend the paper and also reading about Persistent Data Structures. They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.

On top of that, you up reducing lines of code by 60-70% of traditional Java or C#.

[+] adambyrtek|5 years ago|reply
> They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.

I learned about immutable data structures at Rich's conference talk years ago, and the whole concept blew my mind as well. However, what you said above is not strictly correct. The idea is not to store deltas (which is what Subversion does), but immutable copies of objects (in case of Git identified by their SHA-1) that can be used to build multiple versions of a hierarchical data structure (e.g. a list, tree or hash map). Git commit is basically a tree of other objects with some metadata. The problem with deltas is that they need to be iteratively applied to some base snapshot which requires computation (and probably a cache as well), while immutable objects are easily accessible in constant time.

[+] jwr|5 years ago|reply
Thanks to Clojure (and ClojureScript), I was able to build my self-funded SaaS business. There is no way I would have been able to tackle the complexity without Clojure.

I also love the mature approach and mature community. It is, quite simply, a different world from most other software places, and one I enjoy a lot, as I'm mostly focused on building and shipping a large and complex codebase with an extremely small team.

[+] GiorgioG|5 years ago|reply
Can you elaborate on how Clojure helped you tackle the complexity? I've considered learning Clojure (I have the Brave Clojure book sitting not far from me) but I have concerns about the learning curve (I'm a fullstack dev by day (C#/Angular) slowing me down in my own attempt to build a SaaS.
[+] roryrjb|5 years ago|reply
I see nothing but praise for Clojure and for good reason. I experimented with it a bit a few years ago and it seemed to be a more practical and useful version of Lisp, although I say that as someone who wasn't that experienced in Lisp more generally, with only some small experiments in Scheme. Something just as significant was that the ecosystem seemed to be quite sane and I got on really well with Leiningen for example, if you were considering diving in then give it a go.

I personally won't consider it now but not because of the language or the ecosystem but because of the runtime(s). For me the layers of abstraction are too much complexity to not be justified.

[+] puredanger|5 years ago|reply
It seems that rejecting all of the most popular runtimes in the world (JVM, JS, CLR) is an odd thing to have a grudge about.

(Also notable, babashka, a Clojure alternative to bash.)

[+] LandR|5 years ago|reply
I love clojure the language but hate the tooling.

Also a lack of documentation was always a problem, especially around clojure script.

I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Hmmm. That attitude might put people off.

[+] manishsharan|5 years ago|reply
I kind of know where you are coming from. 'Back in the day', early Clojure adopters would ask one to read SICP before asking questions. SICP is indeed the best CS book I have read but that attitude was uncalled for.

But things have changed. Clojure and Clojurescript community is among the most helpful and Stackoverflow questions are promptly answered. If you take the time , like I did, you will find that Clojurescript has awesome tooling. Go though the tutorial on Figwheel.main (https://figwheel.org/docs/ ) and then check out re-frame TodoMVC here https://github.com/day8/re-frame/tree/master/examples/todomv... or this simpler reframe tutorial from here https://opengisgal.wordpress.com/2018/05/19/re-frame-clojure... . And this here is the Reagent cookbook https://github.com/reagent-project/reagent-cookbook . Good Luck!

[+] puredanger|5 years ago|reply
> When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Clojure is not meant to be hard to learn and you are definitely smart enough. There are many, many learning resources for Clojure in every form and it has one of the most helpful communities I'm aware of on Clojurians slack, etc.

[+] lvh|5 years ago|reply
> I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Where did you experience this? That is unacceptable in most of the Clojure communities I know, and the only big one I don't know personally is the subreddit.

(I am not suggesting the subreddit is toxic.)

[+] james-mcelwain|5 years ago|reply
What tooling has given you problems? I think that some tooling assumes familiarity with the Java ecosystem (which is a problem for many things in Clojure land), but otherwise it seems pretty solid to me.
[+] chrisulloa|5 years ago|reply
Clojurians slack is honestly one of the most welcoming and understanding programming communities I've had the pleasure of joining. I frequently ask stupid questions and will occasionally have library maintainers take their time to fill my gaps in understanding. The tooling complaint is valid, but it has been getting better every year. There are plenty of good books for Clojure too, like Brave & True that help you understand the documentation.
[+] mark_l_watson|5 years ago|reply
I used Clojure a lot back in the beginning (contributed a little money, used Clojure fairly intensely on two long term consulting projects). Before that I used Rich's Common Lisp to Java bridge project. I think his impact goes far beyond just Clojure users, having helped push immutable by default, etc. as standard good practices.

I moved away from Clojure because I wanted to move away from the JVM. I did a fun project, a nutrition/cooking web app [1] about 12 years ago in Clojure, but I am slowly porting that from Clojure to Common Lisp for another example program for my book [2]. Perhaps not as well known, but Clojure joins other less commonly used languages like Haskell as languages with fairly good support for deep learning. I feel like any language that does not have a good deep learning story will be limited. Also interesting that Swift has such good access to Apple's deep learning libraries and also a version of TensorFlow. I encourage Clojure developers who work on deep learning support!

[1] http://cookingspace.com [2] https://leanpub.com/lovinglisp

[+] lbj|5 years ago|reply
I've been here for the entire journey, but no one can summarize it quite like Rich. And I bet he could write twice that just on nurturing a community - He spend hours and hours on IRC in the early days, coaching, teaching, correcting and even admonishing bad behavior. He's truly done something remarkable in both building this powerful language that I use daily, and a very helpful, friendly and intelligent community around it.
[+] christophilus|5 years ago|reply
Clojure is hands down my favorite language that I've used so far. Rich, if you read these comments, thanks for all you do!
[+] lukashrb|5 years ago|reply
A great read! What I really appreciate, beside all the technical stuff, is the effort to create a positive and helpful community around Clojure.

From the paper: "When I first was learning Common Lisp I was appalled at the tolerance on Usenet comp.lang.lispof arrogance, hostility, intimidation and other anti-social behavior of the presumed ‘smartest guys in the room’. I was determined (a) that the Clojure community would not be hosted there and (b) that kind of behavior would not be tolerated. Early on, on IRC and the mailing list, I established and required a tone of supportiveness, positivity and respect."

[+] abraxas|5 years ago|reply
I fell in love with Clojure and fell out of it. It is a great language. Lisps have this inherent elegance to them especially in data transformation pipelines.

That said I found its performance still a bit on the wanting side particularly if you avoid type hints. For many tasks the performance is probably acceptable but the backend work I tend to do makes the abstract style get in the way of trying to figure out performance issues.

Also I'm coming around to the imperative style more and more. Sometimes a for loop is all that's needed and makes for a more understandable code than all those hipster maps and folds.

[+] nlitened|5 years ago|reply
What kind of backend work do you tend to do?

I am curious because I am doing real-time financial data processing with sub-millisecond latencies just fine in Clojure. Admittedly, there’s no heavy number crunching in my use case.

[+] ashtonkem|5 years ago|reply
I also found that Clojure is full of weird edge behavior. Sets are unsafe, and might change the type on you. Seq leaks like a sieve, so there are all kinds of places where you have to map seq, and type hinting is the worst. The moment you type hint a Protocol, all kinds of weird shit starts to happen, including the need to import the type hinted classes in the consumer (except Java.util.*), even if you don’t reference it. Oh, and if you type hint protocols wrong, it doesn’t error, but your performance remains bad.

At least it did in 2017. I haven’t used it since then.

[+] raspasov|5 years ago|reply
I can recommend YourKit. It's a paid tool but if you really need it, the small cost is totally worth it.

Usually the bulk of your perf. problem comes from a few hot places that are called often and the JVM needs to do reflection on those every time. Type-hint those and the performance usually goes through the roof.

[+] bpyne|5 years ago|reply
Section 2.4 of the paper gave me a chuckle. Like Rich, I've yet to write a program that didn't involve some kind of database. Organizations care about data not programs. Writing programs just gives them a way of applying organization-specific rules, in the way of transformations, to the data.
[+] imdhmd|5 years ago|reply
As an aside - Is Rich Hickey still not notable enough to have a wikipedia entry? Even the person who has shepherded on this paper has a wiki page.
[+] svat|5 years ago|reply
I've gone ahead and restored the article, but not knowing much about Rich Hickey I hope someone here can edit the article to actually contain some (sourced, i.e. with citations) content, as it's pretty bare right now (looks/looked like this: https://en.wikipedia.org/w/index.php?title=Rich_Hickey&oldid...).

The article was created in 2008 and its deletion was discussed and happened in July 2011: https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio... Someone recreated it (without discussion) in August 2013, and it was reverted to its "redirect" state in January 2014. I suspect that notability has changed now (e.g. Clojure itself is more popular, there are other things, and if nothing else his notability as a speaker is probably worth mentioning) and my guess is that if the article were to be discussed again, it might survive. But this is assuming there is actually content for the article that someone can add to it! Otherwise what's the point of an article about a person that doesn't say anything?

If you look at the other articles mentioned in this thread so far (https://en.wikipedia.org/wiki/Guido_van_Rossum, https://en.wikipedia.org/wiki/Larry_Wall, https://en.wikipedia.org/wiki/James_Gosling, https://en.wikipedia.org/wiki/Martin_Odersky, https://en.wikipedia.org/wiki/Rasmus_Lerdorf, https://en.wikipedia.org/wiki/Yukihiro_Matsumoto, https://en.wikipedia.org/wiki/Don_Syme, https://en.wikipedia.org/wiki/Rob_Pike) all of them contain some information about their article's subjects, beyond simply mentioning the language they were involved in.

In my experience, adding something well-sourced to Wikipedia is usually a more lasting contribution (probably more people will read it, over time) than comments on HN -- the only downside is that it may get reverted quickly on flimsy grounds and you need some experience to make good edits that will stick.

Edit: Or just mention here what can be added to the article and what the source of each bit of information is, and I'll try to add them!

[+] JBiserkov|5 years ago|reply
Given Wikipedia's bizzaro rules, I wouldn't be surprised for him to get a page as an author of the paper ;-)
[+] lymeeducator|5 years ago|reply
Thanks to Rich for designing and thanks to Alex & Rich (and others) for maintaining. It is my favorite language thus far out of (Java, Ruby, Python, Bash, Awk, Erlang, Swift, Tcl and Objective-C).