top | item 10657435

Metabase: Why we picked Clojure

179 points| tlrobinson | 10 years ago |medium.com

183 comments

order
[+] kisstheblade|10 years ago|reply
" However, we had a strong aversion to using Java itself, so the real short list was Scala or Clojure."

Sadly they couldn't articulate any actual problems with Java...

I find it weird that so many developers focus on the "writability" of the language. I mean stuff that java gets criticized for, like verbosity etc. Which I don't even find so bad especially since the newer jdk versions.

But in my opinion the _readability_ of the result is the most important factor. I rather read copy/pasted code than some smart clojure onliner which nobody dares to touch or really understands.

You only write the program once, but you have to read it over and over again when maintaining it and developing new features, and java's maybe a little verbose syntax really helps here, and also the fact that there really is only one way to do a thing (unlike eg. perl).

The tooling, documentation, frameworks, libraries and runtimes are all top notch with java. All other languages seem to have problems in one (or more) of these areas.

[+] pjlegato|10 years ago|reply
After you've been writing it for a while, reading Lisp / Clojure at a glance is not nearly as scary as it seems at first. You get used to it, and that one-liner becomes a paragon of elegant simplicity rather than a strange collection of symbols.

It's just a different mode of scanning than what we've become accustomed to by working in mainly ALGOL-derived languages. With practice, it becomes second nature (and ALGOLs seem needlessly verbose by comparison :)

[+] lostcolony|10 years ago|reply
'Couldn't articulate' vs 'didn't articulate' are two different things.

The tooling, documentation, frameworks, libraries, and runtimes are all there if you want to use them in Clojure, too. So that's not a point in Java's favor, just the JVM's favor.

It is quite possible to write reasonably readable, terse, correct Java programs. I rarely ever see them in the wild though; what I've mostly seen are klugey Spring or JavaEE based monstrosities that are horribly inefficient, and prone to hard to debug concurrency issues.

The average results seen with the language -is- a consideration; while it's hardly scientific (the teams choosing Java tend to be from all walks, and disproportionately from large enterprise teams where no one gets any real say in the matter, that's just the common denominator that management has allowed; the teams choosing, say, Clojure, tend to be smaller and made of good developers who are intellectually curious, and have found themselves in places they can make that decision, so obviously the average for the latter will likely be better), it's still a consideration. If you've written in a wide range of languages, you've -seen- this in yourself; what the language makes easy vs what it makes hard influences how you do things.

[+] j-pb|10 years ago|reply
Clojure allows me to express thoughts that I wouldn't know how to express in for, while, if, variables and objects.

And on readability, what do you prefer?

  List<Integer> even = new List()
  for(int i = 0; i < 100; i++){
    if((i%2)==0){
      even.add(i);
    }
  }
  return even;
vs.

  (filter even? (range 0 100))
[+] a-saleh|10 years ago|reply
In my experience, the readability of Java is not that great, perhaps due to verbosity. Especially when you want to create a DSL with some type-safety guarantees.

Wrapping a head around a factory that is configured by fluent-like-api, with interfaces scattered around 10 different files usually took me longer than understanding idiomatic clojure.

When I was working in clojure, only thing I was really missing was the type-checking. But when I work in java, i miss the REPL much more, than i missed type-checking in clojure :)

[+] vosper|10 years ago|reply
The small amount of development I've done with Clojure convinced me that you just can't ignore Java - you're going to end up spending time reading Java docs, and debugging Java stacktraces. If you really dislike Java I don't think choosing Clojure is a great option, either (I don't know about Scala).
[+] jerf|10 years ago|reply
"Sadly they couldn't articulate any actual problems with Java..."

The first thing they said about Go applies to Java: "Go felt promising but too low level for us to express our query language productively."

You certainly can write that sort of code in Java, as evidenced by the fact that many people have and continue to. But you can get a very big win on that sort of code in a lot of other languages, especially if you're resource-constrained.

[+] lmm|10 years ago|reply
I find the readability is where Java's verbosity really hurts it. E.g. if you bring up the horribly verbose getters/setters in Java many people will respond that you can autogenerate them - which is true as far as it goes. But you can't "autoread" them, and if you have a thousand getters/setters of which ten provide different functionality to the standard field access, that makes the code very hard to read.
[+] wwweston|10 years ago|reply
For me, readability is directly tied to good abstractions (and meta/programming facilities for making good abstractions) and a certain degree of conciseness.

If I have to read 5-10 lines of intermediate code to figure out what's going on, it's less readable than if there's a short expression.

Some languages seem to be driven by fear that if you provide programmers powerful facilities for abstraction that rise to the level of augmenting the language that developers will no longer be able to figure out what's going on without reading a lot of code, so they solve this problem by forcing developers to write (and therefore read) a lot of code.

Java seems to be one of those languages.

[+] lectrick|10 years ago|reply
Yes, Java is a mature language.

But any perusal of rosettacode.org is fairly indicative that other languages, while less mature, have adopted some good ideas.

For example, here's Quicksort in both Java and Elixir:

    public static <E extends Comparable<? super E>> List<E> quickSort(List<E> arr) {
    if (!arr.isEmpty()) {
        E pivot = arr.get(0); //This pivot can change to get faster results
     
     
        List<E> less = new LinkedList<E>();
        List<E> pivotList = new LinkedList<E>();
        List<E> more = new LinkedList<E>();
     
        // Partition
        for (E i: arr) {
            if (i.compareTo(pivot) < 0)
                less.add(i);
            else if (i.compareTo(pivot) > 0)
                more.add(i);
            else
                pivotList.add(i);
        }
     
        // Recursively sort sublists
        less = quickSort(less);
        more = quickSort(more);
     
        // Concatenate results
        less.addAll(pivotList);
        less.addAll(more);
        return less;
     }
    return arr;
     
    }
now Elixir:

    defmodule QuickSort do
      def qsort([]) do
        []
      end
      def qsort([pivot | rest]) do
        { left, right } = Enum.partition(rest, fn(x) -> x < pivot end)
        qsort(left) ++ [pivot] ++ qsort(right)
      end
    end
The advantage here is NOT just an "economy of typing."
[+] Nicolas0682|10 years ago|reply
The readability argument is overused and maybe misunderstood to me.

An experienced programmer is supposed to know the language well as well as the API and frameworks.

From my experience Java is quite verbose and the standard frameworks are among the most complex. It is typical for a clojure/scala/python program to require 1/2 or 1/3 the lines of codes than Java. And they require much less configuration files and annotations.

How much time a developper spend to read his operating system code? How much time a developper spend to read the API implementation code of his language or even the framework ? No time because the code is great, work as expected and doesn't require to be read all the time.

That's what a more expressive language provide. At greater set of standard language construct, API and frameworks that allow you to express the same in a much more concise way. This is much easier to read because the fundations are rock solid and the meaning of a function doesn't evolve over time, it is also nearly bug free. You reduce the accidental complexity.

When you choose a less expressive language like Java, even with the most advenced frameworks you have much more unreliable custom code. You are in fact increasing the readability issue, not reducing it.

Sure there the fact that many more developper master Java and the associated frameworks than say Scala or Clojure. But this also act as a filter.

You can do the same thing with a team of 5 experienced clojure developpers than with 20-30 java dev. A part come from the language expressivity itself. A part is that it is much more likely to get bad developpers in Java that will mess your code and make you loose time. It is easy to find Java dev, a bit less to find great Java developpers. Another part is that the bigger the team, the more is lost in communication, meetings and politics than actually doing things.

So that your choice ! Do you prefer 5 clojure dev, maybe paid 20-30% more or 30 java dev with 5 managers (paid 20-30% more) competiting to get budget and projects ?

[+] tomjen3|10 years ago|reply
This may be due to Android being limited by Java 6, but I find Java code to be so verbose that it hurts readability. Unnecessary try/catch/ignore just to close a resource, tons of factories, classes with static methods because you can't add the functionality to string, etc.

What weirds me out is that they didn't even consider Kotlin an option, since it is essentially a language so that you can code Java, but without the javaness.

[+] jiyinyiyong|10 years ago|reply
Well, not everyone thinks Java is easy to read. I've doing my job with JavaScript and CoffeeScript for more than two years and I still hates JavaScript, let alone Java. And CoffeeScript is way better than it, even it's simply just some syntax sugars. I'm learning Clojure and it's even more expressive than CoffeeScript and that's why I like reading Clojure.
[+] kisstheblade|10 years ago|reply
Ouch, got downvoted in under a minute. Guess it's not cool to say anything positive about java! :) I still would have liked it if the authors of the article had said something about why they had a "strong aversion" for java.
[+] cobaltblue|10 years ago|reply
Clojure wins on readability as well as writability. The easiest way it does that is by drastically reducing the amount of code you have to write for whatever random thing you'd otherwise do in Java. It opens the door even further for additional savings compared to other languages in a similar place like Python or Ruby by providing macros.

Freedom from the Kingdom of Nouns alone gives you a big win in readability. Naming things is hard, and when the thing you care about isn't the Class but the Method then bothering to name a wrapper Class is additional overhead, an extra "what is this in my code?" that you need to make room for in your brain -- additionally it's common practice in Java world to put that Class in its own file all by itself, often in a totally different namespace, and often physically far away from where it's actually used when looking at the files on the filesystem, which further increases the cognitive load of knowing where your code lives. IDEs in the land of Java are helpful only because they are necessary because without them the cognitive load is just too much for any reasonably complicated project.

Freedom from mandatory static typing of the sort Java has saves you from pointless class wrappers, pointless POJOs that must have names and live in their own files and namespaces, you are free to just use maps of keywords to values. Free to work with the data directly rather than an ad-hoc aliasing scheme that might decide to hide things you need or surprise you with nulls. When you want protocol or interface adherence, it is there for you, but it is not mandatory for everything.

Maintaining Clojure is easier because there's less of it. A new hire can be pointed to your code of n files and told to spend a week reading and playing in the REPL, versus pointed to n2 files and told to spend a month reading and inserting breakpoints here and there to understand all the chaotic flows across what and where. You have a legitimate shot at picking 10 random Clojure files out of the project and understanding a bit about the data flows in them in relationship to the project as a whole, but in Java you're going to need a lot more context, especially if you get unlucky and pick out 10 MyCustomException classes. What is there in Clojure can be very dense, but it can be apparent in code review when you went too far, and then it's easy to refactor and you don't need any auto-refactor tools. You just substitute expressions, add comments, and name only what needs to be named, you can do it all without constantly needing to restart your program because (unless you're using JRebel) you added a function here or changed a parameter there.

Java is good for risk-averse corporations -- a single average Dev on an average size team can't do very much, many corporate programmers average around 1000-2000 lines of code per year* even with their IDEs helping them, and because of that they can cheaply be replaced or moved to another part of the project.

[+] iagooar|10 years ago|reply
It's great to see such an "exotic" programming language like Clojure getting some serious traction. And by exotic I mean a language that stands out from the imperative or object-oriented crowd.

I like how Clojure challenges established paradigms with a surprisingly mature ecosystem.

Being a hosted language on the JVM has its pros and cons, but the cleverness and pragmatism of Rich Hickey (creator of Clojure) really stands out. He decided to go with an industry strong, battle-tested environment, effectively standing on the shoulders of giants, while providing a lot of fresh ideas.

Also, the fact of being a Lisp can be seen from different angles: for some it's lots of weirdly nested parentheses, for others it has a beautifully simple, yet extremely powerful and expressive syntax.

But don't let those aspects fool you. Once you scratch a little bit the surface, you will see the extraordinary power of its persistent data structures. For those who don't know what persistent data structures are: in Clojure, oversimplifying, there is no mutation to data structures. Every time you "add" or "remove" a key from a hash map, you are actually creating a copy of it. And in order to make this efficient, you need special data structures with "superpowers", that are not actually creating a whole copy of themselves, but rather modify atomically only a part (hint: trees).

And having those data structures leads to the next big feature of Clojure: concurrency. Not only does Clojure provide a mechanism to dispatch thread-like constructs (core.async), but the whole language is designed around supporting concurrency.

I also want to add that what really surprised me about Clojure is its thriving community and smart people using it. I am pretty sure that if you want to hire great, highly motivated talent, going Clojure will help you with that.

[+] troym|10 years ago|reply
> Once you scratch a little bit the surface, you will see the extraordinary power of its persistent data structures.

I've heard this repeatedly, yet in the examples I've read (for web servers and for web ui work with cljs), they're using atoms with `swap!` to provide "current state".

Could you provide an example of using persistent data structures in a web app? Or am I mis-understanding?

[+] mark_l_watson|10 years ago|reply
I read this article yesterday because I am having a language choice issue, between Ruby and Clojure.

I am in my (almost) mid 60s, and except for being on retainer for advice I don't really accept consulting work anymore. I have a long list of projects that I want to do, mostly open source, and settling on one language for a few years would make things easier.

Clojure and Ruby both support DSL development very well which is important to me and both play very well with Java (via JRuby). I am starting to ask myself though if I wouldn't just be better off with Java 8, even without good support for writing DSLs. I have mostly stopped using Haskell and Scala.

When I was much younger I totally enjoyed learning and using many programming languages but as I get older I feel more like just getting stuff done, especially since I dedicate much less time now to technical pursuits. So in a sense I am facing roughly the same type of decision as the Metabase people.

[+] daxfohl|10 years ago|reply
I follow your blog. I'm a month from 40 and agree. After several years working mostly in Clojure, Scala, and F#, I am finding myself most productive in traditional languages even though they're less interesting. And am finding projects more interesting than implementations, which is probably the way it should be.

I'm finding now that OOP is actually a pretty good model for most things. Loops and mutation reflect reality: you do a task N times and write down each result; you don't take the first N results from an infinite seq of tasks. IoC is useful. In addition having a much larger set of things to copy/paste from github/stackoverflow is nice.

That said, most (all?) of my paid work is pretty banal, nothing that's really begging for functional style. And for whatever reason I've never had much success with REPLs. Getting them updated correctly with the code and dependency changes, typing in what I want to test, then running it always seems to take me longer than just executing the app and testing changes directly.

[+] nkassis|10 years ago|reply
I'd personally look to find what language fills the gap that the current set of languages I know don't fill. Clojure is my filler for lisp language and language with good macros support.

I use Ruby in the slot that many have Python in but with added coolness of metaprogramming not found in most scripting languages.

And to add to that a bit, I found that recently it's been getting harder finding new languages with enough new patterns and paradigms to keep me interested in learning and mastering them.

[+] bribri|10 years ago|reply
Out of curiosity, why not Haskell or Scala if you already know them?
[+] RyanZAG|10 years ago|reply
> .. However, we had a strong aversion to using Java itself ..

Honestly, why? Seems odd given Java's great track record for exactly this problem and huge library and developer share. The code itself is forced to be static and explicit which makes for easy onboarding for new developers.

It's a bit strange to just dismiss Java in the context of the rest of the article. You're kind of staking your company on decisions like these, and making them because of some unquantifiable 'strong aversion' seems like a poor choice.

That said, Clojure or other lisp base definitely seems like the correct choice for a query language / dsl. That's the shining point of lisps after all.

[+] daxfohl|10 years ago|reply
I think the popularity of yesql (https://github.com/krisajenkins/yesql) calls your last point into question. Indeed in my experience, using korma (the most popular clojure SQL-DSL) provided no additional benefit, just additional documentation to read.

I've gone through the FP language snob mill over the last few years, and have to agree -- I don't love it, it doesn't make me feel all chipstered out, but Java is actually a pretty decent language for getting stuff done.

[+] zem|10 years ago|reply
personally, i find java pretty painful to use; it's definitely capable of doing whatever job you have in mind, but when i use it i have the constant feeling i'm using a suboptimal tool. and then it leads to reams of boilerplate all of which has to be maintained, so it's not just a development-time cost; there's the depressing feeling of having to deal with it every time you want to read the code. clojure would be worth it just for the joy of using a tool that fits well into your hand.
[+] sgdread|10 years ago|reply
If there's no single language lock-in, then using polyglot approach, you can get best of both worlds: write DSL parts in Clojure and other parts in Java. We did this recently in one of our porjects (Clojure piece was responsible for interacting with ElasticSearch).
[+] dustingetz|10 years ago|reply
"So, in summary, we chose Clojure for JVM threads + Database drivers, the ability to ship an uberjar and the ease of expressing tree manipulations which were the core piece of complexity in our backend."

for people who may not know clojure here is an example of tree manipulations, it's really powerful especially in UI programming which is all about manipulating recursive structures: https://gist.github.com/dustingetz/c11ae4edb9e0d14787d2

Scala has answers to this too (I would love for someone to reply with the best scala way) but for whatever subjective reasons the clojure way makes me happier for ui work.

As to scala pushing towards typed object mappers, here is one example of generating untyped queries with Anorm [1], I'd be surprised if this couldn't be made to work for their query language, but that said the scala ecosystem has a lot of, shall we say "diversity of opinion" about the Proper Way To Do Things, so I am not surprised that newcomers to the language had a bad time.

[1] (2013) http://www.dustingetz.com/2013/03/26/orm.html

[+] staticelf|10 years ago|reply
This is probably going to be a very unpopular opinion here, but I think Clojure is a horrible language to work with.

I don't have the experience with non imperitive languages which makes learning clojure a really steep curve. Also the error you can recieve with clojure are many times a big mystery, I've never seen as bad error reporting as I've seen with Clojure. Other issues I have with it is that Clojure-fans tends to want to write EVERYTHING in clojure or lisp style code. That includes stuff like Javascript, HTML and similar stuff. Things that just makes it even more complicated.

I have really tried to learn it several times but got so frustrated I quit. We have a small thing at work which is written in Clojure and no one knows how to fix issues with it since no one has any experience with Clojure.

Clojure does not fit my brain.

[+] mpdehaan2|10 years ago|reply
Kind of shocked that building database libraries for python (homebrew install gcc? pip install foo) was "hard" for them.

I find the Python ORM options (sqlalchemy/Django/etc) lot better evolved than what is available for clojure and couldn't image that being a roadblock. Korma is basically a query builder and lacks a lot of things I was used to, and I had to engineer a declarative layer for defining models in the ways I wanted.

Pain in setting up developer machines could be simplified with an ansible playbook pretty easily.

OTOH, one of the worst things I hate is trying to figure out how to automatically get around Sun's paywalls and fiddle with Java classpaths :)

I do feel some level of want to avoid vagrant in the development workflow, in which cases I'm usually fond of just logging into a real VM. I do think it's potentially good for tests.

IDK, I always want to solve the idea of "fragile deploys" with immutable systems in prod. Uberjar was cool, I'll give it that -- but the startup time of clojure was often not worth the wait.

My biggest complaints with clojure is documentation quality, inconsistency in return types (or just willingness to return nil in error scenarios), and tracebacks that are very difficult to sort through.

Clojure also presents some problems with the idiomatic nature of trying to roll up everything into one statement, whereas adding debug and debugging require unrolling it, than packing statements up again to be socially acceptable idiomatic clojure.

Testing in clojure also strikes me as painful, as it tends to punish you with "let" causing an indentation every time you need to assign a variable, something that is useful in tests that want to check results a lot along the way.

My gut feeling is clojure was good for a little light scripting on top of a big Java codebase (kind of like you might use Lua in a game engine) but would not be something I'd pick for a primary language. Primarily using Java libs would also solve some of the ecosystem problems.

[+] vosper|10 years ago|reply
> Python’s Mysql and Postgres database drivers required compilation. This made both developer machine setup as well as deployment more complicated than it really needed to be.

Am I missing something - this was really a such a problem that it was a factor in switching languages?

[+] eterm|10 years ago|reply
I've started interpreting a lot of these "Why we use X" posts as less 'bragging' and more these companies trying to justify their own choices. In part because it perhaps makes them feel less isolated in their choices, and in part also because by hoping that others follow them they'll find it easier to recruit, something that is easily underestimated especially outside of major hubs such as NY, SF, or London.

Perhaps I am being uncharitable. I note with interest they say, "However, we had a strong aversion to using Java itself", but don't explain why they had that aversion, or whether it was based on anything but the fact that Java isn't trendy.

[+] jeffmk|10 years ago|reply
Yes. It's also a problem when you're deploying to, for example, EC2 instances that are too limited -- say a t2.tiny for proof-of-concept work -- to compile the Python lxml library.

That's after you figure out which 50 different *-dev libs you have to sudo apt-get install, in a tedious process of trial-and-error, ./configure, make, and error.

With Clojure this is a non-issue. lein will download what dependencies you need. And they're all already compiled into .jars. This is a benefit of the JVM infrastructure, but we're comparing Python here, so.

And of course you can upgrade the EC2 instance, but an annoying and time-consuming process, since you're not in control of the AWS account yourself, and the entire issue doesn't exist if you're using Clojure.

[+] salsakran|10 years ago|reply
It sounds innocuous, but we were planning on supporting a lot more databases, and when you start needing to build drivers for MySQL, Postgres, SQL Server, Oracle, Redshift, etc etc anything that makes this step more brittle starts to add up.

There are a lot of things that are mostly painless if you're deploying and supporting a single instance that become nightmares at the scale hundreds or thousands of installations you don't control.

[+] willvarfar|10 years ago|reply
It matches my own experience too. Libraries that need to compile c are a deployment nightmare.

Installing Oracle client is even worse though ;)

[+] astine|10 years ago|reply
It might be if you were planning to deploy to end user's machines.
[+] giancarlostoro|10 years ago|reply
One of the things that I love about Clojure is Leiningen. It just takes away the headaches of setting up projects in Clojure. I think the only other tool I'm aware of for another language that is quite as useful is probably NuGet. I've never had issues using lein, just drop it into my ~/bin/ folder (I'm using openSUSE so that folder is in the system path by default), open up a terminal and type in lein and it works. I don't even need clojure installed, just the JVM, it will download clojure for you as needed!

Props to those working on lein.

[+] tunesmith|10 years ago|reply
Can someone expand on what they mean by lightweight threads for Clojure? My understanding is that lightweight threads (green threads) is what java/jvm explicitly moved away from way back when; it's user-mode threads bound to one processor, so if the processor gets busy, then all of the lightweight threads for that processor get blocked. Meanwhile, Java/jvm uses kernel threads and context-switching so you can do real multi-threading.

I'm aware of how Quasar purports to be the best of both worlds, and it looks like Pulsar is a Clojure API for Quasar, maybe that's what they're using.

It doesn't appear Scala has something comparative, but I have a hazy sense that Scala people believe that Akka and monads are better for easy asynchronous programming.

[+] stcredzero|10 years ago|reply
Now, choosing a programming language for a project is usually decided on a mix of what a team is most comfortable with, beer-fueled debates on dynamic vs static typing, benchmarks of marginal relevance and that elusive quest for Dev-Hipster points.

How about this for an interview strategy: Have these discussions. Hire the people that realize most of the above mentioned things are meaningless.

(Clarification: Only the "comfortable" part isn't meaningless. But there too, it's only as meaningful as the placebo effect is significant.)

[+] ionforce|10 years ago|reply
How is "dynamic vs static typing" meaningless?
[+] throwaway999888|10 years ago|reply
I don't think an interview strategy of the form "has the same values and opinions as stcredzero" is going to work for everyone.
[+] biokoda|10 years ago|reply
Considering their criteria Erlang/Elixir seems like a huge omission from the contender list.
[+] melipone|10 years ago|reply
Lisp has been around a while, Java has been around a while. Clojure is the perfect combination and I predict it will be around a while.
[+] melipone|10 years ago|reply
Lisp has been around a while. Java has been around a while. Clojure is the perfect combination and I predict it will be around a while. It's exhilarating to program in Clojure. That's the only word I can find to describe what I feel to program in that language.
[+] moomin|10 years ago|reply
I had this conversation on twitter already: typed object mappers seem like a questionable choice for the problem space. Since it seems like all of their work in Scala used them, I'm not surprised they hated it.

(Disclaimer: I have no particular liking for Scala.)

[+] daxfohl|10 years ago|reply
I'd be interested to understand how Korma solves their problem. It's macro-based so I wouldn't think it would be particularly good at dynamically restructuring queries based on runtime variables.
[+] lectrick|10 years ago|reply
Why wasn't Erlang/Elixir even considered? It checks off all their initial set of requirements
[+] programminggeek|10 years ago|reply
You know what else is easy to work with and deploy? PHP.
[+] johansch|10 years ago|reply
Has anyone here ever heard of Metabase before? Their github page indicates there are 3 (three) contributors. Wouldn't it be safe to assume this post is primarily a marketing venture?