top | item 6388512

Where my mouth is

134 points| momo-reina | 12 years ago |jao.io | reply

80 comments

order
[+] mynameisme|12 years ago|reply
Saying Clojure is a counter example kinda ignores the big reason companies don't want to use Lisp/Haskell/etc. When you're on the JVM, you get to use that gigantic collection of Java libraries and Java community for almost no downside. With Common Lisp/Scheme, so much time is spent reinventing wheels due to smaller communities that they're not worth using for larger projects, in my opinion. Also with Clojure, you always fairly easily port to Java if you decide things aren't going as planned a couple of years down the line, as CodeHaus did with Scala. Don't get me wrong, I think Clojure is by far the best Lisp to choose out there for a product, but that's mainly because it has its big "Java ogre" brother standing behind it.

edit: And also, you don't need to write Java in a factories on top of factories style just because some choose to do that. The same is even more true of xml configuration.

[+] klibertp|12 years ago|reply
> With Common Lisp/Scheme, so much time is spent reinventing wheels due to smaller communities

I'm not sure, but I think that C and C++ ecosystem has at least the same amount of libraries available as Java. And the reason why I mention this is that many Lisps have FFI which makes integrating those libraries almost seamless. Aside from Gambit-C, which compiles Scheme code to C and so supports inline C code, for example Racket has very nice DSSL for FFI, which reduces the need for glue code to one or two lines of code per imported function.

Integration with native Java libraries is easier from JVM languages, of course, but then there are many Lisps for JVM too. And for CLR, where tail call elimination exists. So if you want to write in Lisp, but are worried about lack of libraries, you really have many options. Clojure is alright, but saying that it's the only choice if you need access to libraries is not quite true.

[+] lalc|12 years ago|reply
I have to admit I'm ignorant as to what Java libraries I'm missing out on by not using a JVM language. I hear about this advantage a lot though! What are some examples of good Java libraries that aren't as good in other common languages?
[+] LanceH|12 years ago|reply
That doesn't explain the original adoption of Java, though. Put a billion dollars of marketing into Clojure and see where that gets you. Just the marketing effort alone. Do you guys remember all the managers, directors, secretaries taking Java courses because it was going to change everything? Java rode a big wave of stupid that was the internet boom.
[+] 616c|12 years ago|reply
Wish I could up this one more. I am in the process of playing with Lisp (in my case, Common Lisp, the SBCL variant to be specific) and I would love to see people with production services in any Lisp, or even Scheme.

I have been very attracted to the idea of Lisp, specifically as I discovered teepeedee and teepeedee2 as being some of the first truly fast web servers [0], and now I found wookie,[1] an evented web server running on top of libevent. I really want to play with that.

The closest thing I know of to people running "prod services on Lisp" are the Untyped guys in the Racket Scheme community [2], and not much else. I know pg and company became famous on Viaweb and Lisp, but since then there are scarce examples. However, the amount of bitching libraries and cool books (pg's book included) make me want to buck the trend and do it anyway, even if no one watches and I end up in my own ditch writing code to dig my way out for the all the unstable or non-existent pieces of code waiting for me.

[0] https://github.com/vii/teepeedee2

[1] https://github.com/orthecreedence/wookie/

[2] https://github.com/untyped

[+] sramsay|12 years ago|reply
"I also think that Clojure is particularly enjoyable, elegant and beautiful, while at the same time extremely practical, easy to learn, and well suited to modern application development; it is the best designed programming language I have encountered in my almost-two-decades-long programming career . . ."

I am absolutely with you there, but after writing a reasonably serious, production-level system in Clojure, I suddenly realized a fatal flaw in this and in many other hip and cool languages. It is very, very difficult to write clear, readable code.

I accept that it might just be my lack of skill, but as the stuff I was writing became more complicated, I found myself writing the Lord's prayer on the head of a pin more and more.

In the end, I started to actually long for a language that had fewer language features -- something that allowed far, far less flexibility in the way ideas are expressed. C is far from perfect, but more and more I see the wisdom of one of the early statements in the K&R in which they say that figuring out what to remove is part of the art of creating programming languages (or something like that).

There's no language in which you can't write unreadable gibberish, granted, but some of these newer languages make it damn hard to write non-trivial programs that are easily read by others.

[+] tikhonj|12 years ago|reply
C has quite a few features, and the language design is pretty arbitrary. (Yes, it's often based on (old) hardware, but then hardware design is pretty arbitrary.)

I haven't used Clojure much, so I'll mostly talk about Scheme. I suspect the two are similar in all important ways.

If you look at the matter critically, you'll find that Scheme (and, I suspect, Clojure) are actually simpler and smaller languages than C. Neither has nearly as much syntax. There is far less undefined behavior. C has way more corner cases than Scheme: there are a ton of C quizzes out there which basically nobody can solve; I couldn't imagine much of that for Scheme. You just have to memorize the relatively obscure rules for what types get automatically coerced to each other.

C has whole classes of hard-to-solve bugs that other languages simply do not. Bus errors, segmentation faults, buffer overflows... All the exposed low-level details obscure the meaning of the code; unless they're important to your application, they just make your code harder to follow.

Moreover, common C idioms (some of which are just workarounds for missing features) make code much harder to follow than it should be. Thanks to the use of out parameters, you never really know where the result from a function is, without reading the documentation. I recently wrote some code that depended on ioctl--that single function sometimes returned the result as the return value and sometimes modified fields inside the struct you passed into it! Hard to follow is an understatement. Errors are signalled as integers instead of some semantically meaningful data type.

Of all the languages I know, C is the one that makes it the hardest to write non-trivial programs that are easily read by others. I suspect you're just making the common mistake of confusing familiarity for some sort of innate quality.

[+] dustingetz|12 years ago|reply
"a language that...allowed far, far less flexibility in the way ideas are expressed"

that is a design goal of Java, and it turns out that "everything is a class" is a really expensive way to write software. As in large systems take thousands of man-years and $100MM and contain AbstractFactoryFactoryProviders. Not because the engineers are bad, but because that's what you end up with when everything is a class.

Sometimes, Scala codebases are criticized because they use monads or have too many arcane unicode symbols in them. But I'll take scalaz over AbstractFactoryFactoryProviders any day. At least scalaz makes sense.

[+] ww520|12 years ago|reply
Amen. People often don't realize we read code far far more than we write code.
[+] cmccabe|12 years ago|reply
Yes, absolutely. Anyone can add features to a bloated language. The most beautiful designs know what to leave out and why. C was one of those designs.

If you like the C design aesthetic, you should check out Go. It's like a garbage collected and yes, memory-safe version of C, which substitutes composition for inheritance and goroutines for threads.

[+] pron|12 years ago|reply
Far be it from me to say that the choice of programming language doesn't matter, or that picking one slightly off the trodden path is not a sign of a certain openness of mind. I also think that Clojure is particularly enjoyable, elegant and beautiful, while at the same time extremely practical, easy to learn, and well suited to modern application development; it is the best designed programming language I have encountered in my almost-two-decades-long programming career (with Julia giving it a run for its money if only it had been implemented on top of the JVM as well).

However, I've seen a clear inverse correlation between the magnitude of technical challenge and the emphasis on the choice of programming language. It is usually web application developers that spend the time and effort picking what they deem to be a modern, or "more productive" programming language. It is as it should be, because most productive languages, Clojure included, are usually more productive for applications – not complex basic algorithms, or the development of novel data structures. Application developers spend 90% or more of their time writing and debugging production code, so they'd better spend some time picking and learning a language that makes their work easier.

But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten, and not only because these languages are better suited to their task performance-wise, but because they have other, more important things to think about. If your challenge is not translating a business model into code, but solving a real algorithmic or technical problem, then the programming language really doesn't matter, and you're better off sticking with the one(s) you know best.

If 95% of your time is spent thinking anyway, it doesn't matter if your language can reduce the remaining 5% of actual development by a factor of two or more. IBM's Watson would not have come about sooner if its engineers had chosen Haskell over Java.

ADD.: "Productive" programming language are better when you need to solve the same problem over and over (say retrieve data from a database and generate HTML). But when your problem is more or less one-of-a-kind, these productivity boosts either disappear or don't matter. When you're handcrafting a magnificent chair, picking the best assembly line is irrelevant. Chances are it's been designed to solve other problems and retrofitting it to your requirements is extra work, and even if it wasn't, this chair need only be made once. Mass production tools are unnecessary.

[+] plinkplonk|12 years ago|reply
"But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten, and not only because these languages are better suited to their task performance-wise, but because they have other, more important things to think about. If your challenge is not translating a business model into code, but solving a real algorithmic or technical problem, then the programming language really doesn't matter, and you're better off sticking with the one(s) you know best."

This is so true. They often use something like MATLAB or R, but that doesn't affect your essential point, which is a great one and worth thinking about. How much of what you encode is original/cutting edge thought vs routine (CRUD apps) say.

That said, that particular blade cuts both ways. There is nothing (other than inertia) stopping the "one(language) you know best" being something reasonably advanced, with decent, if not ubiquitous libraries and tooling, like Scala or Clojure or OCaml, or even Common Lisp. Why would you want to use an 'inferior' language if you know a better one? So in the end this argument may just devolve to "Most engineers in charge of implementing systems are more comfortable with Java/C++ than Clojure/OCaml".

Something of medium complexity, between the basic CRUD web app and pioneering projects like Watson, like the (Twitter)Storm library for e.g ,is much easier to do in Clojure than in Java, though it uses many java libraries (e.g: Apache Zookeeper) under the hood.

So we have $baselang for basic webapps, $coollang for medium complexity efforts, and back to $baselang for truly cutting edge work, where thinking effort outweighs coding effort. An odd dynamic, if true.

" IBM's Watson would not have come about sooner if its engineers had chosen Haskell over Java."

This is a strong claim :-)

Even IBM Watson might be easier in Haskell,(it actually uses Java, with "significant chunks" in C++ and Prolog [1]) but someone actually needs to build something like this first. FP proponents should probably focus on building existence proofs of cool systems than resort to argument (imho).

[1] http://blog.reddit.com/2011/02/ibm-watson-research-team-answ...

[+] rayiner|12 years ago|reply
I feel precisely the opposite way. I used to work on developing network algorithms. Our simulations used C++. It was phenomenally unproductive, because in addition to the inherent difficulty of the problem, you had to deal with the limitations of the language. Its a pain to debug C++ in a straightforward "move bits around" app. Its even more painful to get distracted by stack smashing bugs when you're trying to figure out the emergent behavior of a complex algorithm.
[+] jbrechtel|12 years ago|reply
Neither my intuition nor experience lead me to the same conclusion as you.

My intuition says that complex algorithms get even more advantage from more expressive languages. Java and C++ are usually not counted among the more expressive languages.

My experience on two* different projects in very complex domains...one where we used Java and another Clojure was that the Clojure code was more approachable and modeled the domain in an easier to understand way.

Just my 2 cents.

* One project was in the field of power systems, modeling and predicting power flow through the distribution end a power network.

* The other project was doing personalized treatment recommendations based on analysis of genetic data.

[+] jkldotio|12 years ago|reply
Yes the Lisp > Blub argument is usually presented with the assumption that the opposing argument is that Blub > Lisp or at least that Blub = Lisp. In fact the argument is actually that Blub provides access to a vast ecosystem of algorithms written by world class programmers, each specialized in their particular domain, and therefore it's actually mathematics and algorithms (via Blub) > Lisp and self-implementation of those algorithms; or you could restate that as the efforts of scores of world class programmers > than my own efforts. Perhaps Clojure settles that gap by linking a Lisp to the JVM ecosystem, but historically that wasn't the case in these arguments.
[+] lawn|12 years ago|reply
Might I add that a good tool, a good programming language, let's you think about other things than implementation detail. It let's you express things in a more direct and concise manner.

Very smart, very forward thinking people might choose Java or C/C++ because they don't want the extra effort of learning/using a new programming language. But it certainly doesn't mean it's the better choice. Lisp was for example developed explicitly for the use in artificial intelligence by John McCarthy, a very forward thinking and smart person who greatly advanced the field of AI.

I would say that when your problem is a one-of-a-kind, you'd rather think about the problem at hand and think of the implementation as little as possible.

[+] wes-exp|12 years ago|reply
"But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten"

This may be a reflection of the fact that people who write code, but are not really programmers, lack sufficient domain knowledge (in the field of programming) to choose anything other than the "default" choice. It's harder to go off the beaten path when one knows little about the surroundings. That doesn't mean the path is optimal though.

[+] melling|12 years ago|reply
I think reducing a million lines of code by a factor of two is a huge win. Reducing it by half and having the extra type checking in Haskell would be even better. Heck, I'd also take the super fast complilation in Go with the equivalent million lines. Bottom line is that when you're dealing with that much code, those little wins do add up.
[+] lrobb|12 years ago|reply
Yes.. It reminds me of a recent interview with matz:

"The ones who do not challenge themselves to create new things are often falling behind - they learn a hip new language today and try a new web framework tomorrow, but still lack the foresight to invent and to improve."

[+] Serow225|12 years ago|reply
My experiences would tend to agree with you; I worked at The MathWorks (makers of MATLAB) for a while, and I was a little surprised when I noticed that the hard-core devs who developed the guts of the language (parsers, JIT, data structures, math libs) worked purely in C++ often using EMACS or VI, and only rarely would launch a dev MATLAB GUI. I suppose it makes sense that if you're working deep in the internals, all that matters is properly handling all the possible inputs that your function can receive and not really how those inputs get there - that's the job of other teams further up the call chain :) I still found it an interesting lesson.
[+] michaelochurch|12 years ago|reply
But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten, and not only because these languages are better suited to their task performance-wise, but because they have other, more important things to think about.

Yes and no. There are a few things to keep in mind.

First, these fields have a lot of finesse for which there's no substitute for years of trial-and-error. What's the right learning rate for a neural network? (There isn't one; real NN implementations have varying strategies for adapting the LR, but even picking the right one of those requires knowledge both of how NNs work, and the data itself, since noisier data means there won't be monotonic learning progress unless one trains over the whole set-- and that's usually impractical.) What this means is that programmers tend to have a longer lifespan and you don't see as much tech churn.

Also, most machine learning programmers use different languages for exploration and production. For the former, Python, R and Matlab seem to be the favorites. For the latter, C and Java are the traditional incumbents, though I could see Scala or Clojure breaking in. Making Clojure performant isn't elegant, but (a) it's not worse than writing Java or C++, and (b) the same is true of Common Lisp. Typically, though, production machine learning doesn't (or shouldn't) have a high LOC-count, especially because most ML projects are there to do things that would sprawl into unmaintainable complexity if done by hand (e.g. speech recognition).

Clojure won't replace C for low-level system programming that requires manual memory management, but even in machine learning, that doesn't apply to most of the work-- remember: half your time will be spend looking for patterns or exploring methods, many of which won't work-- unless you're working at a very large scale. For most of ML, the performance-intensive part is linear algebra, and you usually will end up using wrappers around Lapack (which is in Fortran) for that.

What I would say about the higher end of programming is that engineers care more about "the right tool for the job" and less about "the tool I like best". They're less wedded to one technology stack, so they're willing to use Java if it's the right language for their problem. They're definitely not your typical Java jockeys, though.

[+] hnha|12 years ago|reply
oh come on mods. this was well titled. you removed the mention of lisp, leaving the title as guesswork about the contents. there is no rule for keeping original titles. please don't butcher them when someone actually adjusted them well.
[+] nemesisj|12 years ago|reply
"...and was finally in a position to really influence our development decisions at every level. Even at the most sensitive of them all: language choice"

When people say that "programming language doesn't matter", they mean it doesn't matter within the broader context of the company. And they're right. Language choice is never the key to success for the reason for failure that we in the tech field might wish it was.

Yes, it'll matter to programmers to some extent, but just like the author talks about how quality programmers will be open to new language choices, those same programmers will be open to getting things done, regardless of language.

Fixating on a programming language choice is premature (and worse, even irrelevant) optimisation for a whole host of problems that just don't affect a startup's success that much. Things like finding the best people, getting customers, and scaling will always be the harder, more critical aspects of running a successful company.

[+] momo-reina|12 years ago|reply
Jao's old blog[1] is a treasure trove of information. Modern Lisp hackers like him, Zach Beane and Edi Weitz are great representatives for the Lisp community: incredibly talented and yet very humble and more than willing to help out those just starting out in Lisp.

[1] http://programming-musings.org/

[+] gnuvince|12 years ago|reply
People who are nervous about hiring people in a relatively unpopular language should watch this video[1] at the 1:01.00 mark by Yaron Minsky.

[1] http://www.youtube.com/watch?v=hKcOkWzj0_s

[+] Cyranix|12 years ago|reply
Imprecise transcript:

  Q: Are you convinced that you're more successful
     because of OCaml?
  A: <jokingly> I certainly have job security! But
     seriously, I think the answer is yes, for many
     reasons that are difficult to tease apart. One
     of the big reasons is that it became easier to
     hire great programmers. We hired many great
     programmers before we switched to OCaml (and
     they have since adopted OCaml) but it took more
     effort to find them than it took to find great
     functional programmers. It's hard to describe
     how significant the effect was, but I believe
     it was important; our language choice impacted
     our tools and results in a positive way.
[+] sklivvz1971|12 years ago|reply
Language choice is mostly a reliability, sustainability, maintainability and team-scalability issue.

I agree that people can learn, bugs in the language can be fixed, lack of libraries can be worked around and unknown pattern space can be navigated.

As with any choice this is a trade off, in this case between time and convenience. Those ugly C-like languages have tons of cheap or free libraries, support communities with real, large scale experience and so on. Smaller languages are much more of an adventure - fun with company (your own?) money. It is not the correct choice in many cases.

None of these problems seems to matter in the beginning, when the team is small and awesome, but they do massively matter later on. The author seems to completely avoid this point, and I urge them to consider it.

Finally, having always been a deeply language agnostic developer, I find all these language-religious arguments to be a bit childish and tiresome.

[+] wiml|12 years ago|reply
> Language choice is mostly a reliability, sustainability, maintainability and team-scalability issue

Yes, but aside from that, Mrs Lincoln, how was the play?

[+] educating|12 years ago|reply
> Deep down, the people in charge either didn’t think languages make any difference or, worse, bought into the silly “availability of programmers” argument. I’m still surprised anyone would believe such a thing. If i guy came telling me that he wanted to program only in, say, Java because that’s what he knows best and that he doesn’t really feel prepared or interested in learning and using, say, Clojure (or any other language, really), i wouldn’t hire him in a million years, no matter what language my project were using, and no matter how many thousands of candidates like this one i had at my disposal.

The problem is not willingness to learn, the problem is the risk of going into a job, becoming less fluent in a language for which there are more job opportunities, and then being able to get paid as much or more with the new skills they learned.

I am very interested in learning Clojure, Scheme, Lisp. But, these languages do not provide job security unless I want to move. Companies want .Net (C# or VB), Java, and JavaScript- w/ other various associated frameworks and libraries. So I have little incentive to opt to use Lisp at work, even if given the opportunity.

I was a long time Java developer, but have been using Ruby for the past several years. It is really not as easy to find Ruby jobs locally as it is in Java or .Net, but I think it will be worth it because the # of Ruby jobs is growing. I don't see that for Clojure, though I really do wish it well. For a language to gain in the job market have, it doesn't need to be the best, it has to be fun, easy, and safe enough to use for the ecosystem to grow around it.

C++ was more fun than Fortran/Cobol.

Java was more fun than C++/.Net was more fun than Visual C++.

Groovy, JavaScript, Python and Ruby are more fun than Java.

PhP was more fun than HTML + CGI. Ruby/Rails is more fun than PhP.

Note: when I say "fun" I don't mean it in the "challenging" sense of fun like Haskell. I mean writing great, cool things quickly and understanding it easily.

[+] mgkimsal|12 years ago|reply
You nailed it.

Company X chooses... ocaml, in this case. Someone taking a job there either needs to know it already, or have a 'willingness to learn'. OK - I'm willing to learn it, but at what expense? I dig in to ocaml for a few years, get really good at it, and find, say, 4 years from now, that the market for ocaml developers is really bad. I've gambled and lost, even if it "paid off" for the company.

Most developers are making rather rational choices, regardless of what tech company founders would like to think.

I do run in to people who are ".net or die" and "java for everything", and they'll never ever consider bothering to learn a new tech stack. And... they chose wisely enough that they may not have to over the next 10-15 years - almost anything you'll want or need to do for a large majority of computing/development needs will probably have .net/java equivalents (whether they're the best implementations or not is irrelevant to these people). But... choosing Haskell as a sole focus... they may end up having to uproot a family to move to another location to get another equivalent Haskell job in a few years, which most people don't want to have to do.

[+] derefr|12 years ago|reply
> Give me programmers willing to learn and eager to use Lisp or Haskell (or Erlang, ML, OCaml, Smalltalk, Factor, Scala...) any day, even if we happen to be using C++, for goodness sake! Those are the ones one needs, scanty as they might be.

Does anyone hire with "eager to learn new languages" as a criterion? It'd be interesting to come up with a technical interview that would test for that. Maybe it'd go something like: "Okay, I trust you got the email yesterday with the syntax of the three languages we made up for the purposes of this test--Blub, Cruft, and Wheeze? Good. Now, we want you to code A* in Blub, and then translate this code sample from Cruft to Wheeze. You've got a half-hour."

[+] brianpgordon|12 years ago|reply
> If i guy came telling me that he wanted to program only in, say, Java because that’s what he knows best and that he doesn’t really feel prepared or interested in learning and using, say, Clojure (or any other language, really), i wouldn’t hire him in a million years

Between JSF, JSP, JMS, JPA, JTA, JCA, JMX, JNA, servlets, SecurityManager, AWT/Swing, and countless other frameworks and technologies, Java is so big that it would take an entire career to become an expert. I think it's reasonable for someone who has invested a lot of their career in getting experience in some of these things to want to continue working in Java rather than let those skills go fallow.

[+] microcolonel|12 years ago|reply
I think he's hit the supply nail right on the head.

Programmers which are more costly are more productive(as a general rule), so finding a few highly-productive professionals willing to make the right decisions, is better than hiring a gaggle of newborns who figure public static is the only way to initialize data.

[+] davexunit|12 years ago|reply
Jao is awesome. He is the author of Geiser, an Emacs mode like SLIME for GNU Guile and Racket. Geiser is an invaluable tool when working with Scheme. Best of luck to him.