top | item 1052858

A startup's "Why we use Lisp" story

113 points| zachbeane | 16 years ago |article.gmane.org | reply

90 comments

order
[+] mojuba|16 years ago|reply
Lisp is a beautiful language but I think the biggest problem with it is its proponents failing to explain the merits. I'm sorry, this post would have probably made a bit more sense 15 years ago, but definitely not now.

> (a) Very fast development that is enabled by CL (e.g., everything from hash tables to string-operators to memory management is automatically included - there is nothing that is not included).

Name a modern mainstream language that doesn't have these things.

> (b) Excellent programming environments - e.g., parentheses-savvy editor.

You haven't seen XCode, Delphi or MS Visual Studio, where, for example, you can jump to the definition of a symbol with "one click", allow interactive step-by-step debugging with variable watch, disassembly, stack trace, etc - I shouldn't really name all the things that are possible in a typical modern IDE. And I don't know any text editor which is not paren-savvy.

> (c) Excellent compiler, especially with declarations, enables very fast code.

A compiler which doesn't "enable very fast code" has no place under the sun nowadays.

> (d) Excellent system stability with no random crashes at all.

Very exciting, although GC-based languages (i.e. those usually lacking pointers) should not crash at all, or if they do crash that's a shame. Stability and robustness of your compiler and your runtime system shouldn't really be mentioned as a merit. If it doesn't meet stability standards, it shouldn't be released.

> (e) Macros and all that.

Finally getting to the point and you say "and all that"? Btw, "all that" includes unification of code and data - something no other language provides, let's say, idiomatically. This is an amazing feature, and in fact Lisp macros are Lisp macros thanks to just that - unification of code and data and symbolic nature of the language.

Memory footprint: megabytes do matter because of the CPU cache. A 30,000 LOC program should take a few megabytes at most and fit a modern CPU cache entirely. Compared to a 50MB program the performance gain can be enormous.

[+] jrockway|16 years ago|reply
You haven't seen XCode, Delphi or MS Visual Studio, where, for example, you can jump to the definition of a symbol with "one click"

I have used XCode, Eclipse, and Visual Studio. Emacs' tag functionality is much easier to use than the equivalent functionality in those editors. ("C-u M-." and "M-star" are especially convenient. And I would have typed an asterisk there, but news' parser is busted.)

The parentheses awareness he is talking about is paredit-mode, which changes Emacs' usual keybindings from operations on lines to operations on S-expressions instead. I never got the hang of it, but Emacs' built-in word/sentence/paragraph movement/editing functionality is essential. I have not seen these features in other editors, including Vim.

The other feature that Emacs has over the "visual" editors that you mention is the ability to interactively change how the editor works. (This is especially nice if you're a lisp programmer, as there is nothing to be confused about; it's just Lisp.) I am not just talking about changing the colors or things like that, I am talking about writing new modes, changing the built-in behavior, and so on. I do this so often I don't even consider it unnatural.

(Something I fixed today; when loading a ".chs" file into the inferior GHCI buffer via C-c C-l, GHCI complains "this is not haskell", which is true. Then it changes the prompt from "My.Module>" to ">", which makes the next C-c C-l command lock Emacs. I debugged this by seeing which command C-c C-l ran by pressing "C-h k C-c C-l". I then pressed ENT to visit the source code for that function, and noticed that it is programmed to hang until it gets output from GHCI that matches a regex. I changed the regex to something more liberal, hit C-M-x to load my changed function, and my problem was solved. Those two minutes of distraction will save me much frustration over the course of the rest of my life. When was the last time you fixed a bug in Eclipse or Visual Studio in two minutes?)

But anyway, Emacs' functionality is not limited to Lisp. I mostly write Perl and Haskell, and Emacs excels at both tasks. It is also a good mail reader, web browser, and IRC client. (Why yes, I am composing this in Emacs.)

allow interactive step-by-step debugging with variable watch, disassembly, stack trace, etc - I shouldn't really name all the things that are possible in a typical modern IDE

But of course, no REPL. I still don't know what a C++ programmer does when he wants to figure out what regexp he needs to match something. Write a driver program around the library, compile it, debug it, recompile it, and finally run it and play with it? Sounds fun...

A compiler which doesn't "enable very fast code" has no place under the sun nowadays.

Apparently the real world is not under the sun.

Memory footprint: megabytes do matter because of the CPU cache. A 30,000 LOC program should take a few megabytes at most and fit a modern CPU cache entirely. Compared to a 50MB program the performance gain can be enormous.

Sure, but you have to realize that most applications that people build with big stacks like CL or Java or C# or whatever are not really CPU-constrained; the context switches kill them. (Wait for user to do something. Wait for database results. Wait to copy response buffer to web server.)

Nobody wants code that runs slowly, but the trade-offs to get something really fast (or to fit in the CPU cache, etc.) are not worth the effort for most people. "80% solution" and all that. And, the CPU cache is all about critical sections, anyway; it's usually your data that doesn't fit in cache and slows everything down.

So anyway, you are underestimating the capabilities of Emacs.

[+] lispm|16 years ago|reply
a) Java is such an example. It does not offer fast development. Contrast it to so-called RAD (Rapid development) tools or rapid prototyping environments, ... Java is a mostly a batch language with a long edit/compile cycle.

b) Right. There are several of those.

c) For dynamic languages there are fewer excellent compilers. If you check the usual benchmarks, Lisp compilers are among the best in that category.

d) That's true. But the reality is different. GCs have bugs. More complex GCs have even more bugs. The FFI may introduce possibilities for errors. I have seen Java guys struggle with obscure heisenbugs. In theory all is mature, in practice it took quite some time to get Java mature and if you read the SUN notes for some of their garbage collectors (the concurrent one, for example), they tell you that they are not stable enough for production. For example the release notes at some point said: "Garbage First (G1) garbage collector Improved reliability and performance Available as 'early access' software. Usage of G1 in production settings without a Java SE for Business support contract is not recommended."

e) The 50 MB program usually will not touch all the code all the time - for example LispWorks can include the function compiler in the delivered application - it is likely not called very often. The 50 MB can also consist of a lot of Lisp data, not only code.

[+] philwelch|16 years ago|reply
"You haven't seen XCode, Delphi or MS Visual Studio, where, for example, you can jump to the definition of a symbol with "one click""

That's an old feature. It used to be called "ctags" and even console-based text editors support it.

"Finally getting to the point and you say "and all that"? Btw, "all that" includes unification of code and data - something no other language provides, let's say, idiomatically. This is an amazing feature, and in fact Lisp macros are Lisp macros thanks to just that - unification of code and data and symbolic nature of the language."

Lisp's problem seems to be that, until you know how to use macros and code/data unification, you can't be easily convinced of their merits. It takes a considerable commitment to learn Lisp before you can reach that level, though.

[+] gchpaco|16 years ago|reply
a) C and C++. Oh, but they're not modern, you say; balls, I say. Both standards were revised very recently and C++ is changing quite a bit.

b) SLIME and SBCL (to pick freebies) crush any of those under their collective boot. Eclipse tries real hard but is let down in a number of ways by Java (instability, the fact that the environment is opaque to the user instead of malleable, etc). LispWorks's GUI is very good, at least as good as Eclipse at the things Eclipse is good at.

c) You would be shocked at how bad things are. GCC, to pick on a weak and somewhat helpless example, tries really hard to optimize C code but is generally content to merely compile C++ code. XCode with its gcc backend does relatively little real optimization of Objective C, certainly nothing compared to what you expect from a modern JIT in Java. Not familiar enough with Delphi or MSVS to pick those apart but I doubt it's that much different.

d) Ditto. Eclipse crashes at least once a day for me during heavy use and suffers bizarre and inexplicable errors even more frequently (fun experiment: create a file Foo.java. Rename it to Bar.java. Try to create Foo.java again. The editor cannot be convinced to read the new Foo.java off disk without a restart). Errors in the native code are not infrequent (I have a bunch of logs here from older Javas where the GC (I think) got confused in the middle of GCing and dumped core), although they seem to be better with Java 6.

e) A comp.lang.lisp audience knows about macros and why they're nice. Other language zealots cannot have their minds made up by a simple reiteration of the benefits of code as data (which I agree are considerable). The other points are somewhat more relevant inasmuch as they challenge generally accepted but really wrong concepts.

And re: memory footprint: be wary of comparing apples to rutabegas. Remember that every C program benefits from libc and its ancillary stuff; on this here Ubuntu system the system libraries required to bring the system up from a dead stop (not any of this /usr/lib detritus) is already 2.5 MB on disk. Also remember that the 50 MB program footprint likely includes the runtime compiler system (so you can hot swap code; very useful) which is something like 42 MB for gcc 4.4 alone (again, disk; memory usage is enormous). I would expect that the amount of code that needs to stay in the CPU cache is equivalent.

[+] malkia|16 years ago|reply
LW (I have LW6) is really great system, for coding, experimenting, and the CAPI user interface is platform-independent (almost, there are some differences, but you can handle them).

I do use MSVC all the time at work, XCode occasionally at home, and have used Delphi quite a lot before my current work.

But I also bought Lispworks Mac/Win32 versions, and I feel much more productive there. Why? Because I have listener, I can quickly recompile the current buffer (Ctrl+Shift+B), or the current enclosing function (Ctrl+Shift+C), or if I'm at the end of expression evaluate it (Alt+X E).

I can keep recompiling the buffer, and all variables declared with defvar would retain their values, I can change object definitions on the fly.

It's just easier to experiment.

Weaknesses - I'm so much used to the MSVC debugging (I Actually used to be Turbo Debugger junkie in the past), that I miss that - it's simply not easy to debug in LW (or EMACS + slime + any other lisp).

But do I need to debug that much more? No - I don't, but that could be because I'm yet at the amateur level.

[+] rosssd|16 years ago|reply
To be fair, this is on a Lispworks mailing list discussing Lispworks in particular, which has it's own IDE that provides all of these features.

'Go To Source', 'Edit Definitions', 'Edit Callers', 'Edit Callees', 'Display Backtrace' are all a keystroke (or extended command call way) along with various tools that may or may not be present in others (I've never used MS Visual Studio, which seems to be a leader in this field)

One of the parts I miss from it was that break points are at a source level, not a line level, which allowed incredibly fine control when tracking down bugs.

[+] zck|16 years ago|reply
>(d) Excellent system stability with no random crashes at all.

This is interesting, considering one of the main reasons Reddit switched from Lisp to Python was because it was crashing often.

[+] lispm|16 years ago|reply
that was another Lisp implementation
[+] skilldrick|16 years ago|reply
I don't understand why a Lisp hacker wouldn't match parens properly when ending a parenthetical statement with a smiley:

  (commentless, of course :)
Hasn't he seen xkcd: http://xkcd.com/541/ ?
[+] mbrubeck|16 years ago|reply
If you do it the double-chinned XKCD way, it messes up your auto-balancing text editor.
[+] zandorg|16 years ago|reply
A friend of mine keeps asking me to dump Lisp and "Get modern" with C#, and I try to explain why I prefer Lisp, but he won't accept it.

It was Paul Graham's essay that encouraged me to try Lisp in 2005.

[+] gridspy|16 years ago|reply
I'm just dreading the day that Microsoft decides that C# isn't modern enough and they want to sell a whole bunch of seats of Visual Studio (insert new name here)
[+] vii|16 years ago|reply
Seems to me this discussion is missing one of the main points of the original post: a massive plug for an unfairly under-appreciated book:

  > "Let Over Lambda" 
  > (which is really quite scary to read - I can't say that I understand 
  > 100% of it - maybe 60% and I am very happy with that level of 
  > comprehension) -- you end up with an enormously powerful set of 
  > programming tools unlike anything else out there.
I really like this book too and recommend it.

http://letoverlambda.com/index.cl/toc

[+] idlewords|16 years ago|reply
"Excellent system stability with no random crashes at all"

This holds for pretty much any language you care to use.

[+] aharrison|16 years ago|reply
People keep saying that, and I can understand where they are coming from, but this isn't the case. Just yesterday, I managed to segfault python 2.6.something. I have no idea why (I really should have figured it out) but my hypothesis is that I freaked out the parser. It was vanilla python code; it should never segfault. I have had the same experience with the JVM: I once segfaulted the parser by incorporating a static string that was hundreds of lines long. Curiously, Eclipse "compiled" it just fine.

Sometimes environments have bugs. If you need an underlying runtime that simply will not crash, using something tried and true may be at the top of your priority list. That might be overkill for most projects, but I can see a guy who wants something so robust he knows its his fault when it breaks.

[+] motters|16 years ago|reply
Apparently iRobot also uses Lisp in an embedded context.
[+] gaius|16 years ago|reply
This is the key point:

I started programming in LISP way back in 1971 on a Univac 1108 mainframe and also implemented a 68000-based Lisp system (~50K lines of real-time assembly) for mobile-robotics use in 1983 - and so know my way around the language.

All Lisp (or Smalltalk, or...) success stories I've read hinge on someone with an enormous amount of experience with the language. I'd argue that someone with that much experience could get the job done in (almost) any language. I'm surprised that someone with that much experience would put it down to language choice rather than deep knowledge of the problem domain.

[+] olavk|16 years ago|reply
That is close to saying that all languages are equivalent given the same amount of experience. I don't buy that. For a specific task some languages may be better suited / more productive than other languages, even given equivalent levels of experience.

I don't believe in a global linear scale of language power ("the blub theory"), but I do believe that some language may be better than others for a specific task given specific constraints.

E.g. if I have equivalent levels of experience in C++ and Python, I'm pretty sure I can write small webapp quicker in Python. OTOH if the languages are very similar, like Python and Ruby, the level of experience is much more important that the relative strengths and weaknesses of the languages.

Of course, it is seldom that you get that kind of fair comparison between two languages - usually everyone has a favorite language they know better than any other.

[+] dkersten|16 years ago|reply
Heres why I like Lisp:

I recently (~5 months ago) started using Clojure heavily and now find myself much more productive using it than using any of the languages that I have considerably more experience with (I used C++ almost exclusively for 4 years for hobby (mostly game related) projects and toy virtual machines/interpreters; Python for ~4 years for prototyping, web development and GUI development; Java for 2.5 years for server (non-web) development - total programming time including hobby, uni and professional = ~10 years).

As an example, I'm very much interested in compilers, virtual machines, programming language design and such, so have been plying around with these in various languages over the years. I wrote my first toy language in VB; wrote a few interpreters, virtual machines and simple parsers in C++ and some parsers, interpreters and assembly code generators in Python; for uni, I implemented a parser and code generator (instruction selection, using maximal munch) in JavaCC - recently I wrote an assembly code generator in Clojure. It took me a weekend and it surpassed the power of anything[1] I'd written in VB, C++, Python or Java. Its flexible and can be easily be extended, its pretty smart and can do some basic optimisation (caching data in registers, function inlining...). I tried writing something similar in Python before, but gave up.

My point is that I have much much deeper knowledge of C++, Python and Java, yet I was able to build something MORE complex and extensible in Clojure, in less time, even though I'd only been using it for a few months.

Yes, my previous experience played some part in this, but I attribute most of it to the flexibility and power Lisp provides me through easy to use and powerful abstractions, flexible and convenient syntax and interactive development.

[1] Of the same scale - any interpreter, code generator, compiler etc. I've written much more powerful systems, of course, but they are huge in comparison and took a lot lot longer than a weekend to write.

[+] lispm|16 years ago|reply
I would say that he has an enormous amount of experience with several languages. He lists on his webpage: "Programming languages used over the past 39-years include C/C++, Common Lisp/CLOS/MOP (and other Lisp varients), ALGOL, FORTRAN, PL/1, SNOBOL, numerous assembly languages, and others."

So, he probably could have written it in C++ (or SNOBOL ;-) ), but he chose to do it in Lisp.

[+] anonjon|16 years ago|reply
All things being equal, lisp was in use much more back in the 70s and 80s than it was in the 90s or 00s. It makes sense for the majority of people who would think to use lisp to be a bit grayer.