top | item 3534746

Why Lua?

232 points| tlowrimore | 14 years ago |blog.datamules.com | reply

138 comments

order
[+] alanfalcon|14 years ago|reply
I took some Visual Basic programming courses in high school, some PASCAL courses in college, and then didn't program anything other than basic HTML/CSS for the next eight years. Last year I programmed an iPhone game in Lua (using the Corona SDK) and it was easy as Tau. Sure a better programmer would have finished the programming aspects in a quarter the time it took me, but I started by looking at Objective-C and I'm not sure I ever would have picked it up without some kind of formal training. Contrast with Corona/Lua, where I watched a few eight minute videos on YouTube and could turn around and make simple but complete programs, and more importantly completely understand what was going on within them. Reading the Lua documentation was a revelation, reading Apple's was a headache.

Why Lua? http://itunes.apple.com/us/app/beat-boxes/id460798042

[+] haberman|14 years ago|reply
The more I learn about Lua's design and implementation, the more impressed I am. It's very rare to see software that does so much with so little code. The design is extremely clean and the code is extremely fast.

The C API is easy to use and gives good performance, and yet encapsulates enough of the VM's implementation that C modules are source and binary compatible with both Lua and LuaJIT, two very different implementations of Lua. Note that the C API was designed before LuaJIT was even written, so this API was designed without the benefit of seeing how different implementations of Lua might vary.

The source is very readable for the most part; my only stumbling block is that their naming of types and variables is extremely terse at the cost of readability sometimes, and the extensive use of macros can sometimes can require a lot of jumping around. But Lua is an impressive achievement that I frequently refer to for design inspiration.

[+] sshumaker|14 years ago|reply
Lua has great C integration, but LuaJIT's is even better. You can natively interface with C code without a recompile, just by defining the C function prototype in Lua, and using it like you would any normal lua function. Better still, LuaJIT can frequently inline the calls to C code, so there's no cross-language marshaling penalty.

Check out http://luajit.org/ext_ffi_tutorial.html

[+] chalst|14 years ago|reply
Norman Ramsey's answer to an SO qn of mine is interesting

    http://stackoverflow.com/a/2102399/222815
Roberto provided a ref to the way Lua implements closures, which is certainly innovative.

One things I like about Lua is that it's TCB is small. This matters when you are embedding code.

[+] chubot|14 years ago|reply
Why isn't Lua more widely used? One reason is a consequence of it being an embedded language. Lua has had 5 major versions which are very incompatible with each other. You're just supposed to stick with the previous version until you upgrade your code.

I read all the Lua papers, and they are quite fond of "getting it right" (which is refreshing). They will break things to get it right, whereas other languages stick with all their old design warts in the name of compatibility. I like this approach, but it comes at the expense of popularity.

Compare Lua with Python, which has had basically 1 major version for 20 years Python 1.x are all compatible with 2.x -- that was a license change; and Python 3 is having adoption troubles which proves the same point. Python 3 was the chance to "get things right", or fix old design warts, but you can see what a big impact it's having on adoption, fragmentation of libraries, etc.

The other reason is that embedding a language isn't as common a use case as a scripting language. I've tried to write stuff in Lua rather than Python, but really Python's "batteries included" philosophy just kills any advantage Lua has.

The main reason I would like Lua is so you can distribute stable (small) executables... but when you're starting out Python is a lot easier. You don't want to download LuaSocket or whatever and mess with the build system.

[+] groovy2shoes|14 years ago|reply
Good article, but one nitpick: not everything in Lua is a table. Tables are a versatile data structure that can be used as arrays, dictionaries, objects, etc. However, Lua provides many types besides tables: numbers, booleans, closures (functions), strings (which are interned), coroutines, and something called userdata (which are typically pointers to something implemented in C).

Another cool thing about Lua, which was mentioned only briefly in the article, is proper tail-call optimization (TCO) like you'd find in Scheme. TCO makes expressing recursive algorithms nicer because you don't have to worry about blowing your stack.

Lua's design philosophy -- giving the programmer a handful of basic yet powerful features -- makes it somewhat Schemy. I suspect that Lua is about as close as you can get to a ball of mud before becoming Lisp.

[+] jballanc|14 years ago|reply
To nitpick your nitpick: in Lua tables are the only composite data structure. Everything you listed is either atomic or opaque (you can hide composite datatypes in userdata, but you can't introspect the values).

If you think about it, this is very similar to C where the only composite datatype is a struct (arrays are just sugar on pointer arithmetic). In fact, I think if you wanted to make a scriptable dialect of C, you'd create Lua.

Of course, by being simple, add in dynamic typing and first-class functions, and Lua does feel a bit like Scheme's kid brother. Or, rather, Lua (in my mind) reveals how C and Scheme aren't so far apart after all!

[+] silentbicycle|14 years ago|reply
Lua has a lot of depth to explore (TCO, coroutines, prototypes, ...), but the language has been carefully designed to only really be like JSON, if that's all you need. The advanced parts stay out of the way.
[+] ktr|14 years ago|reply
Good point. I was thinking (as jballanc sort of mentioned) of this with regards to composite types. E.g., there is no difference between lists versus hashes versus tuples versus objects, etc. But duly noted for any revised edition :)
[+] gregholmberg|14 years ago|reply
Speed& Simplicity

LuaJIT speeds can rival code written in C. And if you are still not getting the speed you want, see my 1st point above (Integration with C and C++ for that matter).

The simplicity of Lua is, I think, really underrated.

It is a good learning exercise to write some small benchmark / utility / tool in Lua so you can get an idea of how quickly you can develop, and how fast the resulting code will be.

Then read through the Lua source [0] to see just how little code is required to give you that platform for your ideas.

[0] http://www.lua.org/source/5.1/lua.c.html

[+] whatajoke|14 years ago|reply
Not criicizing lua, but worth comparing to guile.

> Integration with C (and C++ for that matter)

Guile does it better. You can use shared memory threads in guile without any penalty. Atmost you have to allow for the garbage collector to run when inside FFI functions. But that is a small price to pay in case you need to use multiple parallel-concurrent threads with a single heap.

Guile was built with FFI in mind and has an impressive history. Just take a look at guile gnome bindings.

> Speed and Simplicity

Guile 2 is extremely fast. Not as fast as LuaJIT, but it no reason it won't get there. As for simplicity, take a look at the partial evaluator in trunk of guile 2.

> Education

Guile is good old scheme.

> Functional

Can't get more functional than scheme :)

> Everything is a Table

Well, almost everything is a pair in guile. Vectors and hash-tables are trivially available. Though I recommend to sticking to functional programming in scheme style.

> Consistent

As before, can't get more consistent than scheme.

> Portable

Guile is available on n900. So there.

To continue, guile has continuations (delimited or otherwise), and macros (hygienic or otherwise), both of which are effectively missing in lua.

And guile offers all of this while supporting native threading with a single heap. Sweeet.

[+] sshumaker|14 years ago|reply
Not sure if I'd agree that Guile's FFI bindings are better than LuaJIT's FFI. It's certainly easier to declare a function in LuaJIT - you can take the definition directly from the header file, rather than transcoding it into s-exps.

Compare:

  (define memcpy
	       (let ((this (dynamic-link)))
	         (pointer->procedure '*
	                             (dynamic-func "memcpy" this)
	                             (list '* '* size_t))))
vs.

  ffi.cdef [[
	void * memcpy ( void * destination, const void * source, size_t num );
  ]]
You also get the ability to add metamethods and finalizers to returned C objects (so you can use Lua's built-in GC to clean up after, for example, your FILE*).

As for the speed, there's no reason that a lot of dynamic languages couldn't be as fast as LuaJIT. But none of them are even close. I wish LuaJIT was still up in the computer language shootout. The LuaJIT interpreter (with JIT turned off) is 3x faster than V8 on the PC, and faster than Java on Android. And that's the interpreter - the JITed code is way faster.

Of course, macros are awesome, and a huge point in Guile's favor. On the plus side, Lua is very easy to understand, especially if you're coming from Javascript.

[+] dustingetz|14 years ago|reply
> Can't get more functional than scheme :)

Rich Hickey: "Functional has been applied, you know, for a whole spectrum of things. I think there’s the haskell meaning of functional, which is sort of, the pure notion of function applied to absolutely everything in the language. And then there’s another notion of functional... lisps claimed to be functional early on, only for the simple reason that functions were first class objects or values that you can pass around. Those are the two ends. Somewhere in the middle, i think, is a style of programming that emphasizes programming with pure functions, in other words functions that are free of side effects, that operate only upon their arguments, which are values, and that only produce values as results. So that’s a pure function. Clojure emphasizes the style of programming using pure functions, where lisps, traditionally, have not. You could do it, but it was a choice. The key thing in forcing you to make that choice, is to have the core data structures be immutable, so they now can be treated as values, in a way that mutable things cannot. So Clojure emphasizes functional programming, by providing a set of immutable core data structures, including those ones that you use to represent code. They’re also persistent, which is a special characteristic we could talk about. Because of that, it encourages the functional style of programming, because you cannot change [the core data structures], so you have to write a function that produces a new value as a result. but there’s no enforcement. if you write a function and that does IO, well that’s not a pure function anymore, and I don’t preclude it, and there’s no type system to preclude it, its a dynamically typed language, which just respects the programmer’s right to do whatever they please." [1]

[1] https://docs.google.com/document/pub?id=1Vgbw3hGCye2rtmnZ6iS...

[+] haberman|14 years ago|reply
> And guile offers all of this while supporting native threading with a single heap.

What is Guile's synchronization model for shared-state multithreading?

> Guile is available on n900. So there.

This appears to be a Linux-based phone, so that's not very surprising or impressive. Lua is straight ANSI C and can compile on much more limited systems than a Linux-based smartphone.

One other data point: Guile is 5x the size of Lua, in both source and binary forms.

> To continue, guile has continuations (delimited or otherwise), and macros (hygienic or otherwise), both of which are effectively missing in lua.

One of the authors of Lua made an interesting point in his slides describing Lua 5.2: if you think "goto" is evil, continuations are much worse. And yet it's considered "cool" to support continuations. http://www.inf.puc-rio.br/~roberto/talks/novelties-5.2.pdf

[+] marshray|14 years ago|reply
To echo chubot: Why isn't Lua more widely used?

* It lacks an easy-to-use symbolic debugger.

* It lacks a first-rate IDE.

* It lacks a standard way for people coming from OO/Java to define objects and interfaces.

* It lacks a GUI toolkit.

* It has a great set of manuals. It lacks an O'Reilly book with a woodcut animal on the cover.

* Arrays indexes start at 1.

Except for the last item, these are all relatively small things that are simply waiting for someone to come along and do them. This is a testament to the great design of the language.

And many of these things do exist, it's just the Lua style seems to be to understate the achievements to the point that folks can't tell the big new stuff from the abandoned projects.

[+] fab13n|14 years ago|reply
> * It lacks an easy-to-use symbolic debugger.

> * It lacks a first-rate IDE.

<plug>This is changing http://eclipse.org/koneki/ldt/</plug>;

> * Array indexes start at 1.

This sounds dangerously similar to "I can't use Python because it has significant whitespaces". Developers might like or dislike it, but if you can't get over _that_, you either lack important cognitive abilities, or you're suffering from a very serious case of Asperger's/OCD. I can't imagine that someone seriously hampered by _that_ could ever digest any large API.

However, I'll admit it's a bit irritating when you're frantically switching between C and Lua.

> the Lua style seems to be to understate the achievements to the point that folks can't tell the big new stuff from the abandoned projects.

You're right, organizing the efforts of the Lua "community" would probably be trickier than herding cats, and the Lua core team doesn't seem very interested in helping that happen. I sometimes suspect that even Lispers form a tighter community than Lua hackers...

[+] aeturnum|14 years ago|reply
I also think that Lua has a perception of being an embedded-only language (which it's very good at). The biggest code bases are World of Warcraft interface modifications and other plugin architectures. The features Lua would need to be really competitive as a stand-alone language (ex: good library management[1]) are still developing. I think it has the potential to be very powerful, though.

If you want to have some fun with Lua, check out LÖVE[2].

[1] http://lua-users.org/wiki/LuaModuleFunctionCritiqued [2] https://love2d.org/

[+] corysama|14 years ago|reply
Additionally, it is explicitly a batteries-not-included system. There are lots of libraries for Lua all over the net, but many people stop at "Python's standard library has hundreds of modules including the four I want. Lua's doesn't have the four I want."

Lua's goal is to be no-fat and customizable to only what you need for your unique situation. It's even common practice for systems to cut out stuff like the standard library file I/O module if it's not appropriate for your situation.

[+] sshumaker|14 years ago|reply
Lua's original focus as an embedded language has been a huge handicap here. There's almost no standardization, because each developer builds their own set of libraries based on their needs. It's sort of what server-side javascript looked like before node.js.

Lua is in dire need of an opinionated killer platform. Unfortunately, unfamiliarity with the language (vis-a-vis javascript) means that the road to mass adoption is a tougher one.

[+] anonymoushn|14 years ago|reply
It lacks a standard way for people coming from OO/Java to define objects and interfaces.

If you ask in IRC or google around you can find this pretty easily. The first result on google for query "lua class" is http://lua-users.org/wiki/SimpleLuaClasses.

[+] esrauch|14 years ago|reply
> It lacks a first-rate IDE

That is an interesting claim. What are the first rate IDEs for Javascript, Python, Ruby or C++? That isn't a rhetorical question I mean vim, emacs, textmate all seem equally good for Javascript and Lua. I know people use Eclipse for Javascript but it hardly seems like a killer integration.

[+] ammmir|14 years ago|reply
lua (the language and core libraries) is stable and small. many developers are attracted to node.js or rails due to the community ("look, everyone else is doing it!"), ecosystem (abundance of modules/gems to choose from), and rapid release cycles.

lua seems to have more of a relaxed, niche community among game developers and scripting language embedders.

i've been using lua for about a week so far, for a music player i'm building (http://cloudplay.fm/) and i'm writing the song search/ranking system in lua. it's wonderful to be able to prototype it outside the application and not have to deal with the integration until you need to.

i would consider using lua on the server-side (see http://luvit.io/ for a node.js-style interpretation, although i'd prefer coroutines instead of callbacks) but there needs to be more work on the library front.

to help lua grow, i'd fix these things:

- add unicode support

- bring the CLI up to par with node.js (i use node as a calculator, too)

- build a modern distribution that includes frequently used luarocks

[+] jacques_chester|14 years ago|reply
> add unicode support

This won't happen soon. Lua targets ANSI C and only ANSI C.

C11 includes new character types to support unicode, but I imagine that the Lua implementers will take their time.

Plus there's implications for embedded situations.

If you need unicode (and most do), there are libraries and patches that do a reasonable job.

[+] justincormack|14 years ago|reply
There are several server side Lua options with coroutines.
[+] leafo|14 years ago|reply
Lua is a great platform, but I think there are a lot of areas where it can be made more programmer friendly. That's why I wrote http://moonscript.org/
[+] rlander|14 years ago|reply
I was about to mention how cool Moonscript is, but you beat me to it. =) Thanks for the awesome project!
[+] nilved|14 years ago|reply
MoonScript is just fantastic. List comprehensions and OOP are very welcome additions to Lua.
[+] evanlong|14 years ago|reply
THE FOUNDER OF APEDICK.COM. YOU ARE MY HERO!!!
[+] zvrba|14 years ago|reply
Yet another language with significant whitespace and otherwise python-ish syntax. No thanks. I would have used Python if I wanted to. Plus, any source->source translator makes debugging a pain.
[+] hythloday|14 years ago|reply
I love Lua, but when I read statements like this:

>>> In Python, you might __import__('...')' some module and then the variables of the return value would be accessed like vars(module)

it makes me wonder if the author is just unfamiliar with Python (and by extension, any language other than Lua) or if they're deliberately misrepresenting other languages to make Lua look good (which it definitely doesn't need).

[+] ktr|14 years ago|reply
Sorry, I would actually consider myself quite proficient in Python. What I was trying to [unsuccessfully] show was that the underlying mechanics of Python's import mechanism are different then, say, Python's dict or list implementations. Whereas in Lua everything really is just a table. I ran into this (in Python) when I was trying to dynamically import code based on command line arguments and access the functions in said script as strings. So in Lua, you just `x = require"script" ; x["fn"]` whereas in Python you sort of need to know this information up front unless you go through the machinations I outlined in the article.

But this was obviously a poorly written part of the article that could use improvement.

[EDIT] typos [EDIT2] less arrogance :)

[+] timmaxw|14 years ago|reply
I've used Lua for a couple of personal projects. My main objection to it is that accessing an undefined variable or member returns nil rather than throwing an exception. If you make a typo, you don't find out until you try to call or perform arithmetic on your nil value. Since inserting nil into a table just deletes that key of the table, the use site might be several steps away from the typo.

Other than that, I agree with the other posters here. It's an impressively lightweight and elegant language. It's especially good for embedding: its C integration is next to none, it's easy to sandbox if you want to run untrusted code, and the interpreter doesn't use any global variables.

[+] sshumaker|14 years ago|reply
You can fix that easily with a metatable.

  function nilguard(tbl)
    tbl = tbl or {}
    local mt = {
      __index = function(t,k)
        error("Invalid key: " .. tostring(tbl) .. "[" .. k .. "]")
      end
    }
    setmetatable(tbl, mt)
    return tbl
  end

  local myObj = nilguard({foo=10})
  myObj.crap -- should raise
For undefined global variables, just require("strict")
[+] silentbicycle|14 years ago|reply
You can set __index on the global table's metatable. Basically: "run this hook whenever I try to access an undefined field".

    setmetatable(_G, {__index=function(global_env, name) print("unknown: " .. name) end})
[+] ww520|14 years ago|reply
Does anyone know how good is Lua's support for async IO? Especially the handling of large amount of connections and the memory footprint for each connection?
[+] silentbicycle|14 years ago|reply
Luasocket does async IO. It uses poll or select on Unix (calling it 'select'); I'm not familiar with what it uses on Windows. It's not "web scale", but is very easy to use, and works well enough for a couple hundred simultaneous connections.

Lua's convenient C API means that wrapping libev, libuv, or libevent is really not hard, if you want to go that route. (I have a libev wrapper on github, FWIW.)

The thing to keep in mind is that if you're writing high-throughput servers, you probably want to avoid parsing the network IO via Lua. While Lua strings are fine with arbitrary binary data (i.e., \0s are fine), it interns all strings; you'll have the overhead of converting every read into an atom/symbol, and it will spend a lot of time garbage collecting. (Lua makes this daring trade-off because it's usually a net win, and when it isn't, you still have the option of doing things in C instead.) Lua's great for handling all the control logic, though. Coroutines are particularly applicable there.

In a nutshell: If you're good with C, and understand the issues involved with performant async IO, Lua can work very well.

[+] marshray|14 years ago|reply
I'll spare you the exact results of my recent web search, but my impression is that it seems to be a rapidly developing area. There are definitely some good libraries out there of varying scope and maturity.

Lua seems to be such a good fit for this paradigm that I would bet the community is going to produce some outstanding frameworks. (Was kicking around the idea of implementing something myself.)

[+] simon|14 years ago|reply
The article seemed a little short for me, but then I am actively trying to select between Lua and TCL for some personal scripting projects. There are many fine features with each language and few downsides, so the selection process is hard. Of course, that's a nice problem to have.

TCL is ahead by a nose at this point with Unicode support baked in (vs. using a library) and file system handling built in (again vs. using a library).

[+] groovy2shoes|14 years ago|reply
Tcl is an interesting little language as well, and the niche it was created for is similar to the one Lua was created for. The biggest advantages that Lua has over Tcl, off the top of my head, are speed and lambdas. Tcl's semantics make it very hard to optimize because it is stringly typed (with some bytecode and value specialization in the background, but with conversions between strings and other values as needed).

Tcl's file system interface is definitely an advantage if you're trying to avoid third party libraries. However, depending on what you need to do with Unicode strings, Lua's strings are "8-bit clean" and have no difficulty storing Unicode characters; Lua's string functions (the pattern-matching functions in particular) aren't Unicode aware.

[+] mhd|14 years ago|reply
If you're considering TCL, the Jim implementation might also be worth a look. It has most of the features of the core language, some extensions to enable even more and a very small footprint. If I recall correctly, it's even faster for some things and does lambdas etc. a bit better.

The bad thing about Tcl is probably its library support. It has an extension mechanism, but for a lot of "contemporary" projects (web etc.), you're actually more likely to find working Lua "Rocks" than tcl modules.

jim: http://jim.tcl.tk/

[+] antirez|14 years ago|reply
I suggest you to use Tcl, and I think I'm not biased since for instance the Redis unit test is written in Tcl but I'm using Lua as scripting language for Redis.

But as a language, Tcl will allow you to experiment a completely different way of thinking. Lua is neat but not so different from what you already know.

[+] peterbotond|14 years ago|reply
not often mentioned: tcl's slave interpreter, and command redirections from a slave interpreter to its master. a master interpreter can hold many slaves interps, and any slave can be a safe interpreter not execing anything that might be dangerous. redirecting, by renaming, a slave/safe interps commands the master will know. tcl's threading model (if needed): sending messages, one thread per interpreter, and one thread can have multiple interpreters. tcl-8.6 has a killer object model. tcl's vfs, and starkit/metakit makes distribution simpler.

what i do not use tcl for is: web. unfortunate lack of a framework.

it has good support for downloading stuff, ftp, http, bignums and algorithms if needed.

[+] bitcracker|14 years ago|reply
I was first disappointed ... then amazed!

Disapointed because the author's link to the benchmark website lead me to the wrong impression that Lua would be almost 30 times slower than Java 7. Then I googled around and discovered the awesome LuaJit which seems to be able to compete even with C++ in performance.

What I really like is Lua's code density (see samples in the Shootout's benchmarks). Very impressive!

http://shootout.alioth.debian.org/u32/benchmark.php?test=all...

As an old LISPer and Schemer I would like to know if Lua Macros are really as expressive and powerful as Lisp macros. Some people claim this but I am not convinced (I would like to be convinced). The expressive power may be theoretically equivalent but this is also true for C and Assembler :-) The question is: Are Lua macros as easy to handle as Lisp and Scheme macros?

http://stackoverflow.com/questions/323346/what-can-lisp-do-t...

[+] pheon|14 years ago|reply
The reason its not so popular? IMHO because we live in a society that favors 1sec landing pages, customized per individual options, complexity that rivals air skyscrapers and shrinked wrapped ready-to-go solutions. This is not LUA.

LUA is just one tool in the tool box - its not the tool box.

[+] Hexx|14 years ago|reply
On the integration front, one thing I will also add is that in .NET integrating Lua with your program feels pretty close to being a first-class citizen. Very nice.
[+] meric|14 years ago|reply
In lua, the metatable is the killer feature that python doesn't have.