This is a repost of a rather old blog entry. John Myles White has subsequently become one of Julia's most prolific core developers, having created and contributed a variety of very high quality stats and machine learning packages for (and in) Julia:
The moral of the story might be that John puts his money where is mouth is. He thought Julia was awesome and now he's done a huge amount to make it even more so.
Actually being fast is not the point; naive fib is used because it's a very well known algorithm, easy to implement consistently across multiple languages, that heavily exercises both function calls and numeric addition.
In other words, as much as a 10-line program can be, it's a reasonable proxy for a language's performance on algorithmic code.
It tests how well the compiler deals with function calls, and that's it. Useless for measuring a language's overall speed (as are all micro-benchmarks).
Fibonacci is O(log n) or so in the best case, not quite O(1). Of course you can do a simple table lookup if you're limiting yourself to 32 or 64-bit numbers, but you can go far beyond that.
I can understand the need for new programming languages to be continually developed, especially with differing power continuums. So I have nothing against this post, or Julia itself.
But when someone advertises that they want x feature and y feature in one language, such that the respective languages of x and y are essentially obsolete, I get a little skeptical. I'm certainly not the type of person who tries to vaguely advocate all languages are more or less the same and it's just taste; not at all.
But I think there are reasons why it's intrinsically difficult to design a language to have both the speed of C and the abstraction power of Lisp, for example. Things come at a price; this doesn't mean there isn't generally a clear victor for what you're trying to do, just that getting a paragon of linguistic design is very difficult.
But his test of Julia is promising, though honestly I'd really like to see a speed comparison on at least 100 lines of code, preferably a full application really.
It is hard ... OpenDylan (http://opendylan.org/) manages to mostly pull it off when the right type information is available. It could be smarter at times, but it does pretty well.
In a recent test for someone, they had code running in C++ in 0.5 seconds, Dylan in 0.6.
Our major performance loss in OpenDylan comes when there's still too much generic dispatch going on, but that can usually be reduced through various mechanisms. (And as I said above, there's still avenues where the compiler could do better.)
Another interesting implementation is Gambit Scheme which manages to pull off some really impressive feats of compilation for performance once you declare '(not safe)' although it is still fast with safety enabled.
I should also point out that I completely agree that is intrinsically difficult to design a language to have both the speed of C and the abstraction power of Lisp. That is part of why the job that Jeff Bezanson, who is largely responsible for Julia's amazing performance, has done is so incredible.
It's hard to explain until you start using it, but ubiquitous, fast, expressive multiple dispatch is a truly amazing programming paradigm. I'm not entirely sure how it hasn't been adopted as the core paradigm of any mainstream language in the past.
Miles Lubin and Iain Dunning gave an excellent presentation at MIT's IAP tutorial on Julia in January where they benchmarked rather non-trival code for doing linear programming in a variety of languages:
https://github.com/JuliaLang/julia-tutorial/blob/master/NumericalOptimization/presentation.pdf
http://bit.ly/VQ9rDd (shortened version since that's cut off)
See page 7, "Simplex Benchmarks". Essentially, they implemented the basics you need to do a fast simplex linear programming algorithm in each of C++, Julia, Matlab and Python. Their results were in line with the Julia home page benchmarks:
Julia is between 1-2x slower than good C/C++ code.
PyPy and Matlab are 10-20x slower.
Python is 80-400x slower.
Am I the only one with hopes of Julia actually evolving into a general purpose programming language?
With almost C++ level performance and homoiconicity it seems really sweet and I'd love to see it used instead of Go. Though 1-indexed arrays are definitely annoying as fuck...
baremodule MyWay
ref(x::AbstractArray, i::Int) = Base.ref(x,i+1)
ref(args...) = Base.ref(args...)
end
is not that far away (I'm oversimplifying a little, but not much).
However, please don't do this as it breaks coherence within the community. I found 1-based indexes annoying at first, but have grown quite fond of them since.
1-based arrays are one of the more lambasted traits of Lua (an otherwise supremely elegant language), and as a long-time C/C++ programmer, I personally find 0-based arrays very natural.
However what I've found from doing lots of Lua programming, is that I just don't notice very much in practice.
To some degree, at least, I guess this reflects that one writes different sorts of code in different programming languages. The sort of offset-based address calculations one might do a lot in C just don't seem to happen very much in Lua.
So while my natural inclination is to dislike 1-based arrays, I no longer consider them an automatic demerit....
Can anyone venture a guess on how successful this will be ? Given that R is pretty widespread right now, and Scipy/Numpy/Blaze is looking to win in the future (with the Darpa funding for Continuum[1]).
Wonder if Julia was considered for the Darpa grant.
Wow, the list of modules has gotten very respectable since the first time I heard about Julia.
I have one concern though: how well-tested are they? There's one really prominent contributor (John Myles White), who has written a rather large number of packages. But there are only a limited number of bug-free LOCs that a human can write in a given time-span.
So Julia might well be a better language than R, but is it standard library as reliable? And how much time will it take until it becomes as trusted as that of R?
Julia's benchmarks just turned me off. They are ignoring the "reasonable idioms" of each language, and specifically focusing only on RECURSIVE and ITERATIVE algos. And also ignoring other competing peers like lua based GSL Shell.
Please read
https://groups.google.com/forum/#!topic/julia-dev/2MJQ_uADH4...
> Julia's benchmarks just turned me off. They are ignoring the "reasonable idioms" of each language, specifically focusing only on RECURSIVE and ITERATIVE algos.
I read the benchmarks as an invitation to experiment (which I am) - not as a call to language war; if it doesn't float your boat, then that's cool (but you might want to check it out again in a year or two). Your concerns are specifically addressed on the front page in the benchmark discussion (http://julialang.org):
"These benchmarks, while not comprehensive, do test compiler performance on a range of common code patterns, such as function calls, string parsing, sorting, numerical loops, random number generation, and array operations. It is important to note that these benchmark implementations are not written for absolute maximal performance (the fastest code to compute fib(20) is the constant literal 6765). Rather, all of the benchmarks are written to test the performance of specific algorithms, expressed in a reasonable idiom in each language. ... The point of these benchmarks is to compare the performance of specific algorithms across language implementations, not to compare the fastest means of computing a result, which in most high-level languages relies on calling C code."
We all know C compilers make fast machine code, so it seems kind of silly to tediously write apple-oranges benchmarks in ways which will usually just end up benchmarking language X FFI anyway. Excepting BLAS stuff (as noted) pure Julia is within 2x of C, and that is really the take-home message of the benchmarks.
Here's why these small, hot-point benchmarks matter: the reasonable idiom in Julia is to rewrite slow algorithms in the same language. See this recent contribution for a great example of what is possible:
This combination of meta-programming expressiveness with speed in one language is really quite profound. Among other things, I think it will go a long way towards reducing the impedance-mismatch between "C core developers" and "end-user" communities in other technical computing environments (see NumPy,R).
Julia has a beautiful, small, and stunningly accessible language implementation in C (isolated C++ for LLVM stuff); and it is certainly pragmatic by using BLAS,FFTW, etc. where appropriate --- but the numerical core is defined in Julia itself, and - IMHO - that is a very fundamental shift for building an open-source computing environment and community.
(LuaJIT is a masterpiece and it would be cool to see it in the benchmarks, but there are many reasons why I personally have no interest in numerical work using Lua)
I don't think many projects with such a horrible() distribution system succeeded in the past.
()horrible for package maintainers. Try to create a package from this... There are no releases. I don't see any use of DESTDIR. And bundled libraries are also a major setback. It seems like the developers want to make it as hard as possible to distribute their software.
And I see more and more software going this direction. :(
[+] [-] StefanKarpinski|13 years ago|reply
https://github.com/johnmyleswhite/BloomFilters.jl https://github.com/johnmyleswhite/Calculus.jl https://github.com/johnmyleswhite/Clustering.jl https://github.com/HarlanH/DataFrames.jl https://github.com/johnmyleswhite/DimensionalityReduction.jl https://github.com/JuliaStats/Distributions.jl https://github.com/johnmyleswhite/FileFind.jl https://github.com/JuliaStats/GLM.jl https://github.com/johnmyleswhite/Graphs.jl https://github.com/johnmyleswhite/KLDivergence.jl https://github.com/JuliaStats/LM.jl https://github.com/johnmyleswhite/Languages.jl https://github.com/johnmyleswhite/Loss.jl https://github.com/doobwa/MCMC.jl https://github.com/johnmyleswhite/NHST.jl https://github.com/johnmyleswhite/Optim.jl https://github.com/johnmyleswhite/RDatasets.jl https://github.com/johnmyleswhite/Resampling.jl https://github.com/johnmyleswhite/TextAnalysis.jl https://github.com/johnmyleswhite/kNN.jl
The moral of the story might be that John puts his money where is mouth is. He thought Julia was awesome and now he's done a huge amount to make it even more so.
[+] [-] carlob|13 years ago|reply
Doesn't it require O(n^2) function calls for something that can be computed in O(1)?
[+] [-] lukev|13 years ago|reply
In other words, as much as a 10-line program can be, it's a reasonable proxy for a language's performance on algorithmic code.
[+] [-] codeflo|13 years ago|reply
[+] [-] Jabbles|13 years ago|reply
[+] [-] jules|13 years ago|reply
[+] [-] dagw|13 years ago|reply
[+] [-] mikeash|13 years ago|reply
[+] [-] pav3l|13 years ago|reply
Also, previous discussion: http://news.ycombinator.com/item?id=3784349
[+] [-] simulate|13 years ago|reply
[+] [-] dylangs1030|13 years ago|reply
But when someone advertises that they want x feature and y feature in one language, such that the respective languages of x and y are essentially obsolete, I get a little skeptical. I'm certainly not the type of person who tries to vaguely advocate all languages are more or less the same and it's just taste; not at all.
But I think there are reasons why it's intrinsically difficult to design a language to have both the speed of C and the abstraction power of Lisp, for example. Things come at a price; this doesn't mean there isn't generally a clear victor for what you're trying to do, just that getting a paragon of linguistic design is very difficult.
But his test of Julia is promising, though honestly I'd really like to see a speed comparison on at least 100 lines of code, preferably a full application really.
[+] [-] BruceM|13 years ago|reply
In a recent test for someone, they had code running in C++ in 0.5 seconds, Dylan in 0.6.
Our major performance loss in OpenDylan comes when there's still too much generic dispatch going on, but that can usually be reduced through various mechanisms. (And as I said above, there's still avenues where the compiler could do better.)
Another interesting implementation is Gambit Scheme which manages to pull off some really impressive feats of compilation for performance once you declare '(not safe)' although it is still fast with safety enabled.
[+] [-] StefanKarpinski|13 years ago|reply
It's hard to explain until you start using it, but ubiquitous, fast, expressive multiple dispatch is a truly amazing programming paradigm. I'm not entirely sure how it hasn't been adopted as the core paradigm of any mainstream language in the past.
[+] [-] StefanKarpinski|13 years ago|reply
[+] [-] pjmlp|13 years ago|reply
[+] [-] nnq|13 years ago|reply
With almost C++ level performance and homoiconicity it seems really sweet and I'd love to see it used instead of Go. Though 1-indexed arrays are definitely annoying as fuck...
[+] [-] KenoFischer|13 years ago|reply
[+] [-] snogglethorpe|13 years ago|reply
However what I've found from doing lots of Lua programming, is that I just don't notice very much in practice.
To some degree, at least, I guess this reflects that one writes different sorts of code in different programming languages. The sort of offset-based address calculations one might do a lot in C just don't seem to happen very much in Lua.
So while my natural inclination is to dislike 1-based arrays, I no longer consider them an automatic demerit....
[+] [-] EvanMiller|13 years ago|reply
http://docs.julialang.org/en/latest/stdlib/base/#mathematica...
It even has a digamma, Riemann zeta, and Bessel functions with fractional orders, which you won't find in Excel.
The statistics built-ins are a bit weak, however.
[+] [-] KenoFischer|13 years ago|reply
[+] [-] hammerbacher|13 years ago|reply
[+] [-] sandGorgon|13 years ago|reply
Wonder if Julia was considered for the Darpa grant.
[1] http://www.itworld.com/big-data/340570/python-gets-big-data-...
[+] [-] zvrba|13 years ago|reply
I have one concern though: how well-tested are they? There's one really prominent contributor (John Myles White), who has written a rather large number of packages. But there are only a limited number of bug-free LOCs that a human can write in a given time-span.
So Julia might well be a better language than R, but is it standard library as reliable? And how much time will it take until it becomes as trusted as that of R?
[+] [-] ww2|13 years ago|reply
[+] [-] ihnorton|13 years ago|reply
I read the benchmarks as an invitation to experiment (which I am) - not as a call to language war; if it doesn't float your boat, then that's cool (but you might want to check it out again in a year or two). Your concerns are specifically addressed on the front page in the benchmark discussion (http://julialang.org):
"These benchmarks, while not comprehensive, do test compiler performance on a range of common code patterns, such as function calls, string parsing, sorting, numerical loops, random number generation, and array operations. It is important to note that these benchmark implementations are not written for absolute maximal performance (the fastest code to compute fib(20) is the constant literal 6765). Rather, all of the benchmarks are written to test the performance of specific algorithms, expressed in a reasonable idiom in each language. ... The point of these benchmarks is to compare the performance of specific algorithms across language implementations, not to compare the fastest means of computing a result, which in most high-level languages relies on calling C code."
We all know C compilers make fast machine code, so it seems kind of silly to tediously write apple-oranges benchmarks in ways which will usually just end up benchmarking language X FFI anyway. Excepting BLAS stuff (as noted) pure Julia is within 2x of C, and that is really the take-home message of the benchmarks.
Here's why these small, hot-point benchmarks matter: the reasonable idiom in Julia is to rewrite slow algorithms in the same language. See this recent contribution for a great example of what is possible:
https://github.com/lindahua/Devectorize.jl
This combination of meta-programming expressiveness with speed in one language is really quite profound. Among other things, I think it will go a long way towards reducing the impedance-mismatch between "C core developers" and "end-user" communities in other technical computing environments (see NumPy,R).
Julia has a beautiful, small, and stunningly accessible language implementation in C (isolated C++ for LLVM stuff); and it is certainly pragmatic by using BLAS,FFTW, etc. where appropriate --- but the numerical core is defined in Julia itself, and - IMHO - that is a very fundamental shift for building an open-source computing environment and community.
(LuaJIT is a masterpiece and it would be cool to see it in the benchmarks, but there are many reasons why I personally have no interest in numerical work using Lua)
[+] [-] pygy_|13 years ago|reply
https://github.com/JuliaLang/julia/pull/2278
Do you know if the following is single-threaded? I'm not familiar with python.
[+] [-] dhotson|13 years ago|reply
It's really well designed with loads of great features (and decent libraries) for scientific computing.
[+] [-] mrothe|13 years ago|reply
()horrible for package maintainers. Try to create a package from this... There are no releases. I don't see any use of DESTDIR. And bundled libraries are also a major setback. It seems like the developers want to make it as hard as possible to distribute their software.
And I see more and more software going this direction. :(
[+] [-] pygy_|13 years ago|reply
v0.1 is around the corner, though, so, hopefully, things will start to stabilize.
[+] [-] mynameishere|13 years ago|reply
http://www.youtube.com/watch?v=f7PLcHnMNKE
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] unknown|13 years ago|reply
[deleted]