[Pewpewarrows]

Completely unscientific, but if these outputs are any indication, this is going to be great news for Rubby users in the future...

  $ time ruby -e "puts 'hello world'"                                                                                                                           
  hello world

  real    0m0.184s
  user    0m0.079s
  sys     0m0.092s

  $ time ~/Downloads/topaz/bin/topaz -e "puts 'hello world'"                                                                                                    
  hello world

  real    0m0.007s
  user    0m0.002s
  sys     0m0.004s

[raphael_kimmig]

There is a neural net example benchmark in the topaz git repo. Don't know how representative that example is, but at least startup time shouldn't be dominating the results...

  $ ruby -v
  ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.3.0]
  $ ruby bench_neural_net.rb
  ruby bench_neural_net.rb  17,74s user 0,02s system 99% cpu 17,771 total

  $ bin/topaz bench_neural_net.rb
  bin/topaz bench_neural_net.rb  3,43s user 0,03s system 99% cpu 3,466 total

[ErikRogneby]

should be able to run any number of ruby examples from here: http://benchmarksgame.alioth.debian.org/u32/ruby.php

[pserwylo]

You should check out a tool I heard about recently called "ministat". An example of how to use it to benchmark multiple runs, and then compute the statistics from each run are available here:

http://anholt.net/compare-perf/

Example output:

  +------------------------------------------------------------------------------+
  |                   +                            x                             |
  |                   +                            x                             |
  |           +       +                            x x            x              |
  | +    ++   +++     +                            x xxx xx       x              |
  |++ ++++++++++++++++++                   x  x    x xxx xxx  xxxxx              |
  |++ ++++++++++++++++++ +++ +       ++   xxxxxxxxxx xxxxxxxxxxxxxxxx  xx x    xx|
  |     |______MA______|                        |________A________|              |
  +------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x  57      45.62364     46.437353      45.93506     45.951554    0.19060973
  +  57     44.785579     45.534727     45.042576     45.056702    0.16634531
  Difference at 95.0% confidence
    -0.894852 +/- 0.0656777
    -1.94738% +/- 0.142928%
    (Student's t, pooled s = 0.178889)

[grn]

In order to eliminate the potential bias of startup time I run similar test in many iterations. Here's what I got:

    $ time ruby -e "10000.times { puts 'hello world' }" > /dev/null
    real    0m0.102s
    user    0m0.096s
    sys     0m0.005s

and

    $ time ./topaz -e "10000.times { puts 'hello world' }" > /dev/null
    real    0m0.098s
    user    0m0.071s
    sys     0m0.026s

Any idea why I don't see such big difference?

[fijal]

topaz probably has a pretty slow IO (for bad reasons, it's an RPython problem)

[duaneb]

Because the majority of the cycles ran are probably in 'puts', which is implemented in C if I were to guess.

[emillon]

The overhead of starting the interpreter must be longer that executing the loop.

[rbehrends]

Because a puts loop is I/O bound, not CPU bound.

[fernandezpablo]

what is your 'ruby'?

[draegtun]

To scratch a quick interesting thought that came into my head I just had a look at Cardinal, which is a Ruby implementation running on the Parrot VM - https://github.com/parrot/cardinal

So downloaded & built Cardinal (which went seamlessly however I did have Parrot already installed) then I did same benchmarks alongside ruby1.8 here:

  $ time ruby -e "puts 'hello world'"
  hello world
  
  real	0m0.130s
  user	0m0.049s
  sys	0m0.071s

  $ time parrot-cardinal -e "puts 'hello world'"
  hello world

  real	0m0.057s
  user	0m0.037s
  sys	0m0.019s

Very interesting because I thought Cardinal was supposed to be slow!

I think more diverse benchmarks are required. And when time permitting I might add Topaz & ruby1.9 into the mix.

[Cowen]

Whenever I see frontpages for these kinds of projects like "a faster X" or "X written in Blub", the first thing I want to see on the frontpage is how this new project compares to X in terms of quality and performance. Even specious benchmarks would help more than zero benchmarks.

I wish more frontpages for these kinds of projects would do that.

[pyre]

If they put that on their frontpage, there would be at least 20 posts on here bashing them for it because they didn't get it right (or just accusing them of outright lying/incompetence).

[callum85]

"Even specious benchmarks would help more than zero benchmarks."

I disagree. Zero benchmarks is definitely better than specious benchmarks.

[amalag]

spe·cious /ˈspēSHəs/ Adjective Superficially plausible, but actually wrong: "a specious argument". Misleading in appearance, esp. misleadingly attractive: "a specious appearance of novelty".

So you are saying you would prefer wrong information?

[haven]

Expanding on the concept of entirely unscientific benchmarks, I benched some simple prime number math with Topaz, Ruby, JRuby and RBX.

Looks promising for Topaz! https://gist.github.com/havenwood/4724778

[fijal]

the idea was to implement enough of "hard stuff". So it should not change. but hello world is not a good idea (this one also might actually change due to library loading, but please don't benchmark it like that)

[stcredzero]

> the lack of many core features probably helps out a lot atm.

I don't understand why the lack of features would affect the time of "hello world."

[seivan]

Try comparing with Rubinius

[eranation]

Impressive, how does it compare to JRuby?

[andrewaylett]

That's probably not a fair test to use on JRuby, as the JVM is notoriously slow to start.

[kingkilr]

Topaz doesn't run rails yet (as far as I know, I didn't even dare to try!), so I doubt you'll find any benchmarks ;) There is one benchmark in the bench/ directory of the repository you can try though!

[fusiongyro]

Instead of asking, you could read the linked article, which itself says it isn't complete enough to run Rails yet.

[charliesome]

How the hell is your Ruby that slow to start?

    $ time ruby -e "puts 'hello world'"
    hello world
    
    real    0m0.011s
    user    0m0.008s
    sys     0m0.003s

[d33pika]

The first time I ran:

   time ruby -e "puts 'hello world'"
   hello world

   real	0m0.221s
   user	0m0.005s
   sys	0m0.006s

subsequent times:

   time ruby -e "puts 'hello world'"
   hello world

   real	0m0.008s
   user	0m0.005s
   sys	0m0.003s
 

So, I guess he ran ruby first followed by topaz and ended up with those results

[snissn]

    $ time ruby -e "puts 'hello world'"
    The program 'ruby' can be found in the following packages:
     * ruby1.8
     * ruby1.9.1
    Ask your administrator to install one of them

    real	0m0.060s
    user	0m0.040s
    sys	0m0.016s

[axomhacker]

Don't know if you are trolling. But if this is genuine: this is rbenv telling you that you have multiple rubies installed.

1. Pick one (just for this session): $ rbenv shell ruby1.9.1

2. And then run the example.

By the way 1.9.1 is really old already, 1.9.3 has a lot more bug fixes.