top | item 23237008

(no title)

no_gravity | 5 years ago

I wanted to play with variations of the code. For that it is useful to make it output a "summary" so you know the variation you tried is computationally equivalent.

For the first benchmark, I added a combined string length calcuclation:

    def main():
     r = 0
     for j in range(20):
       for i in range(1000000):
         r += len(str(i))
     print(r)
    main()
When I execute it:

    time python3 test.py
I get 8.3s execution time.

The PHP equivalent:

    <?php
    function main() {
     $r = 0;
     for ($j=0;$j<20;$j++)
       for ($i=0;$i<1000000;$i++)
         $r += strlen($i);
     print("$r\n");
    }
    main();
When I execute it:

    time php test.php
Finishes in 1.4s here. So about 6x faster.

Executing the Python version via PyPy:

    time pypy test.py
Gives me 0.49s. Wow!

For better control, I did all runs inside a Docker container. Outside the container, all runs are about 20% faster. Which I also find interesting.

Would like to see how the code performs in some more languages like Javascript, Ruby and Java.

discuss

order

iruoy|5 years ago

I've added rust using this code

    fn main() {
        let mut r = 0;

        for _x in 0..20 {
            for y in 0..1_000_000 {
                r += y.to_string().len();
            }
        }

        println!("{}", r);
    }
Surprisingly PyPy is the fastest

    % hyperfine target/release/perftest "php perftest.php" "python perftest.py" "pypy perftest.py" -w 3      
    Benchmark #1: target/release/perftest
    Time (mean ± σ):     624.8 ms ±   9.8 ms    [User: 623.0 ms, System: 0.8 ms]
    Range (min … max):   614.5 ms … 644.0 ms    10 runs
    
    Benchmark #2: php perftest.php
    Time (mean ± σ):     697.8 ms ±  18.3 ms    [User: 696.7 ms, System: 1.1 ms]
    Range (min … max):   650.1 ms … 718.0 ms    10 runs
    
    Benchmark #3: python perftest.py
    Time (mean ± σ):      3.326 s ±  0.071 s    [User: 3.313 s, System: 0.003 s]
    Range (min … max):    3.232 s …  3.419 s    10 runs
    
    Benchmark #4: pypy perftest.py
    Time (mean ± σ):     270.7 ms ±   5.7 ms    [User: 257.5 ms, System: 13.0 ms]
    Range (min … max):   257.8 ms … 277.8 ms    11 runs
    
    Summary
    'pypy perftest.py' ran
        2.31 ± 0.06 times faster than 'target/release/perftest'
        2.58 ± 0.09 times faster than 'php perftest.php'
        12.29 ± 0.37 times faster than 'python perftest.py'

steveklabnik|5 years ago

Allocating a string to compute the length and then throwing it away is gonna be real slow, yeah...

dralley|5 years ago

Due to Rust's concern about thread-safety, the standard println! macro acquires a lock prior to writing to stdout. This means that if you're doing a lot of printing, it can be quite slow due to the extra work of acquiring and releasing that lock.

This is fairly well known and if it's a bottleneck in your program the recommendation is to acquire the lock yourself and then write directly to stdout.

eggsnbacon1|5 years ago

My Java timing is very close to Rust. I have a feeling PyPy is eliminating the string conversion as dead code. When I alter my Java toy code to not prevent elimination of the toString() call it runs close to speed of PyPy.

hu3|5 years ago

The next PHP release will sport a JIT compiler [1].

I'd wager it will reach similar performance than pypy.

[1] https://stitcher.io/blog/new-in-php-8

winrid|5 years ago

Didn't read yet, but I imagine it'll store the JIT'd code on disk for further requests?

This will be amazing when it happens.