Some of these tests are clearly testing the compiler more than the OS. Looking over the scimark tests, for example, the FFT, LU, and SOR numbers are (at least) an order of magnitude too low to be using optimized libraries that are part of the system. For these benchmarks, we're seeing the compiler quality of compiler codegen on a reference C implementation (OS tasks like malloc/free efficiency may have a small effect, but it should be essentially negligible).
No one interested in performance would actually use that code on OS X or Ubuntu; you would use a tuned library.
Like the article title, there is no clear winner but it does look like certain operations have been optimised on each platform in response to, I'd guess, expected user needs. Which isn't surprising, but it is interesting seeing what specifically each OS designer thinks is the expected use that does require time spent optimising.
Also, quite an in-depth piece of coverage, a lot of benchmarking doesn't take real world into account and just runs 3DMarks or similar.
If there are cases important to you as a user, file bug reports about the performance. OS developers can (and will) guess about what scenarios are most performance-critical, but feedback from developers and users ("doing X is important to me, and is slow on your system" / "why is Y slower on your system than on your competitors" / etc) is enormously valuable and quite welcome. The squeaky wheel gets the grease.
Speaking as an OS developer, poor performance absolutely is a bug, and your bug report will be taken seriously.
I'd say there is a winner : Linux. Isn't it interesting that a software developped by a team of hacker (I'd say a chaotic, huge and ever growing) can make a monster like Ubuntu and get performances that don't pale against another OS which is designed by a very organised, super funded, super focused team of carefully selected programmers...
Ok, I assume a rather romatic view of linux development, but I'd bet I'm not that far from reality :-)
I believe there is an order of magnitude more people/companies involved on the entire Linux ecosystem than there's people involved on OS X. Obviously there's an overlap, some Apple engineers work on open source components, but even then the OS X stack shares more with FreeBSD than Linux.
Linux is very well funded too und the developers are some of the most talented system programmers of our time. Just some facts from Annual Linux Development Report:
* Seventy-five percent of all kernel development is done by developers who are being paid for their work.
* The top 10 organizations sponsoring Linux kernel are Red Hat, Intel, Novell, IBM, Texas Instruments, Samsung, Oracle and Google - all big boys
* Linux has Linus :)
Actually I think Linux has an organizational advantage, because companies can share risks.
That gcc and a lot of the open source technology used by both OSs, is a win for the Open Source movement, and also may explain why performance is similar.
A lot of the linux and GNU software is hardly just developed by "teams of hackers" at this point. As someone who got sent to a RedHat class for a week ( migrating from hpux -> linux), I can attest Linux has large corporate support at this point. My former employer wouldn't have touched it if it wasn't supported.
What a waste of time, OS X has a usable, stable graphical interface, Ubuntu doesn't ( a lot of the lack of usability comes from the lack of quality of third-party apps ). The only use for Ubuntu is as a server, which means testing things for speed that are server related. And only on a Mini.
And yes, I've tried both Ubuntu 12.04 and OS X 10.8 for several days as my only work-based machine to road test both properly.
You're going to get flack for that opinion, but you have a point.
The truth is, Unity sucks. A lot. Even after you get used to it. It was a poor decision by a set of developers out of touch with reality or their user-base, and it's only beginning to recover and move in the right direction slowly.
Everyone I know is migrating to Mint and Xfce/KDE4/Cinnamon (Gnome 3 fork), and with good reason.
OSX is still way ahead of the curve, but Linux (not necessarily only Ubuntu) still has a chance. The idea of Linux is still great: we're hackers, so if you feel there's a problem, help fix it. Not easy, but the fact that you can is pretty cool.
Whether you find OS X usable or not is subjective. I personally don't, preferring window managers over fully fledged desktop environments and such.
In any case at least GNU/Linux distros can easily be customized to fit one's needs. That's an important factor in the software usability metric that is often forgotten.
OS X does have a slick interface, but that interface is not very configurable (as opposed to say KDE4). When it works for you, it works well, when it doesn't there's not a lot you can do about it.
Also there's more and better and easier access to Unix tools on Ubuntu, just about everything is an apt-get install away. I've used third party package systems for OS X but it looks like there's no clear winner among them at least last time I looked, and some software just isn't available in recent versions without compiling or troublshooting it yourself, which I find to be a big waste of time. YMMV.
Intel's official Linux graphics drivers are a part of Mesa. You're probably confusing it with the Mesa software renderer, which is a fallback if you have no accelerated drivers.
that's incorrect. mesa is the 3d graphic library on top of the native intel hd graphic drivers.
the comparison actually make sense. you're not going to intel's osx drivers running in linux and you're not going to get mesa and intel's linux drivers running in osx, you realize that right? right? Oh right it's just trendy to hit on Phoronix for no reason.
Well, they're testing the same games in the different environments to decide which environment is the fastest. That's very much apple to apple.
I'd like to hear your test that's so much more accurate between Linux and Apple gaming with the same hardware. PLEASE ENLIGHTEN US.
I haven't yet benchmarked any flavour of Linux running natively on my current-gen Ivy Bridge 13" MacBook Pro, but I have compared Windows 7 to both System 10.7 and 10.8. Consistently Win7 out performs OS-X on "raw" math-heavy benchmarks.
I've only uploaded my results form Geekbench-2, but the other benchmarking utils show similar numbers:
Geekbench-2 Scores and [Results]:
OS-X 10.8: 8743 [1]
Windows 7: 10092 [2]
Getting a Linux install to run via Bootcamp on this MBP is one of my next projects (most likely Gentoo), and I'm curious to see how well its performance stacks up against the other two. That is taking some time as... well, some things just don't work yet. (Thankfully I don't have to deal with the hybrid GPU setup that's in the Retina MBP.)
Agreed. Unfortunately, nearly every hardware review like this does that. I almost invariably skip to the last page to read the conclusion first, and only poke through the middle pages if I'm curious why the reviewer drew the conclusion they did.
This one is particularly bad, though, because the pages aren't labeled, and there's no table of contents. Which makes "poke through the middle pages" for the purpose of finding a specific test related to the conclusion hard.
We all know it's to bump up the number of page views and therefore ad revenue, but I wonder: For a review like this, how many ad impressions do you need to break even on the reviewer's time, hardware cost (if any), and other overhead? Is ridiculous pagination a necessary evil for these sorts of reviews to exist? Or is it solely to pad margins?
The text loads quickly enough, but the pictures really kill it. I clicked page 2 and got one line of text and a bunch of white, and decided to make that the extent of my user experience.
Lots of data, but preciously little information. For example:
- Why is python so much slower on Mac OS X? if I had to guess, I would say it is more due to a version difference than due to a compiler change. It might also be some particularly slow library code on Mac OS X, though.
- What is causing the sometimes huge differences in CPU-only tasks such as the matrix computations? Compiler, system overhead due to inefficient code, system overhead because the system also performs other work (for example, did the OS index files during some of the runs?), or maybe architectural choices that lead to system overhead (for example, a scheduler might switch tasks more often in order to keep UI response low)?
With such info, this would have been a great article. As it stands now, all I conclude from it is that OpenGL is slower under Ubuntu, and that Linux filesystems are faster (but there, it would help to know why. HFS probably is a large part of the explanation, but it also might be that Mac OS ports of various tools do slightly different things, e.g. By flushing to disk more aggressively.
Things like TLB might also affect your performance, at the cost of memory fragmentation. For example bigger page size (1, 2 or 4MB) would need only two 2 page directory/table lookups, while 4096 pages would need 3.
It's just one of the examples. For example 4096bytes vs 4MB page size:
I was just about to try Xubuntu on my MBP, but after seeing those power consumption stats I don't think I can justify it.
Sad but true: Linux worked better in the old school Desktop era when batteries, power consumption, sleep/hibernation and wireless networks weren't there to mess things up.
All these benchmarks are great and all... but in terms of using Linux on a Macbook Pro, the shitty support for the multitouch trackpad/touchpad alone makes linux barely usable. Unless this has changed recently, in which case someone please correct me
I'm kind of wondering why you would use a Mac without using OSX. The only reason to have that computer is to run the OS that ONLY runs on that computer. Otherwise you can get a better computer for less money that is more flexible.
I love my linux for developing, but no other laptops come close to touching my air when it comes to aesthetics. Which, as a hobbyist painter/photographer, does matter to me.
As someone with a macpro version 1 tower, I can be heartened that I can install the latest Linux, even though apple no longer supports my machine with its OS upgrades. Its only a matter of time (and lack of security updates...) before it would be a very large well designed paperweight
Of course I'm still running snow leopard a 2 versions back OS.
I agree with the rest of the responses you are bound to receive—Apple laptops are pretty solid machines, they easily run multiple operating systems (including but not limited to the respectable OSX), I'm a nerd so "why not play with even more toys?", and I really like the feel of the keyboard.
Just because OSX only runs on Mac hardware doesn't mean that's the only reason to use it as you assume. The non-Apple competition is pretty much crap if you are looking for a high end laptop.
[+] [-] stephencanon|13 years ago|reply
No one interested in performance would actually use that code on OS X or Ubuntu; you would use a tuned library.
[+] [-] nicholassmith|13 years ago|reply
Also, quite an in-depth piece of coverage, a lot of benchmarking doesn't take real world into account and just runs 3DMarks or similar.
[+] [-] stephencanon|13 years ago|reply
Speaking as an OS developer, poor performance absolutely is a bug, and your bug report will be taken seriously.
[+] [-] frownie|13 years ago|reply
I'd say there is a winner : Linux. Isn't it interesting that a software developped by a team of hacker (I'd say a chaotic, huge and ever growing) can make a monster like Ubuntu and get performances that don't pale against another OS which is designed by a very organised, super funded, super focused team of carefully selected programmers...
Ok, I assume a rather romatic view of linux development, but I'd bet I'm not that far from reality :-)
[+] [-] SIULHT|13 years ago|reply
[+] [-] hcarvalhoalves|13 years ago|reply
So, yeah, you're being romantic.
[+] [-] macco|13 years ago|reply
* Seventy-five percent of all kernel development is done by developers who are being paid for their work. * The top 10 organizations sponsoring Linux kernel are Red Hat, Intel, Novell, IBM, Texas Instruments, Samsung, Oracle and Google - all big boys * Linux has Linus :)
Actually I think Linux has an organizational advantage, because companies can share risks.
[+] [-] acomjean|13 years ago|reply
A lot of the linux and GNU software is hardly just developed by "teams of hackers" at this point. As someone who got sent to a RedHat class for a week ( migrating from hpux -> linux), I can attest Linux has large corporate support at this point. My former employer wouldn't have touched it if it wasn't supported.
[+] [-] davidjgraph|13 years ago|reply
And yes, I've tried both Ubuntu 12.04 and OS X 10.8 for several days as my only work-based machine to road test both properly.
[+] [-] calinet6|13 years ago|reply
The truth is, Unity sucks. A lot. Even after you get used to it. It was a poor decision by a set of developers out of touch with reality or their user-base, and it's only beginning to recover and move in the right direction slowly.
Everyone I know is migrating to Mint and Xfce/KDE4/Cinnamon (Gnome 3 fork), and with good reason.
OSX is still way ahead of the curve, but Linux (not necessarily only Ubuntu) still has a chance. The idea of Linux is still great: we're hackers, so if you feel there's a problem, help fix it. Not easy, but the fact that you can is pretty cool.
[+] [-] mhd|13 years ago|reply
Not that Ubuntu/Unity fares better in that department, but right now, all the major players aren't really doing a swell job with their desktops.
[+] [-] lumberjack|13 years ago|reply
In any case at least GNU/Linux distros can easily be customized to fit one's needs. That's an important factor in the software usability metric that is often forgotten.
[+] [-] shell0x|13 years ago|reply
[+] [-] sixbrx|13 years ago|reply
Also there's more and better and easier access to Unix tools on Ubuntu, just about everything is an apt-get install away. I've used third party package systems for OS X but it looks like there's no clear winner among them at least last time I looked, and some software just isn't available in recent versions without compiling or troublshooting it yourself, which I find to be a big waste of time. YMMV.
[+] [-] darkstalker|13 years ago|reply
They're comparing apples to oranges. Mesa are like default graphic drivers, with very little or no hardware acceleration.
[+] [-] krakensden|13 years ago|reply
[+] [-] zobzu|13 years ago|reply
the comparison actually make sense. you're not going to intel's osx drivers running in linux and you're not going to get mesa and intel's linux drivers running in osx, you realize that right? right? Oh right it's just trendy to hit on Phoronix for no reason.
Well, they're testing the same games in the different environments to decide which environment is the fastest. That's very much apple to apple.
I'd like to hear your test that's so much more accurate between Linux and Apple gaming with the same hardware. PLEASE ENLIGHTEN US.
[+] [-] cydonian_monk|13 years ago|reply
I've only uploaded my results form Geekbench-2, but the other benchmarking utils show similar numbers:
Geekbench-2 Scores and [Results]:
Getting a Linux install to run via Bootcamp on this MBP is one of my next projects (most likely Gentoo), and I'm curious to see how well its performance stacks up against the other two. That is taking some time as... well, some things just don't work yet. (Thankfully I don't have to deal with the hybrid GPU setup that's in the Retina MBP.)[+] [-] mcormier|13 years ago|reply
[+] [-] jpk|13 years ago|reply
Agreed. Unfortunately, nearly every hardware review like this does that. I almost invariably skip to the last page to read the conclusion first, and only poke through the middle pages if I'm curious why the reviewer drew the conclusion they did.
This one is particularly bad, though, because the pages aren't labeled, and there's no table of contents. Which makes "poke through the middle pages" for the purpose of finding a specific test related to the conclusion hard.
We all know it's to bump up the number of page views and therefore ad revenue, but I wonder: For a review like this, how many ad impressions do you need to break even on the reviewer's time, hardware cost (if any), and other overhead? Is ridiculous pagination a necessary evil for these sorts of reviews to exist? Or is it solely to pad margins?
[+] [-] lathamcity|13 years ago|reply
[+] [-] Someone|13 years ago|reply
- Why is python so much slower on Mac OS X? if I had to guess, I would say it is more due to a version difference than due to a compiler change. It might also be some particularly slow library code on Mac OS X, though.
- What is causing the sometimes huge differences in CPU-only tasks such as the matrix computations? Compiler, system overhead due to inefficient code, system overhead because the system also performs other work (for example, did the OS index files during some of the runs?), or maybe architectural choices that lead to system overhead (for example, a scheduler might switch tasks more often in order to keep UI response low)?
With such info, this would have been a great article. As it stands now, all I conclude from it is that OpenGL is slower under Ubuntu, and that Linux filesystems are faster (but there, it would help to know why. HFS probably is a large part of the explanation, but it also might be that Mac OS ports of various tools do slightly different things, e.g. By flushing to disk more aggressively.
[+] [-] malkia|13 years ago|reply
It's just one of the examples. For example 4096bytes vs 4MB page size:
http://en.wikipedia.org/wiki/File:X86_Paging_4K.svg
http://en.wikipedia.org/wiki/File:X86_Paging_4M.svg
[+] [-] unicornporn|13 years ago|reply
[+] [-] navpatel|13 years ago|reply
[+] [-] donniezazen|13 years ago|reply
[+] [-] indiecore|13 years ago|reply
[+] [-] antirez|13 years ago|reply
[+] [-] ImprovedSilence|13 years ago|reply
[+] [-] JCB_K|13 years ago|reply
[+] [-] acomjean|13 years ago|reply
Of course I'm still running snow leopard a 2 versions back OS.
[+] [-] calinet6|13 years ago|reply
[+] [-] kevinpet|13 years ago|reply
I'm pretty sure Linux support for a MacBook Air is pretty good, seeing how that's Linus's primary machine: http://techcrunch.com/2012/04/19/an-interview-with-millenium...