top | item 12092571

Blind-tested soloists unable to tell Stradivarius from modern violins

310 points| ZeljkoS | 9 years ago |thestrad.com | reply

246 comments

order
[+] jarmitage|9 years ago|reply
I used to work at a musical instruments startup and we tried blind tests of prototypes to assess which combination of materials recipes/sensor design/firmware algorithms etc were 'the best'.

Ultimately these tests only reveal the biases of the players. They will prefer – and actually perform better with – that which they believe to be configured in a certain way, regardless of whether it is in reality or not. We had one prototype that us engineers hated to death because the only difference between it and the rest was that it was a different colour, but ALL the players hailed it as the golden standard.

The same goes for the famous 'Does Fuck All' button on recording mixing desks, which you can use to tell players that you've made the change that they wanted to satisfy them. And also when you're buying speakers it's very common to be shown the same set of speakers three times and have three opinions.

I just recently tour managed a friend and would set up her mic sound every night, and each time I would need to play some sort of subtle trick with her so that she felt things were just right. No amount of 'rational tweaking' could achieve the same.

[+] dzdt|9 years ago|reply
Take that as a proof that Stradivari really was VERY good. Compared to the very best that can be produced centuries later, with much better technology available, his instruments hold their own.

Contrast with, say, athletes. The very best runners from a century ago wouldn't qualify for the olympics today.

It does say that the extra value in a Stradivarius is in the history or mystique (or more cynically, the branding) as opposed to the sound.

But the sound is there as well, just it can be matched by a modern top quality instrument.

[+] et-al|9 years ago|reply
I understand your point of the "best of the past" still holding up to the best now, but considering that the violin is a "classical instrument", are contemporary violin manufacturers trying to replicate the past because that's the template to follow?

If it was possible for a violin to objectively sound better than a Strad, would listeners interpret necessarily it as better?

[+] bratsche|9 years ago|reply
> Take that as a proof that Stradivari really was VERY good

The other thing that I think is very interesting is that Stradivari really developed his own pattern for violins, while modern makers have a lot of knowledge to build upon.

He may not have been the first to copy Strads (or he may have been, I'm just not sure) but Vuillaume became very famous for making masterful copies of some Strads. In 1855 he purchased 144 instruments by Stradivari and some other of the "great Italian masters", I think mostly as a dealer but he also set out making great copies of Le Messie (a Strad), Il Cannone (a Guarneri, which belonged to Paganini), and some others.

I think Vuillaume's violins then became patterns for other instrument makers to use, and we have this legacy now where most modern violins are patterned after a Strad or a Guarneri.

[+] mkl|9 years ago|reply
> Contrast with, say, athletes. The very best runners from a century ago wouldn't qualify for the olympics today.

I agree with your point, but this isn't true at all [1]. The rules have changed so dramatically that you get a completely false idea. E.g. Jesse Owens ran on cinders, started without blocks, and did not have a training regimen anything like today's pros.

Perhaps a better argument is that the Stradivarius instruments which survive are not the best few, selected over centuries, but ~60% of all the instruments Stradivari ever made. I.e. he was consistently great.

[1] https://www.ted.com/talks/david_epstein_are_athletes_really_...

[+] Retric|9 years ago|reply
We don't use high tech manufacturing because people assume violins should be made to look and sound like Stradivarius. That and it's also a tiny market.
[+] rikkus|9 years ago|reply
This isn't a better/worse comparison. It's a differentiation-on-feel-and-sound comparison. The only thing this proves is that it's now possible to replicate the feel and sound of a Stradivarius to an impressive degree.

Double blind recorded audio tests often produce the same result: Listeners can't tell the difference between two DACs, two codecs, two amplifier mains cables(!) but as soon as the difference[s] are audible above a threshold, preference comes into play.

However a modern violin is produced, and however it is objectively 'better' than a Strad (if that's even possible to quantify) it will never be subjectively better for all players or listeners, if there is an audible difference - and therefore Strads will always be seen as the 'best'.

[+] justusw|9 years ago|reply
There is a great episode on Planet Money about Strads and other objects that have a high perceived value due to the history and branding that come with them.

They cite the same 2012 Indianapolis study mentioned in this article. The researchers have an amazing eye for detail. A perfume was applied to all violin's chin rests so that musicians would not be able to tell them apart based on smell. Tells you a lot about brand perception using all our senses.

This is the Planet Money episode: http://www.npr.org/sections/money/2016/06/22/482936331/episo...

[+] flopto|9 years ago|reply
Planet money's coverage is great; I think a bunch of the posts below would be answered by listening to the podcast.

One of the main things I remember taking away from the podcast was that the soloists in the study were not the most famous "best" violinists in the world (they may not even have been professionals?). One of the violin makers they interviewed thought this was the biggest weakness of the study, and he was confident he could tell the difference (not that that means too much).

[+] saiya-jin|9 years ago|reply
similar to say katanas - best hand-made today are probably better than best of the past (after few hundred centuries steel is not the same, even if blade was very well taken care of). yet old preserved blades are priced as luxury collectibles, not only due to interest of various anime watchers. and all they do is to collect dust on somebody's wall
[+] tibbon|9 years ago|reply
I've been playing guitar (and other stringed && fretted instruments) for around 23 years (including a music major in college).

I don't think old guitars are inherently better, and I have gotten my hands on some rather amazing pieces (1914 Gibson Loar F5, 1959 Les Paul, 1955 Telecaster, etc) and they were really fun. I've also played some guitars that people like collecting that were absolute garbage instruments, including a 1966 Gibson 330 that was just horrid.

But here's the weird thing that I do not understand, the hallmark of a great guitar to me isn't the sound, it's the "songs stored in it". Yea, that's some hippie-woo-bullshit, but almost every guitar I pick up that I like has me instantly playing something that I've never played before in my life. Some guitars, despite being technically excellent, just don't have them in there. Other guitars, if I pick them up in the studio I've suddenly written some really interesting pieces.

This probably actually just comes down to comfort, feedback loop response of sound, weight, small details, etc. I don't think there's any memory to an instrument that stores these; but at the same time, I can go through a wall of guitars and pick out my favorite ones this way. There is only a slight correlation to price/age on these.

[+] yolesaber|9 years ago|reply
I was in a music store six months ago buying strings for my old squire electric. I was planning on actually sitting down and learning guitar -finally- after years of procrastination. As I approached the register I saw an old Danelectro convertible from 1966 hanging by the counter. Now I hadn't really any chops at all, I could bang on G C F progressions but not much else. But when I picked up that guitar I immediately played a beautiful country twanging melody and I was able to ride it into a nice satisfying ending. The guy behind the desk was surprised and starting asking me if I played in any bands / made musician small talk. I didn't know what to say, but I bought that guitar and have been playing on it so much since. Before that, I had been unable to concentrate and maintain a practice regimen past two weeks.

A poor craftsman might blame his tools, but the best tools do have history and lineage to them.

[+] 6stringmerc|9 years ago|reply
Hi there fellow picker, I'm in about the same range of time with a guitar in my hands. Extensive bouts of hours upon hours in the woodshed. Bought & sold my fair share of different types. Just love playing more than anything.

I'm actually second generation and my Dad worked in a music shop & as a guitar teacher back in the 65-70 period of time. He routinely commented to me that some instruments were just horrible quality to deal with. Low tech and low standards of production. He could understand the "rarity" or "scarcity" value (though hated the inflated prices non-playing collectors did to guitars) but told me time and again that better construction techniques of today are worth appreciating.

And, honest to goodness, I've got a similar voodoo-like relationship regarding one particular guitar of his. It's a 1981 Fender Strat Elite, an un-loved high-end thing he picked up and just kept around. It's got this weird triangular 'V' cut to the neck, a hard ebony fretboard, and when the garbage floating trem is locked down, it plays like nothing else. I've written more riffs, ideas, and can do so again and again without fail every time I touch it. The strangest thing, and while I don't care to put a technical description on why it's magic (I do like the Washburn Dime Custom Shop necks, which also had a pronounced 'V'), I can't deny there's just something to the relationship. Cool to see it's not just me.

[+] Intermernet|9 years ago|reply
Regarding "songs stored in it", I've also found that certain guitars, as well as certain amp / head / mic combos inspire certain styles. For instance, I was fiddling around a few months ago and somehow managed to replicate very closely the sound that Neil Young used for a lot of the Dead Man soundtrack (I was messing around with a Gibson LP Junior, Marshall JTM45 re-issue amp, Marshall MX112 cab, Rode NT3 off-axis mic). This inspired me to improvise for hours around the main theme from Dead Man.

Strangely, I then moved to my favorite acoustic (Maton 1973 CW80) and discovered that it loves being played in the same style! I'd never assumed that a piece that relies so heavily on dissonant electric reverb would translate so well to a clean acoustic sound. I now tend to improvise on my Maton in a completely different style as I've discovered that it has a whole new family of "songs stored in it"!

Regarding price/age to quality, one of my most played guitars is actually a cheap nylon Martinez (around $150) that I bought so I could take it camping etc. and not worry about it being broken or stolen. It's got a great sound, and is incredibly playable.

[+] TylerE|9 years ago|reply
One of my pet theories is that the 'vintage era' guitars were mostly made with old growth timber. These days, unless you're dealing with extremely specialist woods (like ash that's sat under a swamp for centuries), that's just not the case.
[+] squozzer|9 years ago|reply
That's a good story. Personally I always figured Slash could play an Epi as well as a Gibson. But my modest abilities probably need a more expensive guitar as a talisman. This is good news for the makers of expensive guitars.
[+] arnold_palmur|9 years ago|reply
1959 Les Paul... That's the holy grail right there - what was that like?
[+] StavrosK|9 years ago|reply
I've never understood the mismatch in this test. Inability to distinguish isn't the same as "being better" (strictly speaking, it's pretty much the opposite). What if people could tell Stradivarius from modern violins, because modern violins sounded better?

I understand that this test proves that modern violins sound just as good as Stradivarius, but how do you tell which sounds better? I guess you make a blind test where you ask people which of the two things they like best...

The test says that they couldn't distinguish if the violins were old or new, but they got 31/33 guesses wrong, which is a 93% failure rate, a.k.a. 93% success rate, far higher than the 50% you'd expect by chance.

They seem to be conflating two tests, one being "Can you tell which violin is new?" and one being "Which one do you prefer?"

Maybe the title should be "blind-tested soloists overwhelmingly prefer modern violins to Stradivarius".

/rambling

[+] daveguy|9 years ago|reply
> I understand that this test proves that modern violins sound just as good as Stradivarius, but how do you tell which sounds better?

> this test proves that modern violins sound just as good as Stradivarius

> how do you tell which sounds better?

The test just proved that neither sounds better, that modern ones sound just as good. The same level of sounding good means neither one is better than the other, right?

Edit: also, FTA -- "In total, 33 of the soloists’ guesses were wrong and 31 right, with 5 indeterminate." That's about 50/50 not 93%. They are the same. There is no difference except lore.

[+] curveship|9 years ago|reply
I think you may have misunderstood the article.

There were two tests, one to distinguish strad from modern, one to score which instrument musicians liked best. In the first, they didn't get 31 of 33 wrong, they got 33 wrong and 31 right, virtually a 50/50 split. In the preference test, modern violins placed first and second, with the top place substantially above the others, though the article doesn't give the full score breakdown.

[+] wrsh07|9 years ago|reply
I'd prefer your title. It seems more honest, and there's no reason to say that's not the case.

But I think pointing out that musicians can't distinguish between these instruments is essential to debunking this notion that Strads are somehow uniquely better than anything else.

Turns out, Strads are high end violins that sound great after two hundred years. If you're a professional violinist, don't waste your money to "further your career."

[+] grenoire|9 years ago|reply
You are not suppposed to divide 31 by 33, but by 64. It is actually very close to the 50% expectation.
[+] elihu|9 years ago|reply
There's also a distinction to be made between the questions "does a Stradivarius sound different than a modern violin" and "can a performer successfully identify a Stradivarius?"

Supposing that a performer does not in fact know what a Stradivarius sounds like and is unable to determine which violin is the Stradivarius does not mean that the Stradivarius does not in fact sound different from a modern instrument never mind which one of those sounds "better".

(I believe there was also a problem with the original test that I don't know if they addressed this time around, which is that it's pretty hard to get permission even to do things like change the strings on a very old violin. So, if they're testing a Stradivarius with old strings against a modern violin with new strings, that's not exactly a fair test.)

[+] Houshalter|9 years ago|reply
According to the article they really did prefer modern instruments.

>The results revealed the two most-preferred instruments to be modern, while in third place was a violin from Stradivari’s ‘golden period’. At the opposite end of the scale a Stradivari drew the poorest result and a modern instrument was placed second-last.

But they couldn't distinguish which instruments were which reliably. Suggesting that even if there is a preference, it's very weak and probably doesn't matter very much.

[+] tsm|9 years ago|reply
As a professional violinist who's played both a Strad and a del Gesu (and who spent several months with a 1770 Nicolo Gagliano), and who studies with someone whose daily driver is a del Gesu (and who's been playing antique Cremonese instruments for the past 25 years), I always have problems with this sort of study.

As my teacher put it, "Modern instruments may have a voice that's as good as an antique's, but they only have the one voice. Due to how age works with violins, antique violins have many beautiful voices. In a couple hundred years a 21st century instrument with the same craftsmanship and material quality as Strads and del Gesus—and these instruments do exist—will be as good as a the Strads and del Gesus do now. But right now that's not the case."

And I've found this to be true myself. When I pick up a modern violin it might take some time to figure out how to make it sound best, but then I'm done. With an antique instrument, it turns into this indefinitely-long relationship where I continuously learn new things and find new voices and tone colors in the instrument. When I returned the aforementioned Gagliano after three months of heavy use, I felt like I'd only just begun to discover the range it had available.

And it really is about age for age's sake, not quality. I own a good-but-not-great Edinburgh-made violin from 1807 appraised at $15K, and I've seriously played a modern-made violin appraised for $60K. The "default voice" of the $60K violin is better, but the antique instrument is definitely the more versatile one.

I know we like to be objective and scientific here on HN, but don't discard the fuzzy talk of "forming a relationship with the violin" too soon. Even if we don't understand the science behind it yet, it seems that there's something to it.

A better experiment that'll never happen because of logistics:

Present a professional violinist with ten antique violins, ten modern violins, and a range of high-quality bows (the bow plays a HUGE role in how the violin sounds and responds). Give him/her a few hours to choose the favorite of each type of violin and a complementing bow. Let him/her play both violins for a year and then report back on which was preferred. Repeat for a statistically-significant number of violinists.

[+] i000|9 years ago|reply
> the antique instrument is definitely the more versatile

Not definitely. Only subjectively with an n=1 and botched experimental design. Please repeat blinded, a statistically-significant number of times with different violonists.

[+] Johnny_Brahms|9 years ago|reply
I suspect there might be something to that. I worked as a professional musician in an orchestra for a while before tinnitus ended my career. I played a professional bassoon by Fox (an american company that makes good bassoons) and switched to the "original" bassoon maker - Heckel. Even though both instruments were new, and even though the "base sound" of the fox was good - if not even better than my heckel - I very much preferred the Heckel. Even though the basic sound was maybe a bit worse on the heckel, the posibilities were greater.

That is an important part of what constitutes a good instrument.

[+] Etheryte|9 years ago|reply
So essentially the only difference between the test that you proposed and what was conducted is that yours is longer and the conducted test was double blind?
[+] tetraodonpuffer|9 years ago|reply
if you allow me a devil's advocate comment, don't you think the fact that playing a modern violin and figuring out how to making it sound the best and being 'done' could be because the modern violin is constructed better?

If a modern violin has tighter tolerances maybe it is less sensitive to moisture / temperature changes, making it sound 'the same' under a larger set of conditions than the antique instrument.

The antique instrument being less 'perfect' could be more sensitive to day-to-day environmental changes, making it more challenging to play (as you have to learn to compensate for these) but not necessarily because it is a better instrument, but possibly because it is a worse one technically speaking.

Now of course having an instrument that you have to 'figure out' all the time likely makes you a much better player, able to adapt to other instruments more easily and so on, so it might well be that a 'less perfect' instrument is the superior one, but just wondering about this

[+] tinco|9 years ago|reply
In truth it would be rather embarrassing if a top of the line modern violin could not at least equal a Stradivarius. Besides the world economy being much stronger, the technological advancements and the sheer degree of knowledge transfer makes it almost inevitable.

Anyone can fire up youtube and immediately peer into the workshops of master luthiers. I bet you could take a couple million and build a top class violin workshop from scratch in just a few years.

Imagine how cool it would be if the big SV aristocracy instead of spending millions on building sailboats, they'd invest in developing musical instruments? As much faster the USA-17 is than any hundred year old yacht, as much more beautiful a violin could be than a stradivarius.. perhaps.

[+] michaelbuckbee|9 years ago|reply
Growing up, I still remember articles about the "lost art of making violins" and how there was all this speculation about what made them so special. Was it a chemical or fungus in the river the logs were floated in? Some lost technique? A particular sequence of aging, resting, bending the wood?

Turns out it's none of that and just branding (albeit branding around having a great product at that point in history).

[+] blhack|9 years ago|reply
Isn't the appeal of a strad much more "I am playing an instrument which was built in the 17th century, one that has been played by hundreds of years worth of masters, played for kings and queens, played almost exclusively by people who have dedicated their lives to the same thing I have dedicated my life to", and not "sure sounds better than the other ones!"?

If I could sit down at Da Vinci's desk, in his library...don't you think the importance of that would be the history behind it, and absolutely not that somehow the lighting in the library made reading better, or the sound of the birds outside made retaining information easier or some silliness like that?

Strads, like really anything else related to art, are about context.

[+] mcphage|9 years ago|reply
Something I don't see mentioned here, that I remember from when this first came out, was that the modern violins were $100,000+ instruments, not just run-of-the-mill violins.
[+] perlgeek|9 years ago|reply
Note that in 2010 study, the test asked the question "which of these violins would you like to take home?", not "which of these do you think is a Stradivarios?" (see http://www.thestrad.com/cpt-latests/stradivari-loses-out-in-...)

The follow-up study seemed to ask a similar question.

So these studies tried to find out which violins are most popular among high class violin players, not (primarily) whether they were able to tell the difference.

I've read elsewhere about the first study that the the participants were quite surprised that people thought they couldn't identify the Stradivarius, because they were never asked to.

[+] vilhelm_s|9 years ago|reply
In this study they did both, they had a long session where the players where asked to rank the instruments based on how they liked them (and rate various characteristics like loudness and timbre), then at the end they had a "new or old" question:

"Next they were presented with a series of violins (one at a time, in random order) and given 30 s to play each one before guessing what kind of instrument it was. If a soloist was unclear about the meaning of the question, he/she was prompted to guess whether the violin was new or old. The series consisted of (i) that player’s favorite old violin; (ii) the player’s favorite new violin; (iii) an old and a new violin the player found unsuitable; (iv) the old violin and the new violin that, in session 1, were most often included in top-four lists and that were on average most highly ranked within those lists; and (v) the old and the new violin that were most often rejected as unsuitable in session 1."

[+] white-flame|9 years ago|reply
Another issue with the 2010 study is that the performers were not allowed to tune or adjust the Strads in any way, but could do so with the new ones, so the old violins sounded objectively worse. And yes, they could absolutely tell which was which, and that the modern ones sounded better often due to the fine tuning in the setup.
[+] HenryTheHorse|9 years ago|reply
I've owned cheap guitars and I have a "nice" Fender. I'd much rather play the Fender. But as any guitar player will tell you, you can put a cheap Strat knock-off in Jimi Hendrix's hands and it will still sound like Jimi playing his Strat.

Conversely, a Yehudi Menhuin or a Joshua Bell need a good instrument. At their level of playing, little things affect their performance.

[+] fhood|9 years ago|reply
Great, now I want to see this with tube amps, and pre-67 strats. Because I am now even more convinced that musicians are full of crap.
[+] dzdt|9 years ago|reply
The full text PNAS article is here : http://m.pnas.org/content/111/20/7224.full.pdf?with-ds=yes

Some notes. The study started with a pool of 13 new and 9 old instruments, and pre-selected down to the test set of the best 6 new and 6 old among these.

Among the test set, new instruments had a slightly higher average score than old ones, and the top two instruments were new. But all soloists picked a mix of new and old instruments in their top 4, and in the overall rankings new and old are well mixed. From high to low based on combined session scores, the list goes NNOONNOONONO.

Of the soloists, 7 prefer an old instrument for their performances, 2 a new one, and 1 switches between an old and new.

[+] InclinedPlane|9 years ago|reply
I'm tempted to believe these results, as I suspect a lot of the reputation of stradivarius violins is overblown. However, there is almost always a huge problem with these sorts of tests: the pepsi challenge problem. With subjective measurements it can often be difficult to robustly test the differences between two things in a short sample period. People may say one or another sounds better but that might only apply for a small sample. Or, people may just be unable to explicitly say which one they actually prefer when confronted with the choice in the moment. Whereas with regular use over extended periods of time they may be able to develop a stronger preference and be able to pick out differences explaining that preference.

As a sort of proof of concept, aurally, I've noticed that there are several popular songs which have very distinctive audio glitches in them (typically high pitched beeps). The thing is, these glitches are not noticeable by most people when casually listening (testified by the fact that the songs are out in public release with such glitches) but once you notice them you begin to notice them every time, and it detracts from the enjoyment of the song. Similar effects on a subtler level surely exist for almost all music, so I wouldn't take the idea that listening to two separate samples of music and being forced to express a preference in the moment is at all a reasonable measurement except in more pronounced cases.

[+] Zigurd|9 years ago|reply
I'm very sceptical of experiment design that relies on self-reporting. It is super-easy to get people to accept amazingly compromised compressed versions of original visual and audible information.

But what is that measuring? That we can't sense the difference? That information from our senses is lost when encoding it in our nervous system? That our conscious minds cannot introspect or report the difference? We know our conscious minds are unreliable reporters of what our subconscious is doing or deciding. On the other hand it is easy to show that the scene we think we are seeing is created in the mind. It is easy to trick a person into thinking identical shading is different, and different shading identical. What does that actually tell us about how to compress video?

I would prefer to go with physiology. If our sense organs can resolve it, and the nervous system can delivery the sensory information, the information is important, no matter what words our unreliable consciousness says.

[+] stevefeinstein|9 years ago|reply
Is it not possible to just mechanically test the sound of each instrument in a sound proof chamber to identify the unique characteristics of each?

There is no better, there is only what's so. Measure that and if YOU like one better, make another one that produces the same sound.

Probably not as easy as it sounds (pun intended).

[+] sandGorgon|9 years ago|reply
this is the actual PDF to the study - http://members.home.nl/p.brandt/2014%20PNAS%20Fritz%20six%20...

Interestingly, not even the maker of the top instrument (which garnered 26 points against the Stradivarius) knows about the result. It was deliberately kept hidden.

I really hope they let the world know - something this good needs to be celebrated and rewarded. It is unfair otherwise.

[+] continuations|9 years ago|reply
So a Stradivarius is the 17th century equivalent of $2000 per feet gold plated "audiophile" speaker cables?
[+] huxley|9 years ago|reply
No, the test doesn't show that Stradivarius violins are junk but rather that the best violins by modern violin makers are able to challenge and surpass Stradivarius violins.
[+] ceejayoz|9 years ago|reply
No, a Strad is like finding a pair of speakers from the 1950s that are just as good as the best modern audiophile ones.
[+] Shivetya|9 years ago|reply
No, I would instead state is as modern expensive violins are of the same quality if not better. However you first have to remove the cues players have in identifying the violin they are using to remove inherent bias